Repository navigation
XLSX: keep unmodelled parts on save (#330) - #333
Merged
Merged
Conversation
Pass-through of package content GridCraft does not model: on read the raw parts are kept with their content types and relationship graph per owner (package root, workbook, sheet); on write they get fresh part names where needed and new relationship ids, and the r:ids in kept XML are rewritten. Kept: custom document properties, ribbon customisation, add-in task panes, custom XML, links to external workbooks with <externalReferences> (never followed), connections, sheet query tables, printer settings with the <pageSetup r:id> (storytold#324), form and ActiveX controls, OLE objects and sheet backgrounds with their VML shapes (merged with GridCraft's note shapes). Content whose graph reaches parts GridCraft rewrites is dropped with a warning; sheet content goes with its sheet and is not copied by Copy Sheet. Fixes storytold#324 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # ROADMAP.md # crates/model/src/lib.rs # crates/xlsx/src/sheet_read.rs # docs/file-format-parity.md # docs/gaps.md
…o the kept total Content types (and extensions) of kept parts come from the input file and were written into [Content_Types].xml unescaped, so a quote or angle bracket injected XML; they are now attribute-escaped. Each sheet re-read and kept its VML drawing (up to 16 MB) outside MAX_KEPT_TOTAL, so many sheets sharing one highly compressed drawing amplified memory; the drawing is now read within and charged to the remaining kept-bytes allowance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
|
Merged — thank you for the contribution! (Before merging I pushed two hardening fix-ups with tests: content types copied from the input file are now attribute-escaped in [Content_Types].xml, and kept VML drawings are charged to the MAX_KEPT_TOTAL budget.) If you haven't already, come join us on the ArtCraft Discord: https://discord.gg/artcraft |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #330 (slicers, timelines and the data model are not done, see below)
Fixes #324
Problem
Opening and saving an
.xlsxdropped every part GridCraft doesn't model: custom XML, links to external workbooks, data connections, query tables, printer settings (#324), form controls, OLE objects… (docs/gaps.md§3). The reader only warned (or said nothing) and the writer regenerates the package from the model.Design (
crates/xlsx/src/passthrough.rs)gridcraft_model::Preserved(newpreserved.rs): kept parts (name, content type, bytes, their own rels with ids unchanged), the owner's relationships to them (full type URI, target, external flag), raw owner XML fragments, and the sheet's VML drawing.Workbook.preservedholds package-root and workbook content,Sheet.preservedthe sheet's. Both are#[serde(skip)](not in the JSON format) andArced, so undo snapshots stay cheap.[Content_Types].xml, bytes read with limits). External targets are kept as text and never opened, followed or refreshed.custom-properties,extensibility(customUI),webextensiontaskpanes.customXml,connections, andexternalLinktogether with<externalReferences>. The links are kept all or nothing, because[1]in a formula is the position in that list. Formulas such as[1]Sheet1!$A$1were already kept as text, so they still resolve in Excel.queryTable(relationship only),printerSettings+ the<pageSetup r:id>(XLSX save drops printerSettings parts and the pageSetup r:id #324), and the raw<picture>,<oleObjects>and<controls>elements (also insidemc:AlternateContent). The worksheet root's namespace declarations are added to each fragment. The rels the fragments reference are kept too. Controls and OLE objects also keep the sheet's legacy VML drawing, minus its note shapes, plus the VML's own rels (preview images).customXml/,xl/externalLinks/,xl/embeddings/,xl/ctrlProps/,xl/printerSettings/…). Otherwise (e.g.xl/media/) they get a freshkept{N}name. Each gets a content-type override and its rels file with targets renamed (absolute targets). The owner relationships get new ids. Ther:ids in the kept fragments are rewritten with quick-xml (any attribute in the relationships namespace). The fragments are emitted in schema order:externalReferencesafter<sheets>;pageSetup r:id;picture/oleObjects/controlsafter<legacyDrawing>and before<tableParts>.o:idmap data="1,2", ids_x0000_s2049…) so they can't collide with the control/OLE shape ids that<control shapeId>/<oleObject shapeId>refer to. Without notes, the kept VML is written on its own. This is text-based because Excel's VML is often not well-formed XML (<br>).sheet.movewithcopydoes not copy it (one line inengine/src/cmd/sheet.rs).unwrap/indexing on file data.vbaProjectis in the rewritten/excluded list, so the two don't double-write). Feature/table slicers #193 models table slicers but doesn't write them to XLSX. Slicers aren't passed through here, so no double-write later.Tests (built in code, no Excel files)
crates/xlsx/src/tests/passthrough_tests.rs:unmodelled_parts_survive_a_round_trip: a package with every kept kind plus a note. After read → write, every kept part is byte-identical exactly once and the package is valid: every part has a content type, every internal target exists, no duplicate ids or overrides. The external path is unchanged and external.[1]formulas are unchanged. ThepageSetup,oleObject,objectPrandcontrolr:ids resolve to the right bytes. Element order is checked. The VML merge keeps the control shapes, replaces the old note and puts ours in block 2. Re-reading gives the same kept content.edited_workbooks_keep_consistent_relationships: inserts a sheet before, renames the sheet, removes its note and edits a cell. The rels are still consistent and the VML is written alone. Deleting the sheet drops its content and keeps the workbook-level content.printer_settings_and_page_setup_id_survive_save: the XLSX save drops printerSettings parts and the pageSetup r:id #324 repro (openpyxl-style package).damaged_kept_content_is_dropped_without_failing_the_read: missing target, a cyclic custom-XML pair (kept), kept content pointing at a worksheet, a link without a part (drops all links), damaged rels, a missing printer-settings part, a control without VML / with an unknown r:id.oversized_kept_parts_are_dropped: a part over the per-part cap.Commands run (in a worktree, per crate):
Not run: the full
cargo xtask ci(CI covers it).Not done
sheetIdand drawing graphic frames, all of which GridCraft regenerates.xl/model/item.data, Power Pivot) and Power Query output stored as table query tables (thequeryTablehangs off the table part and needstableType="queryTable"/queryTableFieldIdon the table).extLstentries in sheet and workbook XML,legacyDrawingHF(header/footer pictures), chart details.Unverified assumptions (needs confirmation in Excel / corpus)
/xl/...) for kept parts. openpyxl writes those and Excel opens them, but this wasn't checked here.[Content_Types].xmlas overrides. Names were not checked against Excel output.🤖 Generated with Claude Code