Skip to content

docs: classify the fragment's non-Lance referenced files by format - #8562

Merged
Xuanwo merged 6 commits into
lance-format:mainfrom
LuciferYang:docs/fragment-referenced-lance-files-boundary
Aug 31, 2026
Merged

docs: classify the fragment's non-Lance referenced files by format#8562
Xuanwo merged 6 commits into
lance-format:mainfrom
LuciferYang:docs/fragment-referenced-lance-files-boundary

Conversation

@LuciferYang

Copy link
Copy Markdown
Contributor

Fragment::referenced_lance_files documents which of a fragment's referenced files it does not yield, and the list named two of the four kinds. A fragment can also reference external row-version metadata through created_at_version_meta and last_updated_at_version_meta, both RowDatasetVersionMeta::External(ExternalFile) just like the external row-id file.

Classify by format instead of enumerating, so the boundary stays correct as kinds are added: a deletion file is .arrow or .bin per DeletionFileType::suffix, and external row-id or row-version metadata is an ExternalFile, a (path, offset, size) byte range rather than a Lance file.

This matters because the omission reads as a statement about what is referenced at all. Dataset::referenced_files in #8097 refuses outright on each of those three external kinds, precisely because they are referenced but not enumerated by any per-file walk; a reader of this doc could reasonably conclude row-version metadata is not a referenced file.

Docs only, no code change. cargo fmt --all --check, cargo clippy -p lance-table --all-targets -- -D warnings, and RUSTDOCFLAGS="-D warnings" cargo doc -p lance-table --no-deps are clean; lance-table format tests pass (32).

The list named two of the four kinds. A fragment can also reference
external row-version metadata through created_at_version_meta and
last_updated_at_version_meta, both raw byte ranges like the external
row-id file. Classifying by format instead of enumerating keeps the
boundary correct as kinds are added.
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Aug 17, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 17, 2026
@lance-gatekeeper lance-gatekeeper Bot removed the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 26, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 26, 2026
@lance-gatekeeper lance-gatekeeper Bot removed the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 26, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 26, 2026
@lance-gatekeeper lance-gatekeeper Bot removed the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 27, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 27, 2026
@lance-gatekeeper lance-gatekeeper Bot removed the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 31, 2026
lance-gatekeeper[bot]

This comment was marked as outdated.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 31, 2026
@lance-gatekeeper lance-gatekeeper Bot removed the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 31, 2026

@lance-gatekeeper lance-gatekeeper Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gate recommendation: approve.

The latest merge from main leaves the documentation patch unchanged. The storage-based classification still matches the current deletion-file suffixes and external row-id and row-version byte-range representations without changing behavior.

@lance-gatekeeper lance-gatekeeper Bot added the K-approved Latest Gatekeeper recommendation permits acceptance. label Aug 31, 2026
@Xuanwo
Xuanwo merged commit 9452040 into lance-format:main Aug 31, 2026
38 checks passed
@LuciferYang

Copy link
Copy Markdown
Contributor Author

Thank you @Xuanwo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation K-approved Latest Gatekeeper recommendation permits acceptance.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants