This repository was archived by the owner on Oct 3, 2026. It is now read-only.
Repository navigation
fix(snapshotter): atomic + complete layer manifest - #45
Merged
Merged
Conversation
writeLayerManifest had two independent defects: 1. Non-atomic write: os.WriteFile could leave a truncated manifest on a crash mid-write, unlike the fsmeta/VMDK write path which already uses temp + rename. 2. Lost positional alignment: the digest-only format silently dropped fallback blobs (snapshot-<id>.erofs, no content digest), so a fallback layer in the chain produced fewer lines than the device= options and desynced every layer after it. The manifest is the authoritative source for VMDK layer-order verification, so it must be 1:1 with device=. Changes: - New atomicWriteFile helper (temp in same dir + fsync + rename, temp cleaned up on any error). writeLayerManifest now uses it. - writeLayerManifest emits one line per layer in oldest-first order: "sha256:<hex>" for content-addressed blobs, "blob:<basename>" for fallback blobs. Line count now equals the device= count so consumers detect truncation/desync instead of misaligning silently. - Tests for both writeLayerManifest (digest-only, interleaved fallback, empty, no leftover temp) and atomicWriteFile. - readLayersManifest (integration) is forward-compatible: it already filters by the sha256: prefix and skips blob: lines; only its doc comment is updated to document the new format. Note: qemubox's manifest reader should be updated to recognize blob: lines to validate completeness by count; today it ignores them without breaking. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
checkNoLoopDevice did a single immediate FindByBackingFile check after the temp mount cleanup returned. The kernel releases loop devices asynchronously after LOOP_CLR_FD / autoclear, so the backing file can still resolve to a device for a short window, causing flaky failures: loop device /dev/loop1 still attached to .../layer.erofs (seen in TestWithStackedErofsTempMount and TestWithActiveSnapshotMountOverlay) Poll with a 5s deadline, matching the existing mountutils.verifyNoLoopDevice and loop.verifyDetached helpers, which already handle the same race. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
writeLayerManifesthad two independent defects:Non-atomic write —
os.WriteFilecould leave a truncatedlayers.manifeston a crash mid-write, unlike thefsmeta.erofs/merged.vmdkwrite path which already uses temp + rename.Lost positional alignment — the digest-only format silently dropped fallback blobs (
snapshot-<id>.erofs, which have no content digest). A fallback layer in the chain produced fewer lines than thedevice=options (mountFsMetaemits one per layer, never skipping), desyncing every layer after it. The manifest is documented as the authoritative source for VMDK layer-order verification, so it must be positionally 1:1 withdevice=.Changes
New
atomicWriteFilehelper: temp file in the same dir →fsync→rename, with the temp removed on any error path.writeLayerManifestnow uses it.writeLayerManifestemits one line per layer in oldest-first (VMDK/device=) order:sha256:<hex>for content-addressed blobsblob:<basename>for fallback blobs with no content digestLine count now equals the
device=count, so consumers detect truncation/desync instead of misaligning silently.Tests for
writeLayerManifest(digest-only, interleaved fallback preserves position, empty, no leftover temp) andatomicWriteFile(content/perm, overwrite without leftover temp, error on missing dir).readLayersManifest(integration test) is forward-compatible — it already filters by thesha256:prefix and skipsblob:lines; only its doc comment is updated to document the new format.Also: flaky loop-detach assertion (was #46)
checkNoLoopDevice(differroot tests) did a single immediateFindByBackingFilecheck after cleanup. The kernel releases loop devices asynchronously afterLOOP_CLR_FD/ autoclear, causing flakyloop device ... still attachedfailures inTestWithStackedErofsTempMountandTestWithActiveSnapshotMountOverlay. Now polls with a 5s deadline, matching the existingmountutils.verifyNoLoopDevice/loop.verifyDetachedhelpers. Test-only.Follow-up (outside this repo)
qemubox's manifest reader should be updated to recognize
blob:lines to validate completeness by count. Today it would ignore them without breaking.Testing
task fmt && task lint && task test— all green (lint 0 issues).GOOS=linux go vet ./internal/differ/— clean (the differ test fix only compiles under linux).🤖 Generated with Claude Code