This repository was archived by the owner on Oct 3, 2026. It is now read-only.
Repository navigation
feat(snapshotter): quiesced hot-commit path + lock-gate contract tests - #47
Merged
Merged
Conversation
Pausing a VM (QMP stop) does not release QEMU's image lock, so a commit of a paused container's active snapshot hit the same lock gate as a running one (rwlayer.img is locked by the VM). The freeze that spinbox now does on pause makes the on-disk ext4 consistent, but the gate could not tell a frozen-paused VM from a live one. Add an opt-in bypass: when the Commit call carries the containerd.io/snapshot/erofs.quiesced label, getCommitUpperDir mounts the rwlayer via MountExt4(WithoutImageLock()), which skips the exclusive lock gate and mounts ro,norecovery (the paused VM still holds the image open, so no journal replay must be attempted). The label is the runtime's assertion that it has paused + frozen the guest; using it without a freeze risks a torn read, which the runtime owns. Tests (the contract was entirely untested end-to-end before): - locked image without the label: Commit fails with ErrFailedPrecondition and never falls back to the empty fs/ (no root needed; the OFD-lock conflict is detected before any mount syscall) - stopped container (no lock): reads the guest upper/, produces a valid non-empty EROFS layer - quiesced label + held lock: commit succeeds reading ro,norecovery, while the same held lock without the label is rejected - commitOptsQuiesced label parsing Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
The hot-commit primitive qemubox uses is rootfs.CreateDiff, which for an
active snapshot calls diff.Compare(View(parent), Mounts(active)) and
forwards its diff.Opts. Compare's active/ext4 mount paths
(withActiveSnapshotMount, withExt4UpperMount) take the same exclusive image
lock as the commit gate, so CreateDiff against a paused VM's snapshot hit
"rwlayer.img is locked by the VM" - the commit-side bypass added earlier
did not cover this path.
Compare now reads the quiesced label from its opts
(diff.WithLabels{containerd.io/snapshot/erofs.quiesced: "true"}), threads it
to the ext4 mount, and mounts via MountExt4(WithoutImageLock()) (ro,
norecovery). The label is request-scoped and consumed here - it is stripped
so it never persists on the produced layer blob. Without it, a held image
still fails loudly.
This is what actually lets a runtime build a new image from a paused+frozen
container via rootfs.CreateDiff.
Tests (root): withUpperMount over a lock-held ext4 fails with
ErrFailedPrecondition by default and succeeds with quiesced=true reading the
frozen upper; Compare consumes the quiesced label without persisting it on
the blob.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the snapshotter side of the hot-commit gap, and adds the lock-gate contract tests that were entirely missing.
The problem
Pausing a VM (QMP
stop) does not release QEMU's image lock — the lock lives with the open block backend, not vCPU execution. Soctr snapshots commit/nerdctl commiton a paused container's active snapshot hit the exact same gate as a running one:rwlayer.img is locked by the VM. spinbox's pause now does FIFREEZE before the QMP stop, so the on-disk ext4 is consistent — but our OFD-lock gate couldn't tell a frozen-paused VM from a live one and rejected the commit it was meant to enable.The fix (opt-in, runtime-driven)
A cooperating runtime that has paused + frozen the guest sets the
containerd.io/snapshot/erofs.quiescedlabel on the Commit call. The snapshotter then mounts the rwlayer viaMountExt4(WithoutImageLock()):ro,norecovery— no journal replay against an image the VM still holds open; the freeze already flushed it.The label is the runtime's assertion that it quiesced; using it without a real freeze risks a torn read, and that contract is owned by the runtime. Without the label, commit of a held image still fails loudly by design.
Threading:
Commitreads the label from the commit opts (commitOptsQuiesced) →commitBlock(quiesced)→getCommitUpperDir(quiesced)→MountExt4.MountExt4keeps its old 2-arg signature for all existing callers (variadicExt4Opt); only the quiesced commit passesWithoutImageLock().Tests (the contract had zero end-to-end coverage)
Commitfails withErrFailedPreconditionand never falls back to the emptyfs/. No root needed — the OFD-lock conflict is detected before any mount syscall.upper/, produces a valid non-empty EROFS layer (root + mkfs.ext4/erofs).ro,norecovery; the same held lock without the label is rejected (root + mkfs.ext4).commitOptsQuiescedlabel parsing (incl."false"and unrelated labels).All four pass as root in the privileged Docker image; the full Linux suite is green (3× to rule out the known loop-device overlay flake); lint clean on golangci-lint 2.11 and 2.12; non-Linux build OK.
Follow-up (spinbox side, separate repo)
The matching runtime change: on hot commit, qemubox pauses + freezes, then calls
SnapshotService.Commit(..., snapshots.WithLabels{quiescedLabel: "true"})directly (the active snapshot key forctr runis the container ID) instead of thenerdctl commitcold path. The thaw/resume on the other side completes the round-trip.🤖 Generated with Claude Code