Skip to content

extractor: add a total size limit across the whole extraction - #16

Merged
unxed merged 1 commit into
mainfrom
lunobot/agent-lb3-zip56/lunobot-3/5-extractor-total-size-limit
Sep 27, 2026
Merged

unxed merged 1 commit into
mainfrom
lunobot/agent-lb3-zip56/lunobot-3/5-extractor-total-size-limit

Conversation

@unxed

@unxed unxed commented Sep 27, 2026

Copy link
Copy Markdown
Owner

Summary

Extractor already bounds how big any one extracted file may be
(WithExtractorMaxFileSize, 1GB default) and how far any one entry may
expand relative to its own compressed size (WithExtractorMaxRatio,
4000:1 default), but nothing bounded the sum of what an extraction
writes. An archive of many entries, each individually within both of
those limits, could still expand to a total bounded only by the
archive's own size times the ratio limit: a 100 MB archive could write
hundreds of gigabytes to disk without a single error.

  • Add WithExtractorMaxTotalSize, a new limit checked against the
    extraction's own running total (the same counter Written() already
    reports), with a 32GB hardcoded default alongside the existing two.
    Zero disables it; a negative value is refused, same as the other two
    limits.
  • An extraction that would write past it fails with an error wrapping
    the new ErrTotalSizeLimit.
  • Propagated to the inner extractor solid archives build, the same way
    the other two limits already are.
  • The solid-fallback scratch copy is deliberately excluded from the
    total, for the same reason the per-file limit already excludes it:
    those bytes are removed again before the extraction returns and are
    not part of its output.

Test plan

  • go build ./... && go vet ./...
  • go test ./... (full suite, unchanged behavior for existing
    archives)
  • New TestExtractor_TotalSizeLimit / TestExtractor_TotalSizeLimit_Disabled
    cover an archive of ten entries that each stay under the per-file and
    ratio limits but whose sum trips (and, with the limit at zero,
    doesn't trip) the new total limit.
  • TestExtractorOptions_RejectNegativeLimits extended to cover a
    negative WithExtractorMaxTotalSize.

Touch #5

Lunobot-3

🤖 Generated with Claude Code

WithExtractorMaxFileSize and WithExtractorMaxRatio each look at one
entry at a time, so an archive of many entries that each stay under
both limits could still expand to an unbounded total: a 100 MB archive
of many small, individually-innocent records could write hundreds of
gigabytes to disk without tripping either check.

Add WithExtractorMaxTotalSize, checked against the extraction's own
running total (the same counter Written() already reports), with a
32 GB default alongside the existing hardcoded defaults. Zero disables
it, a negative value is refused, matching the other two limits. The
solid-archive scratch copy is deliberately excluded, same as the
existing per-file limit: those bytes are removed again before the
extraction returns and are not its output.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@unxed
unxed merged commit 7c66894 into main Sep 27, 2026
37 of 38 checks passed
@unxed
unxed deleted the lunobot/agent-lb3-zip56/lunobot-3/5-extractor-total-size-limit branch September 27, 2026 04:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant