Reduce tar extraction overhead and improve async archive creation

[JeremyKuhne]

Reduce filesystem calls and per-entry allocations during tar extraction, and batch writes during file-backed archive creation. No public API is added.

Changes

• Restore file timestamps through the existing output handle, flush asynchronously with cancellation, and right-size tiny-file buffers without disabling buffering.
• Use span-based path walking and reuse ordinary extraction-root resolution on Windows and Unix. For an empty ordinary destination, skip descendant-link probes until a symbolic-link entry is encountered; disable the shortcut before validating that entry. Nonempty destinations and linked roots retain descendant checks.
• Use a 64 KiB buffer for owned archive output and sequential-read hints for Windows source files. Caller-provided streams are unchanged.

Performance

BenchmarkDotNet Release API measurements on Windows 11 / Ryzen 7 9700X, with the same 12.0.0-dev runtime before and after. Windows extraction compares the original implementation with this update; creation measurements cover the unchanged final creation implementation. Paired cells are sync / async. Allocation is cumulative managed memory per archive, not peak memory; negative percentages mean improvement.

Real archives

The workloads extract the .NET 11 RC SDK (~726 MB payload) and Runtime (~87 MB) tar/tar.gz archives, and repack the same trees as uncompressed tar.

Scenario	Before ms	After ms	Time change	Before allocated MiB	After allocated MiB	Allocation change
SDK tar extraction	4,732.22 / 4,890.40	2,719.13 / 2,928.43	-42.5% / -40.1%	68.58 / 73.33	17.82 / 22.36	-74.0% / -69.5%
SDK tar.gz extraction	5,348.27 / 5,710.17	3,651.80 / 3,895.51	-31.7% / -31.8%	73.40 / 85.64	22.67 / 31.39	-69.1% / -63.3%
Runtime tar extraction	168.88 / 166.04	98.76 / 129.75	-41.5% / -21.9%	1.67 / 1.91	0.47 / 0.73	-71.9% / -61.9%
Runtime tar.gz extraction	296.93 / 321.80	245.12 / 270.64	-17.4% / -15.9%	1.93 / 3.33	0.73 / 2.13	-62.1% / -36.0%
SDK tar creation	514.71 / 805.56	532.43 / 774.07	+3.4% / -3.9%	12.339 / 18.595	12.397 / 18.445	+0.5% / -0.8%
Runtime tar creation	36.92 / 57.84	35.93 / 52.49	-2.7% / -9.2%	0.396 / 0.676	0.455 / 0.726	+14.8% / +7.5%

SDK tar extraction saves about two seconds per archive: 40-43% less time and 70-74% less allocation. Gzip extraction is approximately 32% faster.

Synthetic workloads

Small contains 32 tiny files, Tiny contains 4,096 256-byte files, Mixed contains 256 files totaling ~17 MiB, and Large contains four 16 MiB files.

Scenario / operation	Before ms	After ms	Time change	Before allocated KiB	After allocated KiB	Allocation change
Small / extract	22.32 / 22.69	12.61 / 14.04	-43.5% / -38.1%	412.37 / 446.04	107.89 / 134.88	-73.8% / -69.8%
Tiny / extract	2,584.33 / 2,599.62	1,488.62 / 1,607.03	-42.4% / -38.2%	48,442.99 / 52,591.72	11,492.77 / 14,367.03	-76.3% / -72.7%
Mixed / extract	175.75 / 175.66	99.89 / 127.46	-43.2% / -27.4%	2,772.90 / 3,036.30	780.92 / 1,019.31	-71.8% / -66.4%
Large / extract	28.55 / 32.00	23.12 / 31.19	-19.0% / -2.5%	32.52 / 70.04	16.58 / 53.80	-49.0% / -23.2%
Small / create	2.27 / 3.22	2.06 / 3.14	-9.0% / -2.6%	65.22 / 106.77	125.22 / 165.84	+92.0% / +55.3%
Tiny / create	155.06 / 276.39	148.84 / 241.71	-4.0% / -12.5%	6,870.59 / 11,960.01	6,930.59 / 11,927.19	+0.9% / -0.3%
Mixed / create	17.82 / 29.35	17.43 / 25.25	-2.1% / -14.0%	461.72 / 786.82	521.72 / 839.63	+13.0% / +6.7%
Large / create	27.86 / 37.33	26.75 / 38.99	-4.0% / +4.4%	11.66 / 23.84	71.66 / 82.29	+514.7% / +245.1%

Creation benefits are primarily asynchronous. The larger owned output buffer adds 60 KiB per synchronous archive.

Linux SDK tar

Full before/after on WSL2 Ubuntu 26.04 using its native ext4 filesystem: the original implementation before this PR versus the complete current PR, with the same Release 12.0.0-dev runtime and SDK tar archive. Each cell is sync / async, matching the Windows tables.

Scenario	Before ms	After ms	Time change	Before allocated MiB	After allocated MiB	Allocation change
SDK tar extraction	2,561.52 / 2,918.09	2,221.94 / 2,613.40	-13.3% / -10.4%	76.07 / 78.94	16.44 / 18.97	-78.4% / -76.0%

SDK tar extraction saves 340 ms sync / 305 ms async, with 76-78% less allocation.

Validation and scope

• Analyzer-enabled Release product builds pass for all three library TFMs on Windows and Linux.
• 7,028 Windows tests pass. Linux: 6,335 pass, eight privileged-only cases skipped; a separate privileged run passes 54 expanded cases.
• New coverage includes late symbolic links, existing directory links, root/ancestor retargeting, missing destinations, enumeration-permission fallback, and cancellation after copying. The three cancellation cases fail against the unfixed PR binary and pass with token forwarding.
• Independent archive checks cover contents, file timestamps, populations, and hard-link identities. macOS execution was not tested.
• The empty-destination shortcut assumes the destination namespace is not modified externally during extraction; it is not handle-anchored race protection.

Passed on Windows:

.\.dotnet\dotnet.exe build src\libraries\System.Formats.Tar\src\System.Formats.Tar.csproj -c Release /p:RunAnalyzersInBuild=true
.\.dotnet\dotnet.exe build src\libraries\System.Formats.Tar\tests\System.Formats.Tar.Tests.csproj -c Release /t:Test /p:Outerloop=true /p:RunAnalyzersInBuild=true

The corresponding .dotnet/dotnet build commands also pass in WSL with Linux paths and the same properties.

Related to dotnet/runtime#135128.

Note

This description was generated with GitHub Copilot.

[azure-pipelines]

Azure Pipelines:
Successfully started running 3 pipeline(s).
13 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-formats-tar
See info in area-owners.md if you want to be subscribed.

[Copilot]

🔵 Needs a closer look

Security-sensitive cross-platform path handling changed substantially, and macOS execution was not validated.

0 open findings

What changed in this PR

Optimizes tar extraction and file-backed archive creation while preserving extraction security and metadata behavior.

Changes:

• Reduces path-resolution calls and per-entry allocations.
• Improves buffering, async flushing, timestamps, and sequential reads.
• Adds cross-platform regression and cancellation coverage.

File	Description
TarFile.ExtractToDirectory.Stream.Tests.cs	Covers empty roots and late links.
TarFile.ExtractToDirectory.File.Tests.Windows.cs	Tests Windows reparse-root handling.
TarFile.ExtractToDirectory.File.Tests.Unix.cs	Tests Unix links and permissions.
TarFile.ExtractToDirectory.File.Tests.cs	Adds root-retargeting test support.
TarEntry.ExtractToFile.Tests.cs	Tests cancellation and timestamps.
TarWriter.Windows.cs	Adds sequential-read hints.
TarFile.cs	Adds output buffering and extraction context.
TarEntry.Windows.cs	Optimizes Windows path resolution.
TarEntry.Unix.cs	Optimizes Unix path resolution.
TarEntry.cs	Reworks containment checks and file extraction.
System.Formats.Tar.csproj	Includes required Windows interop sources.

🧠 Review effort: Balanced