Python "zipfile" can't detect "quoted-overlap" zipbomb that can be used as a DoS attack

Bug report

Bug description:

Just found this vulnerability in the latest Python 3.11.5 (and previous 3.10.10).

If we craft a zipbomb using the "quoted-overlap" way (as mentioned https://www.bamsoftware.com/hacks/zipbomb/), this can't be detected by Python's zip file and the zip will be extracted and thus potentially cause a DoS attack by consuming all the storage.

This issue is related to CVE-2019-9674 but not the same. CVE-2019-9674 is talking about the "normal" overlap-zipbomb which is a "full" overlap. This can already be detected by Python's new version of zipfile. However, when we craft a "quoted-overlap" zip, as indicated by https://www.bamsoftware.com/hacks/zipbomb/, python can't detect and happily starts to extract.

For example, the following is the python to extract a zip file, 116 KB before extraction, goes to as large as 17GB after extraction. The size after extraction can be easily increased to multi TBs or even PBs by adjusting the zip-creation.

import zipfile
import sys
import os

def extract_zip(zip_path):
    """
    Extracts the contents of a ZIP file to the current directory.

    :param zip_path: Path to the ZIP file
    """
    if not os.path.exists(zip_path):
        print(f"Error: {zip_path} does not exist.")
        return

    with zipfile.ZipFile(zip_path, 'r') as zip_ref:
        zip_ref.extractall()
        print(f"Extracted contents of {zip_path} to the current directory.")

if __name__ == "__main__":
    if len(sys.argv) != 2:
        print("Usage: python extract_zip.py <path_to_zip_file>")
        sys.exit(1)

    zip_file_path = sys.argv[1]
    extract_zip(zip_file_path)

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Linked PRs

Additional context: This was reported to us on the Python Security Response Team on 2023-09-06. We ultimately concluded it should be handled in public. The form of zip file attack is not "new" (as evidence by that zipbomb link) and several other zip implementations have had to deal with this over the years. Ours still needs to.

I'll summarize more of our discussions there in a followup comment.

@SethMichaelLarson

While we do at least have an existing warning on ZipFile.extractall() in our docs "Warning: Never extract archives from untrusted sources without prior inspection." - that text goes on to focus on other reasons. We expect that there are people who don't actually understand the full implications of handling untrusted compressed complex-referential data like zips and just call an API like .extractall() without sufficient checks regardless.

@SethMichaelLarson & @di looked into PyPI's warehouse to check how PyPI handles this situation with regards to uploaded packages. It already has a decent seeming zip file checking loop in place that rejects zip files who's extracted size would be huge compared to the input as seen in pypi/warehouse#13877 - that seems to reject the example zipbombs already. @pradyunsg later confirmed that while pip itself doesn't do such checks (it probably should) "we do benefit from PyPI's checks in the default configuration and those benefits transfer to mirrored PyPI indexes as well." - so in effect we don't currently believe PyPI can host zipbomb wheels - ameliorating this for the community's pip install invocations by default.

Workarounds & Recommendations

Document the workaround

We should centralize our guidance for Python users on how to avoid the problem when processing untrusted zip files, perhaps with a code snippet such as the overall extraction size checking loop from Warehouse as an example of the kinds of protection users of zipfile today should employ. Having applications and libraries do that is good defense in depth and doesn't require a patched Python runtime.

We also recommend that code handling untrusted zip files always run in a resource constrained container regardless.

Fixing

Improving our zipfile internals to do consistency checks to raise an exception when processing zip files with overlapping sections or inconsistent metadata or whatnot (all of the kinds of things that enable this form of malicious structured file).

How feasible it'll be to backport that is unknown. But as this is only a DoS, for something that most code isn't doing (processing untrusted zips), and it can be worked around, I don't think focusing on designing this backporting this all the way back to 3.8 is worth spending extra time on if it requires any.

Reaching out to specific applications that process arbitrary zip files to make sure they're aware of this issue is a good idea.

pythongh-109858: Protect zipfile from "quoted-overlap" zipbomb

No. All necessary backports were merged. Thank you for the reminder.

pythongh-109858: Protect zipfile from "quoted-overlap" zipbomb (pytho…

Add @requires_zlib() decorator for pythongh-109858 tests (pythonGH-11…

Does anyone here have any contacts in GitHub who know about GHSA?

Their entry for this (CVE-2024-0450) GHSA-jm46-725r-hh9v says that 3.12.2 and 3.11.8 are vulnerable, but AFAICS those 2 versions contain the fix (see gh-109858 mentioned in 3.12.2 release notes and 3.11.8 release notes).

I've attempted to open an improvement PR here but I'm not sure if it will be accepted seeing as it says 'Unreviewed' and it seems python is not on the list of things they care about; because the "suggest improvements" button (via which I did it) forced me to select "Affected products" where [c]python was not an option 😢 the nearest thing was 'pip'.

AFAICS this GHSA data feeds through into tools like Grype which currently say "Fixed?"=unknown and have no "Fixed in" so it would be good to know how to correct the record on which python versions are vulnerable/fixed.

@sparrowt

Thanks for reporting this @sparrowt, I'll take a look and update the CVE records accordingly. In theory this will trickle down into GitHub Security Advisories as well since they source a lot of their information from that database.

Amazing thank you! I'm not sure how the affected versions / patched versions thing is going to work given GHSA seems to only support packages within certain 'ecosystems' but seemingly not the ecosystem runtime (e.g. python) itself:

I've fixed the metadata for the CVE. I believe the "syncing" process will happen outside of the manual update process but could be wrong? The GHSA team is aware of the whole ecosystem issue for updating GHSA records.

[3.8] pythongh-109858: Protect zipfile from "quoted-overlap" zipbomb (p…

Thanks! Sadly it's still wrong on GHSA-jm46-725r-hh9v

[CVE-2024-0450] Protect zipfile from "quoted-overlap" zipbomb

pythongh-109858: Protect zipfile from "quoted-overlap" zipbomb (pytho…

Add @requires_zlib() decorator for pythongh-109858 tests (pythonGH-11…

Navigation Menu

Repository navigation

Python "zipfile" can't detect "quoted-overlap" zipbomb that can be used as a DoS attack #109858

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Workarounds & Recommendations

Document the workaround

Fixing

32 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Python "zipfile" can't detect "quoted-overlap" zipbomb that can be used as a DoS attack #109858

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

gpshead commented on Sep 25, 2023

gpshead commented on Sep 25, 2023

Workarounds & Recommendations

Document the workaround

Fixing

32 remaining items

serhiy-storchaka commented on Feb 5, 2024

sparrowt commented on Apr 3, 2024

sethmlarson commented on Apr 3, 2024

sparrowt commented on Apr 3, 2024

sethmlarson commented on Apr 3, 2024

sparrowt commented on May 15, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions