Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

zipimport.zipimporter breaks for zips containing files with size > 0xFFFFFFFF #118107

Closed
jsirois opened this issue Apr 19, 2024 · 2 comments
Closed
Labels
3.13 bugs and security fixes release-blocker type-bug An unexpected behavior, bug, or error

Comments

@jsirois
Copy link
Contributor

jsirois commented Apr 19, 2024

Bug report

Bug description:

The following test in the Pex project fails under CPython 3.13.0a6:
https://github.com/pex-tool/pex/blob/27c2db2bf26039bef41323c964bc4e0317a7b4f5/tests/test_pex_builder.py#L520-L547

The failure includes stderr output from zipimporter like:

Failed checking if argv[0] is an import path entry
Traceback (most recent call last):
  File "<frozen zipimport>", line 98, in __init__
  File "<frozen zipimport>", line 520, in _read_directory
NameError: name 'struct' is not defined. Did you forget to import 'struct'?
  File "/tmp/pytest-of-jsirois/pytest-14/test_check0/too-big.pyz", line 1
    PK-
SyntaxError: source code cannot contain null bytes

Inspection of #94146 indicates the code added to handle too large files is missing an import: https://github.com/python/cpython/pull/94146/files#r1572552939 The tests added in that PR do stress too many files for a 32 bit zip, but they do not stress too big a file for a 32 bit zip; so it makes sense that this import omission slipped past.

CPython versions tested on:

3.13, CPython main branch

Operating systems tested on:

Linux

Linked PRs

@jsirois jsirois added the type-bug An unexpected behavior, bug, or error label Apr 19, 2024
facebook-github-bot pushed a commit to facebookincubator/cinder that referenced this issue Apr 19, 2024
Summary: see python/cpython#118107

Reviewed By: bowiechen

Differential Revision: D56360822

fbshipit-source-id: 20f2d27f99ad0999a6b98c730a67aeac11c568eb
@gpshead gpshead added the 3.13 bugs and security fixes label Apr 22, 2024
@encukou
Copy link
Member

encukou commented May 3, 2024

Marking as potential release blocker.

gpshead pushed a commit to gpshead/cpython that referenced this issue May 4, 2024
The code that handles too large files and offsets was missing an import.
encukou pushed a commit that referenced this issue May 7, 2024
Add missing import to code that handles too large files and offsets.
Use list, not tuple, for a mutable sequence.

Add tests to prevent similar mistakes.

---------

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
@encukou
Copy link
Member

encukou commented May 7, 2024

Thank you for the report and fix!

@encukou encukou closed this as completed May 7, 2024
SonicField pushed a commit to SonicField/cpython that referenced this issue May 8, 2024
Add missing import to code that handles too large files and offsets.
Use list, not tuple, for a mutable sequence.

Add tests to prevent similar mistakes.

---------

Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.13 bugs and security fixes release-blocker type-bug An unexpected behavior, bug, or error
Projects
Development

No branches or pull requests

3 participants