Import errors with loo long entry in sys.path or PYTHONPATH in Windows

[devdanzin]

Bug report

Bug description:

Having too long entries in sys.path or PYTHONPATH in Windows will cause different errors in imports. This was first reported on #122170, where there is some useful discussion.

> $Env:PYTHONPATH="a" * 33000
> py -3.13
Exception ignored in running getpath:
Traceback (most recent call last):
  File "<frozen getpath>", line 668, in <module>
OSError: failed to make path absolute
Fatal Python error: error evaluating path
Python runtime state: core initialized

Current thread 0x00005a8c (most recent call first):
  <no Python frame>

>>> import sys
>>> sys.path.insert(0, "a" * 33000)
>>> import email
Traceback (most recent call last):
  File "<frozen importlib._bootstrap_external>", line 1512, in _path_importer_cache
KeyError: 'aaaaaaaaaaaaaaaaaaaaaaaa[...]aaaaaaaaaa'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<python-input-6>", line 1, in <module>
    import email
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1322, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1262, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1555, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1527, in _get_spec
  File "<frozen importlib._bootstrap_external>", line 1514, in _path_importer_cache
  File "<frozen importlib._bootstrap_external>", line 1490, in _path_hooks
  File "<frozen importlib._bootstrap_external>", line 1714, in path_hook_for_FileFinder
  File "<frozen importlib._bootstrap_external>", line 173, in _path_isdir
  File "<frozen importlib._bootstrap_external>", line 158, in _path_is_mode_type
  File "<frozen importlib._bootstrap_external>", line 152, in _path_stat
ValueError: stat: path too long for Windows

CPython versions tested on:

3.13, CPython main branch

Operating systems tested on:

Windows

Linked PRs

• gh-122353: Handle ValueError during imports #122389

[eryksun]

As discussed in #122228, on Windows going back to 3.3, the C path_t converter that's used by many builtin os functions is implemented to raise ValueError if a path is longer than 32767 characters. Whether or not that behavior gets changed, calling builtin os functions should still handle ValueError if one of the arguments is a path. For example, a path string in sys.path might contain null characters, or, on POSIX, it might not be encodable using the filesystem encoding (e.g. if the filesystem encoding is ASCII because locale coercion and UTF-8 mode are disabled).

[zooba]

I'm not convinced that these errors should be handled. OSErrors are caused by things outside of the control of the caller (assuming correct inputs), but ValueErrors indicate that the caller passed something invalid. A path longer than 32K is never valid, so it never should've gotten in there, so the caller should be informed - in this case, the user.

What's the real scenario where this mattered? All I see is a very contrived scenario to cause the error, but no information about how it was encountered or why simply ignoring corrupt data would've been a better result.

[serhiy-storchaka]

It is possible that the path to the .py file is below the 32K limit, but the path of the corresponding .pyc file exceeds it. Impossibility to create a .pyc is not fatal, this error should be silently ignored.

[zooba]

Impossibility to create a .pyc is not fatal, this error should be silently ignored.

Agreed.

But again, let's take each case like this individually - not being able to iterate over files in an invalid sys.path entry is fatal in other ways already, so it should be fatal in this way too.

[devdanzin]

I'm not convinced that these errors should be handled.

Me neither, I created these issues so informed decisions could be made on the need to guard against them. Thanks for analyzing
the reports!