pip fails to parse install_requires containing local wheel with spaces in path

[basnijholt]

Description

pip fails to install a package when its setup.py's install_requires contains a local wheel file path that includes spaces. The error indicates that pip is unable to correctly parse the requirement string, specifically it has trouble with the "Expected end or semicolon (after URL and whitespace)" part.

I run into this problem in the unidep package (basnijholt/unidep#211).

Expected behavior

pip should correctly parse the install_requires and install the local wheel, even if its path contains spaces.

pip version

24.3.1

Python version

3.13.1

OS

macos and ubuntu verified

How to Reproduce

1. Setup:

   # Create a directory with a space
   mkdir "my folder"
   
   # Download a wheel into that directory
   wget -O "my folder/pipefunc-0.46.0-py3-none-any.whl" https://files.pythonhosted.org/packages/67/d2/3eb2021d2e0329c96c92f78e98c92bc9dbdfa94b647ddbf9f625567aa8db/pipefunc-0.46.0-py3-none-any.whl
   
   # Create a minimal setup.py
   cat <<EOF > setup.py
   from setuptools import setup
   
   setup(
       name="my-local-package",
       version="0.1.0",
       install_requires=[
           "pipefunc-0.46.0-py3-none-any.whl @ file://$(pwd)/my folder/pipefunc-0.46.0-py3-none-any.whl"
       ],
       py_modules=["my_module"],
   )
   EOF
   
   # Create an empty module file
   touch my_module.py

2. Command:

   pip install --verbose .

Output

pip fails with the following error:

at 17:39:19 ❯ pip install --verbose  .
Using pip 24.3.1 from /Users/basnijholt/micromamba/envs/unidep/lib/python3.13/site-packages/pip (python 3.13)
Processing /Users/basnijholt/Code/unidep/tmp
  Running command python setup.py egg_info
  error in my-local-package setup command: 'install_requires' must be a string or iterable of strings containing valid project/version requirement specifiers; Expected end or semicolon (after URL and whitespace)
      pipefunc-0.46.0-py3-none-any.whl @ file:///Users/basnijholt/Code/unidep/tmp/my folder/pipefunc-0.46.0-py3-none-any.whl
                                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /Users/basnijholt/micromamba/envs/unidep/bin/python3.13 -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize

  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)

  __file__ = %r
  sys.argv[0] = __file__

  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"

  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/Users/basnijholt/Code/unidep/tmp/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base /private/tmp/pip-pip-egg-info-nxuxn1lh
  cwd: /Users/basnijholt/Code/unidep/tmp/
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[ichard26]

Two things:

• The error is being raised by the build backend (setuptools). pip is not at fault

• The error is legitimate. If I pass this requirement specifier to packaging, the reference parsing library used by pip and setuptools, it will error out.

Python 3.12.4 (main, Jun 21 2024, 18:39:32) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from packaging.requirements import Requirement
>>> Requirement("pipefunc-0.46.0-py3-none-any.whl @ file://$(pwd)/my folder/pipefunc-0.46.0-py3-none-any.whl")
Traceback (most recent call last):
  File "/home/ichard26/.local/lib/python3.12/site-packages/packaging/requirements.py", line 36, in __init__
    parsed = _parse_requirement(requirement_string)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ichard26/.local/lib/python3.12/site-packages/packaging/_parser.py", line 62, in parse_requirement
    return _parse_requirement(Tokenizer(source, rules=DEFAULT_RULES))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ichard26/.local/lib/python3.12/site-packages/packaging/_parser.py", line 80, in _parse_requirement
    url, specifier, marker = _parse_requirement_details(tokenizer)
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ichard26/.local/lib/python3.12/site-packages/packaging/_parser.py", line 113, in _parse_requirement_details
    marker = _parse_requirement_marker(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ichard26/.local/lib/python3.12/site-packages/packaging/_parser.py", line 145, in _parse_requirement_marker
    tokenizer.raise_syntax_error(
  File "/home/ichard26/.local/lib/python3.12/site-packages/packaging/_tokenizer.py", line 167, in raise_syntax_error
    raise ParserSyntaxError(
packaging._tokenizer.ParserSyntaxError: Expected end or semicolon (after URL and whitespace)
    pipefunc-0.46.0-py3-none-any.whl @ file://$(pwd)/my folder/pipefunc-0.46.0-py3-none-any.whl
                                       ~~~~~~~~~~~~~~~~~^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ichard26/.local/lib/python3.12/site-packages/packaging/requirements.py", line 38, in __init__
    raise InvalidRequirement(str(e)) from e
packaging.requirements.InvalidRequirement: Expected end or semicolon (after URL and whitespace)
    pipefunc-0.46.0-py3-none-any.whl @ file://$(pwd)/my folder/pipefunc-0.46.0-py3-none-any.whl

According to the dependency specifier specification, whitespace is used to denote the end of an URL specifier:

Non line-breaking whitespace is mostly optional with no semantic meaning. The sole exception is detecting the end of a URL requirement.

You will need to escape the space using percent-encoding.

pipefunc-0.46.0-py3-none-any.whl @ file://$(pwd)/my%20folder/pipefunc-0.46.0-py3-none-any.whl

I will note that the bare URL syntax is a pip extension. It's not part of the specification, and pip will handle unescaped spaces just fine (as long as it's passed as a single argument, aka with quotes).

[basnijholt]

Thank you so much @ichard26, using %20 was the missing piece for me.

[ichard26]

I just made an edit with a callout that passing the bare URL is a pip feature, not defined by a standard, so that's why you see differing behaviour w/ whitespace FYI.

I'm glad that I could help!