Pipfile package specifier for different platform not honored

[jtmoon79]

• Be sure to check the existing issues (both open and closed!), and make sure you are running the latest version of Pipenv.
• Check the diagnose documentation for common issues before posting! We may close your issue if it is very similar to one of them. Please be considerate, or be on your way.

Issue description

Specifier platform_system and sys_platform is not respected during pipenv update.

On Ubuntu 22, I have a Pipfile that has a package that is Windows only (py2exe). I add specifier platform_system = "== 'Windows'" (also tried sys_platform = "== 'win32'"). However, pipenv update fails with ERROR:pip.subprocessor:python setup.py egg_info exited with 1 (and then more errors). It appears to attempt to build or install the Windows only package py2exe.

Package py2exe will fail to install on a Linux system (expected)

$ python -m pip install py2exe
Collecting py2exe
  Using cached py2exe-0.12.0.2.tar.gz (124 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/tmp/pip-install-4_zxof8f/py2exe_0a5c891ec79746dfb9d0fecce1c30b33/setup.py", line 13, in <module>
          raise RuntimeError("This package requires Windows")
      RuntimeError: This package requires Windows
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Expected result

pipenv update will not try to download or build or install a package that has specifiers against it's use.

Actual result

pipenv update downloads and builds a package that it should not.

Steps to replicate

1. On a Linux system; I used Ubuntu 22, Python 3.10

   $ lsb_release  --description
   Description:    Ubuntu 22.04.2 LTS
   $ python --version
   Python 3.10.12
   $ python -c "import sys; print(sys.platform)"
   linux
   

2. create a scratch directory
3. Create Pipfile

   [[source]]
   name = "pypi"
   url = "https://pypi.org/simple"
   verify_ssl = true
   [packages]
   py2exe = { version = "*", sys_platform = "== 'win32'"}
   

   This Pipfile package declaration follows the example used in the current docs

   Here’s an example Pipfile, which will only install pywinusb on Windows systems
   ...
   pywinusb = {version = "*", sys_platform = "== 'win32'"}

4. create a virtual environment, source it, update common install+build tools

   /usr/bin/python -m venv --copies .venv
   source ./.venv/bin/activate
   python -m pip install --upgrade setuptools pip wheel
   

   For me, this installed setuptools version 68.2.2, pip version 23.3.1, wheel version 0.41.3
5. install latest pipenv

   python -m pip install pipenv
   

   For me, this installed 2023.10.24
6. run pipenv update

$ python -m pipenv update
Running $ pipenv lock then $ pipenv sync.
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✘ Locking Failed!
⠦ Locking...False
ERROR:pip.subprocessor:python setup.py egg_info exited with 1
[ResolutionFailure]:   File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/resolver.py", line 645, in _main
[ResolutionFailure]:       resolve_packages(
[ResolutionFailure]:   File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/resolver.py", line 612, in resolve_packages
[ResolutionFailure]:       results, resolver = resolve(
[ResolutionFailure]:   File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/resolver.py", line 592, in resolve
[ResolutionFailure]:       return resolve_deps(
[ResolutionFailure]:   File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 908, in resolve_deps
[ResolutionFailure]:       results, hashes, internal_resolver = actually_resolve_deps(
[ResolutionFailure]:   File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 681, in actually_resolve_deps
[ResolutionFailure]:       resolver.resolve()
[ResolutionFailure]:   File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 442, in resolve
[ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv run pip install <requirement_name> to bypass this mechanism, then run $ pipenv graph to inspect the versions actually installed in the virtualenv.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: metadata generation failed

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/__main__.py", line 4, in <module>
    cli()
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/vendor/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/vendor/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/vendor/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/cli/command.py", line 546, in update
    do_update(
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/routines/update.py", line 59, in do_update
    do_lock(
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/routines/lock.py", line 65, in do_lock
    venv_resolve_deps(
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 849, in venv_resolve_deps
    c = resolve(cmd, st, project=project)
  File "/tmp/a/.venv/lib/python3.10/site-packages/pipenv/utils/resolver.py", line 718, in resolve
    raise RuntimeError("Failed to lock Pipfile.lock!")
RuntimeError: Failed to lock Pipfile.lock!

8. in Pipfile, comment the line with the package py2exe (so the line is ignored) then pipenv update will succeed

   $ python -m pipenv update
   Running $ pipenv lock then $ pipenv sync.
   Locking [packages] dependencies...
   Locking [dev-packages] dependencies...
   Updated Pipfile.lock (ebffa69a1fa192d1cef7cb42ad79231ca976565c5ce371a70160b3048d3cbc06)!
   Installing dependencies from Pipfile.lock (3cbc06)...
   All dependencies are now up-to-date!
   

I only tested this apparent bug with the following specifiers:

• py2exe = { version = "*", platform_system = "== 'Windows'"}
• py2exe = { version = "*", sys_platform = "== 'win32'"}

---

$ pipenv --support

Pipenv version: '2023.10.24'

Pipenv location: '/tmp/a/.venv/lib/python3.10/site-packages/pipenv'

Python location: '/tmp/a/.venv/bin/python'

OS Name: 'posix'

User pip version: '23.3.1'

user Python installations found:

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.10.12',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.15.90.1-microsoft-standard-WSL2',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP Fri Jan 27 02:56:13 UTC 2023',
 'python_full_version': '3.10.12',
 'python_version': '3.10',
 'sys_platform': 'linux'}

System environment variables:

• SHELL
• WSL2_GUI_APPS_ENABLED
• WSL_DISTRO_NAME
• WT_SESSION
• PIPENV_ACTIVE
• NAME
• PWD
• LOGNAME
• PIP_PYTHON_PATH
• HOME
• LANG
• WSL_INTEROP
• LS_COLORS
• VIRTUAL_ENV
• WAYLAND_DISPLAY
• LESSCLOSE
• TERM
• LESSOPEN
• USER
• PIP_DISABLE_PIP_VERSION_CHECK
• DISPLAY
• SHLVL
• VIRTUAL_ENV_PROMPT
• PYTHONDONTWRITEBYTECODE
• XDG_RUNTIME_DIR
• PS1
• WSLENV
• LOCALE
• LC_ALL
• GCC_COLORS
• XDG_DATA_DIRS
• PATH
• HISTFILESIZE
• HOSTTYPE
• PULSE_SERVER
• WT_PROFILE_ID
• SSH_AUTH_SOCKD
• OLDPWD
• _
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

• PIPENV_ACTIVE: 1

Debug–specific environment variables:

• PATH: ...
• SHELL: /bin/bash
• LANG: en_US.UTF-8
• PWD: /tmp/a
• VIRTUAL_ENV: /tmp/a/.venv

---

Contents of Pipfile ('/tmp/a/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[packages]
#py2exe = { version = "*", platform_system = "== 'Windows'"}

Contents of Pipfile.lock ('/tmp/a/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "ebffa69a1fa192d1cef7cb42ad79231ca976565c5ce371a70160b3048d3cbc06"
        },
        "pipfile-spec": 6,
        "requires": {},
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {},
    "develop": {}
}

[matteius]

Its a recent change to sys_markers and os_markers to fold them, but the sdist has to be able to build on the locking platform -- if it cannot be built you can translate those markers directly to the markers key and it will skip trying to install/fold the sub-dependencies of that package.

[matteius]

Sorry -- wrong thread on that last message. (so i deleted it) the message from 3 hours ago is relevant though.

[jtmoon79]

you can translate those markers directly to the markers key

Thanks @matteius
I'm not sure what this means. Can you provide an example for my case?

[matteius]

@jtmoon79 certainly, instead of:
py2exe = { version = "*", sys_platform = "== 'win32'"}
try:
py2exe = { version = "*", markers = "sys_platform== 'win32'"}

[jtmoon79]

py2exe = { version = "*", markers = "sys_platform== 'win32'"} worked on my coverlovin2 project.

Thanks @matteius !