[BUG] State pip.installed fails when upgrade is True with pip version 20.3

[ramboramjad]

Problem description

function pip.installed fails when running it with upgrade: True with pip3 v10+. After looking through the code I noticed that the state calls the function pip.list_all_versions. This module in turn runs the command [your running python] -m pip install {pkgname}==versions in my case it will run python3 -m pip install pip==versions the idea behind the "random" word versions is that pip would return all available versions if it mismatches. However in pip version 10+ it does not return the from versions: xx xx anymore. Due to this it gives an error and pip will not be installed.

Testing/Research

output from the command line when running the command myself with pip v9.0.3:

# python3 -m pip install pip==blas
Collecting pip==blas
  Could not find a version that satisfies the requirement pip==blas (from versions: 0.2, 0.2.1, 0.3, 0.3.1, 0.4, 0.5, 0.5.1, 0.6, 0.6.1, 0.6.2, 0.6.3, 0.7, 0.7.1, 0.7.2, 0.8, 0.8.1, 0.8.2, 0.8.3, 1.0, 1.0.1, 1.0.2, 1.1, 1.2, 1.2.1, 1.3, 1.3.1, 1.4, 1.4.1, 1.5, 1.5.1, 1.5.2, 1.5.3, 1.5.4, 1.5.5, 1.5.6, 6.0, 6.0.1, 6.0.2, 6.0.3, 6.0.4, 6.0.5, 6.0.6, 6.0.7, 6.0.8, 6.1.0, 6.1.1, 7.0.0, 7.0.1, 7.0.2, 7.0.3, 7.1.0, 7.1.1, 7.1.2, 8.0.0, 8.0.1, 8.0.2, 8.0.3, 8.1.0, 8.1.1, 8.1.2, 9.0.0, 9.0.1, 9.0.2, 9.0.3, 10.0.0b1, 10.0.0b2, 10.0.0, 10.0.1, 18.0, 18.1, 19.0, 19.0.1, 19.0.2, 19.0.3, 19.1, 19.1.1, 19.2, 19.2.1, 19.2.2, 19.2.3, 19.3, 19.3.1, 20.0, 20.0.1, 20.0.2, 20.1b1, 20.1, 20.1.1, 20.2b1, 20.2, 20.2.1, 20.2.2, 20.2.3, 20.2.4, 20.3b1, 20.3, 20.3.1)
No matching distribution found for pip==blas

output from the command line when running the command myself with pip v20.3:

# python3 -m pip install pip==blas
ERROR: Could not find a version that satisfies the requirement pip==blas
ERROR: No matching distribution found for pip==blas

Here is my minion debug output:

[DEBUG   ] pip: Using pip from currently-running Python
[DEBUG   ] An exception occurred in this state: 'NoneType' object is not subscriptable
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/state.py", line 2154, in call
    *cdata["args"], **cdata["kwargs"]
  File "/usr/lib/python3.6/site-packages/salt/loader.py", line 2087, in wrapper
    return f(*args, **kwargs)
  File "/var/cache/salt/minion/extmods/states/pip_state.py", line 867, in installed
    **kwargs
  File "/var/cache/salt/minion/extmods/states/pip_state.py", line 330, in _check_if_installed
    desired_version = available_versions[-1]
TypeError: 'NoneType' object is not subscriptable
[ERROR   ] An exception occurred in this state: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/salt/state.py", line 2154, in call
    *cdata["args"], **cdata["kwargs"]
  File "/usr/lib/python3.6/site-packages/salt/loader.py", line 2087, in wrapper
    return f(*args, **kwargs)
  File "/var/cache/salt/minion/extmods/states/pip_state.py", line 867, in installed
    **kwargs
  File "/var/cache/salt/minion/extmods/states/pip_state.py", line 330, in _check_if_installed
    desired_version = available_versions[-1]
TypeError: 'NoneType' object is not subscriptable

My state snippet:

pip3:
  pip.installed:
    - name: pip
    - upgrade: True

Note: if i comment the - upgrade: True It runs fine.

Versions report:

Salt Version:
           Salt: 3001

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.8.1
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.11.1
        libgit2: Not Installed
       M2Crypto: 0.35.2
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: 1.3.12
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: 3.9.7
         pygit2: Not Installed
         Python: 3.6.8 (default, Nov 16 2020, 16:55:22)
   python-gnupg: 0.4.4
         PyYAML: 3.13
          PyZMQ: 17.0.0
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.1.4

System Versions:
           dist: centos 7 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-1160.6.1.el7.x86_64
         system: Linux
        version: CentOS Linux 7 Core

[welcome]

Hi there! Welcome to the Salt Community! Thank you for making your first contribution. We have a lengthy process for issues and PRs. Someone from the Core Team will follow up as soon as possible. In the meantime, here’s some information that may help as you continue your Salt journey.
Please be sure to review our Code of Conduct. Also, check out some of our community resources including:

• Community Wiki
• Salt’s Contributor Guide
• Join our Community Slack
• IRC on Freenode
• SaltStack YouTube channel
• SaltStackInc Twitch channel

There are lots of ways to get involved in our community. Every month, there are around a dozen opportunities to meet with other contributors and the Salt Core team and collaborate in real time. The best way to keep track is by subscribing to the Salt Community Events Calendar.
If you have additional questions, email us at core@saltstack.com. We’re glad you’ve joined our community and look forward to doing awesome things with you!

[andreasnuesslein]

according to the the underlying pip problem/ticket: pypa/pip#9139

you can use pip install --use-deprecated=legacy-resolver $1==version at the moment.

[MrMino]

FYI even if my PR for pip goes through, this will not work after packaging drops LegacyVersion support, as pip uses packaging to parse these requirements. See pfmoore's comment.

[sagetherage]

we fixed what we can for Aluminium closing for now @MrMino let me know if you want to open this back up or if you want to submit a new issue for your latest comment. Both work I would prefer to have a new issue if you don't mind.

[MrMino]

@MrMino let me know if you want to open this back up or if you want to submit a new issue for your latest comment.

@sagetherage This wasn't my issue, I was just informing you about part of a feature being removed from Pip.

[ahrex]

@sagetherage would it be OK to re-open this ticket?

I'm using salt version 3003 (Aluminium) and I still experience the problem.

Thanks!

Edit: it turns out Debian bullseye's version of python3-pip, 20.3.4-4, breaks with salt version 3003. Upgrading pip to 21.2.4 fixes the issue. Feel free to keep the issue closed, I think the proper way of pursuing this is with the Debian team. Thanks!