Seemingly hard dependency on pycrypto prevents recommended operation

[Vaelatern]

Description of Issue

Failure to run saltstack due to a collection of issues.

I run salt in a test config with: salt-ssh '*' test.ping --roster-file=./roster

First it depends on pycrypto. This is because in my system in /usr/lib/python3.8/site-packages/salt-3000.1-py3.8.egg-info/requires.txt I can see pycrypto outright called out.

Then I tried removing that line. After all, I do have M2Crypto version 0.35.2 installed, and as I understand it, salt will try it.

Then it failed again because The 'msgpack!=0.5.5,<1.0.0,>=0.5' distribution was not found and is required by salt which is in fact true: on the change to python3, my distribution changed from msgpack 0.6.2 to msgpack 1.0.0.

Is there a reason to max-version-lock msgpack below 1.0.0? Is there a reason the egg seems to now (since the previous version on py2: 2019.2.0) depend on pycrypto?

Is there some workaround, something obvious I am missing?

Steps to Reproduce Issue

From Void Linux, xbps-install salt and try any command.

Versions Report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

$ salt --versions-report
Traceback (most recent call last):
  File "/bin/salt", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3251, in <module>
    def _initialize_master_working_set():
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3234, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3263, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pycrypto>=2.6.1' distribution was not found and is required by salt

$ salt --versions-report
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 791, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (msgpack 1.0.0 (/usr/lib/python3.8/site-packages), Requirement.parse('msgpack!=0.5.5,<1.0.0,>=0.5'), {'salt'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/bin/salt", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3251, in <module>
    def _initialize_master_working_set():
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3234, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 3263, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 585, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python3.8/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'msgpack!=0.5.5,<1.0.0,>=0.5' distribution was not found and is required by salt

Yes, the following, after I remove pycrypto and msgpack lines from /usr/lib/python3.8/site-packages/salt-3000.1-py3.8.egg-info/requires.txt does in fact list pycrypto. This confuses me too.

$ salt --versions-report
Salt Version:
           Salt: 3000.1

Dependency Versions:
           cffi: 1.12.3
       cherrypy: Not Installed
       dateutil: 2.8.0
      docker-py: Not Installed
          gitdb: 2.0.5
      gitpython: 2.1.11
         Jinja2: 2.10.1
        libgit2: Not Installed
       M2Crypto: 0.35.2
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 1.0.0
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 3.9.7
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.8.1 (default, Dec 23 2019, 13:11:45)
   python-gnupg: Not Installed
         PyYAML: 5.3
          PyZMQ: 19.0.0
          smmap: 2.0.5
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.3.2

System Versions:
           dist: void rolling void
         locale: utf-8
        machine: x86_64
        release: 5.4.21_1
         system: Linux
        version: void rolling void

[Noah-Huppert]

I can reproduce this issue.

[frogunder]

@Vaelatern Thank you for reporting this issue.

I see you are using Python 3.8, salt does not currently support Python 3.8.
There are work being done to hopefully support Python 3.8 in the near future.

@dmurphy18 Can you take a look at this and see if anything stands out to you with regards to pycrypto and msgpack errors?

Thanks