Python 3.8 Support

[s0undt3ch]

Description of Issue

Support Python 3.8

Arch Linux updated to python 3.8 and one of the tests upgrades the system, which means, while the test suite runs, we upgrade from 3.7 to 3.8, thus failing at the very end.

https://jenkinsci.saltstack.com/job/pr-kitchen-archlts-py3/job/PR-55276/4/

Current stopgap: #55309

[s0undt3ch]

Problematic code:

• from platform import _supported_dists

[Ch3LL]

• Need to have a running list of deprecation's which will affect Salt, for example: abc from collections in Py 3.8. Plan to have public list in GitHub ? where contributors / Salt can add to the list and generate Issues such that Salt can be updated.

[Ikke]

For Alpine, the lack of python3.8 support prevents salt-minion from running, so this is not just a test failure, but actual run-time errors:

$ salt-minion
Traceback (most recent call last):
[..]
  File "/usr/lib/python3.8/site-packages/salt/grains/core.py", line 40, in <module>
    from platform import _supported_dists
ImportError: cannot import name '_supported_dists' from 'platform' (/usr/lib/python3.8/platform.py)

[itamarjp]

Python 3.8 will land fedora soon. (already there for Fedora 32)
https://koji.fedoraproject.org/koji/packageinfo?packageID=9781

[Vaelatern]

This crops up after working around problems in issue #56551 specifically from platform import _supported_dists

[chalbersma]

I think this will block support for Ubuntu 20.04 as it uses python3.8 by default.

[Ch3LL]

can you try this PR #56031?

[h0jeZvgoxFepBQ2C]

Ran into this as well.. Would be great to have this fixed soon.

[asvany]

this bug still alive on mandjaro linux.
"ImportError: cannot import name '_supported_dists' from 'platform' (/usr/lib/python3.8/platform.py)
"

[terminalmage]

@asvany _supported_dists is no longer referenced in Salt code as of the upcoming 3001 (Sodium) release, aside from in a comment.

% git show v3000:salt/grains/core.py | fgrep _supported_dists
from platform import _supported_dists
_supported_dists += ('arch', 'mageia', 'meego', 'vmware', 'bluewhite64',
# Please see os_data() and _supported_dists.
             linux_distribution(supported_dists=_supported_dists)]


% git show v3001rc1:salt/grains/core.py | fgrep _supported_dists
# Please see os_data() and _supported_dists.

[s0undt3ch]

@asvany As @terminalmage mentioned, this issue should be fixed.
What version of salt are you running?
Please try v3001rc1.

[asvany]

┌──[as@as17]─[/tmp]─[○] [p jún 12 - 12:23]
└──● virtualenv py_env_salt
created virtual environment CPython3.8.3.final.0-64 in 391ms
creator CPython3Posix(dest=/tmp/py_env_salt, clear=False, global=False)
seeder FromAppData(download=False, pip=latest, setuptools=latest, wheel=latest, via=copy, app_data_dir=/home/as/.local/share/virtualenv/seed-app-data/v1.0.1)
activators BashActivator,CShellActivator,FishActivator,PowerShellActivator,PythonActivator,XonshActivator
┌──[as@as17]─[/tmp]─[○] [p jún 12 - 12:23]
└──● . ./py_env_salt/bin/activate
(py_env_salt) ┌──[as@as17]─[/tmp]─[○] [p jún 12 - 12:24]
└──● pip install salt-ssh
Collecting salt-ssh
Downloading salt-ssh-3000.3.tar.gz (15.2 MB)
|████████████████████████████████| 15.2 MB 5.5 MB/s
ERROR: Command errored out with exit status 1:
command: /tmp/py_env_salt/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6vboudje/salt-ssh/setup.py'"'"'; file='"'"'/tmp/pip-install-6vboudje/salt-ssh/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-yublgzzw
cwd: /tmp/pip-install-6vboudje/salt-ssh/
Complete output (14 lines):
Traceback (most recent call last):
File "/tmp/pip-install-6vboudje/salt-ssh/salt/version.py", line 15, in
from platform import linux_distribution as _deprecated_linux_distribution
ImportError: cannot import name 'linux_distribution' from 'platform' (/usr/lib/python3.8/platform.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-install-6vboudje/salt-ssh/setup.py", line 126, in <module>
    exec(compile(open(SALT_VERSION).read(), SALT_VERSION, 'exec'))
  File "/tmp/pip-install-6vboudje/salt-ssh/salt/version.py", line 22, in <module>
    from distro import linux_distribution
ModuleNotFoundError: No module named 'distro'
----------------------------------------

ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
(py_env_salt) ┌─[✗]──[as@as17]─[/tmp]─[○] [p jún 12 - 12:24]
└──●

[louwers]

salt-ssh doesn't seem to be working when only Python 3.8 is installed on the host.

$ sudo -E salt-ssh tgt test.ping
tgt:
    ----------
    retcode:
        10
    stderr:
    stdout:
        ERROR: Python version error. Recommendation(s) follow:
        Install Python 2.6 / Python 3 Salt dependencies on the Salt SSH master 
        to interact with Python 2.6 / Python 3 targets
$ sudo -E salt-ssh -r tgt "python --version"
tgt:
    ----------
    retcode:
        0
    stderr:
    stdout:
        Python 3.8.3

Edit: my bad, I'm running an old version of salt-ssh

$ salt-ssh --version
salt-ssh 2019.2.4 (Fluorine)

Edit2: with salt-ssh 3001.1

tgt:
    ----------
    retcode:
        1
    stderr:
        Traceback (most recent call last):
          File "/var/tmp/.root_97d892_salt/salt-call", line 27, in <module>
            salt_call()
          File "/var/tmp/.root_97d892_salt/pyall/salt/scripts.py", line 426, in salt_call
            import salt.cli.call
          File "/var/tmp/.root_97d892_salt/pyall/salt/cli/call.py", line 5, in <module>
            import salt.utils.parsers
          File "/var/tmp/.root_97d892_salt/pyall/salt/utils/parsers.py", line 27, in <module>
            import salt.config as config
          File "/var/tmp/.root_97d892_salt/pyall/salt/config/__init__.py", line 99, in <module>
            _DFLT_IPC_WBUFFER = _gather_buffer_space() * .5
          File "/var/tmp/.root_97d892_salt/pyall/salt/config/__init__.py", line 88, in _gather_buffer_space
            import salt.grains.core
          File "/var/tmp/.root_97d892_salt/pyall/salt/grains/core.py", line 40, in <module>
            from platform import _supported_dists
        ImportError: cannot import name '_supported_dists' from 'platform' (/usr/lib/python3.8/platform.py)
    stdout:

[Ch3LL]

@louwers i'm not seeing from platform import _supported_dists anywhere in the code for 3001.1. i'm guessing that is still the 2019.2.4 code on the target host. It should have updated it to 3001.1 but can you wipe that out -w and make sure you are still not seeing the issue?

[cobordism]

For the record, I am seeing the same error in qubesOS using a debian-11 template:

Error on updating debian-testing: Command '['sudo', 'qubesctl', '--skip-dom0', '--targets=debian-testing', '--show-output', 'state.sls', 'update.qubes-vm']' returned non-zero exit status 20
debian-testing:
      ----------
      _error:
          Failed to return clean data
      retcode:
          1
      stderr:
          Traceback (most recent call last):
            File "/var/tmp/.root_62a99a_salt/salt-call", line 27, in <module>
              salt_call()
            File "/var/tmp/.root_62a99a_salt/pyall/salt/scripts.py", line 440, in salt_call
              import salt.cli.call
            File "/var/tmp/.root_62a99a_salt/pyall/salt/cli/call.py", line 5, in <module>
              import salt.utils.parsers
            File "/var/tmp/.root_62a99a_salt/pyall/salt/utils/parsers.py", line 28, in <module>
              import salt.config as config
            File "/var/tmp/.root_62a99a_salt/pyall/salt/config/__init__.py", line 100, in <module>
              _DFLT_IPC_WBUFFER = _gather_buffer_space() * .5
            File "/var/tmp/.root_62a99a_salt/pyall/salt/config/__init__.py", line 89, in _gather_buffer_space
              import salt.grains.core
            File "/var/tmp/.root_62a99a_salt/pyall/salt/grains/core.py", line 40, in <module>
              from platform import _supported_dists
          ImportError: cannot import name '_supported_dists' from 'platform' (/usr/lib/python3.8/platform.py)

[s0undt3ch]

@cobordism which version of salt is that?

[s0undt3ch]

see @Ch3LL's comment above

[cobordism]

@cobordism which version of salt is that?

I'm afraid I'm not sure. It's whatever is in Qubes 4.0.3.

dom0:
salt-minion --version gives me 2017.7.1 (Nitrogen), as does salt-call --version and salt-proxy --version
Those are all the salt-* commands I can see in the dom0 system.

AppVM:
However in the fedora appvm I can see salt-ssh --version as 3000.2.

Perhaps this is an issue I have to raise with Qubes?
https://github.com/QubesOS?q=salt

EDIT:
Looks like it was fixed a few days ago:
QubesOS/qubes-issues#6188

[s0undt3ch]

Indeed, 2017.x is no longer supported.