Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest 5.0.1 cannot start with Python 3.5.2 #5594

Closed
hexaclock opened this issue Jul 10, 2019 · 31 comments
Closed

pytest 5.0.1 cannot start with Python 3.5.2 #5594

hexaclock opened this issue Jul 10, 2019 · 31 comments
Labels
status: needs information reporter needs to provide more information; can be closed after 2 or more weeks of inactivity

Comments

@hexaclock
Copy link

python3 -mpytest -p no:cacheprovider test_example.py

[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/pytest.py", line 102, in <module>
[13:54:06]    raise SystemExit(pytest.main())
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/_pytest/config/__init__.py", line 55, in main
[13:54:06]    config = _prepareconfig(args, plugins)
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/_pytest/config/__init__.py", line 200, in _prepareconfig
[13:54:06]    pluginmanager=pluginmanager, args=args
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/pluggy/hooks.py", line 289, in __call__
[13:54:06]    return self._hookexec(self, self.get_hookimpls(), kwargs)
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/pluggy/manager.py", line 87, in _hookexec
[13:54:06]    return self._inner_hookexec(hook, methods, kwargs)
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/pluggy/manager.py", line 81, in <lambda>
[13:54:06]    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/pluggy/callers.py", line 203, in _multicall
[13:54:06]    gen.send(outcome)
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/_pytest/helpconfig.py", line 89, in pytest_cmdline_parse
[13:54:06]    config = outcome.get_result()
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/pluggy/callers.py", line 80, in get_result
[13:54:06]    raise ex[1].with_traceback(ex[2])
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/pluggy/callers.py", line 187, in _multicall
[13:54:06]    res = hook_impl.function(*args)
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/_pytest/config/__init__.py", line 661, in pytest_cmdline_parse
[13:54:06]    self.parse(args)
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/_pytest/config/__init__.py", line 869, in parse
[13:54:06]    self._preparse(args, addopts=addopts)
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/_pytest/config/__init__.py", line 803, in _preparse
[13:54:06]    self._initini(args)
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/_pytest/config/__init__.py", line 728, in _initini
[13:54:06]    args, namespace=copy.copy(self.option)
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/_pytest/config/argparsing.py", line 116, in parse_known_and_unknown_args
[13:54:06]    optparser = self._getparser()
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/_pytest/config/argparsing.py", line 86, in _getparser
[13:54:06]    optparser = MyOptionParser(self, self.extra_info, prog=self.prog)
[13:54:06]  File "/home/automake/venvs/testenv/lib/python3.5/site-packages/_pytest/config/argparsing.py", line 333, in __init__
[13:54:06]    allow_abbrev=False,
[13:54:06]TypeError: __init__() got an unexpected keyword argument 'allow_abbrev'

Caused by this commit: d72fb73

It seems allow_abbrev doesn't exist as a valid kwarg to argparse.ArgumentParser in Python 3.5.2:

>>> parser = argparse.ArgumentParser(prog='PROG', allow_abbrev=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: __init__() got an unexpected keyword argument 'allow_abbrev'
@asottile
Copy link
Member

CC @Zac-HD

@asottile
Copy link
Member

Are you sure you're running a released version of python3.5?

allow_abbrev was added in 3.5a3:

$ git log -G allow_abbrev  # while on the 3.5 branch of cpython

...

commit 8089cd642fa8b29e852506218b6355be064c2bd5
Author: Berker Peksag <berker.peksag@gmail.com>
Date:   Sat Feb 14 01:39:17 2015 +0200

    Issue #14910: Add allow_abbrev parameter to argparse.ArgumentParser.
    
    Patch by Jonathan Paugh, Steven Bethard, paul j3 and Daniel Eriksson.

$ git describe --contains 8089cd642fa8b29e852506218b6355be064c2bd5
v3.5.0a2~158

@asottile
Copy link
Member

I cannot reproduce with 3.5.2:

root@fe033779f895:/# python --version
Python 3.5.2
root@fe033779f895:/# pytest t.py
============================= test session starts ==============================
platform linux -- Python 3.5.2, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: /
collected 1 item                                                               

t.py .                                                                   [100%]

=========================== 1 passed in 0.03 seconds ===========================

nor with 3.5.0:

root@b014377c2521:/# python --version
Python 3.5.0
root@b014377c2521:/# pytest t.py
============================= test session starts ==============================
platform linux -- Python 3.5.0, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: /
collected 1 item                                                               

t.py .                                                                   [100%]

=========================== 1 passed in 0.03 seconds ===========================

@asottile
Copy link
Member

@hexaclock I suspect you have either a poisoned lib directory (maybe from running virtualenv against the same dir with an older python?) or a poisoned sys.path (maybe from PYTHONPATH environment variable?)

could you provide reproduction steps?

@asottile asottile added the status: needs information reporter needs to provide more information; can be closed after 2 or more weeks of inactivity label Jul 10, 2019
@asottile
Copy link
Member

closing as invalid -- if you have reproduction steps please comment back and it can be reopened

@zzzeek
Copy link

zzzeek commented Jul 21, 2019

I just got this with Python 3.7, but I have no idea why. As I'm about to reinstall everything and I might never see it again, here it is:


platform linux -- Python 3.7.3, pytest-5.0.1, py-1.5.4, pluggy-0.12.0
rootdir: /home/classic/dev/zzzcomputing/release_scripts/_release_work/SQLAlchemy-1.3.6, inifile: setup.cfg
plugins: forked-0.2, xdist-1.29.0
gw0 C / gw1 C / gw2 C / gw3 C[gw0] node down: Traceback (most recent call last):
  File "/home/classic/.venv3/lib/python3.7/site-packages/execnet/gateway_base.py", line 1072, in executetask
    do_exec(co, loc)  # noqa
  File "<string>", line 1, in do_exec
  File "<remote exec>", line 242, in <module>
  File "<remote exec>", line 217, in remote_initconfig
  File "/home/classic/.venv3/lib/python3.7/site-packages/_pytest/config/__init__.py", line 709, in fromdictargs
    config.parse(args, addopts=False)
  File "/home/classic/.venv3/lib/python3.7/site-packages/_pytest/config/__init__.py", line 869, in parse
    self._preparse(args, addopts=addopts)
  File "/home/classic/.venv3/lib/python3.7/site-packages/_pytest/config/__init__.py", line 803, in _preparse
    self._initini(args)
  File "/home/classic/.venv3/lib/python3.7/site-packages/_pytest/config/__init__.py", line 728, in _initini
    args, namespace=copy.copy(self.option)
  File "/home/classic/.venv3/lib/python3.7/site-packages/_pytest/config/argparsing.py", line 116, in parse_known_and_unknown_args
    optparser = self._getparser()
  File "/home/classic/.venv3/lib/python3.7/site-packages/_pytest/config/argparsing.py", line 86, in _getparser
    optparser = MyOptionParser(self, self.extra_info, prog=self.prog)
  File "/home/classic/.venv3/lib/python3.7/site-packages/_pytest/config/argparsing.py", line 333, in __init__
    allow_abbrev=False,
TypeError: __init__() got an unexpected keyword argument 'allow_abbrev'

@zzzeek
Copy link

zzzeek commented Jul 21, 2019

I had the Python 2 argparse port sitting in my python 3.7 venv. just a heads up this might cause confusion.

@asottile
Copy link
Member

even with that I can't reproduce :S

$ pip install pytest argparse
Requirement already satisfied: pytest in ./venv/lib/python3.6/site-packages (5.0.1)
Requirement already satisfied: argparse in ./venv/lib/python3.6/site-packages (1.4.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in ./venv/lib/python3.6/site-packages (from pytest) (0.12.0)
Requirement already satisfied: py>=1.5.0 in ./venv/lib/python3.6/site-packages (from pytest) (1.8.0)
Requirement already satisfied: wcwidth in ./venv/lib/python3.6/site-packages (from pytest) (0.1.7)
Requirement already satisfied: importlib-metadata>=0.12 in ./venv/lib/python3.6/site-packages (from pytest) (0.18)
Requirement already satisfied: attrs>=17.4.0 in ./venv/lib/python3.6/site-packages (from pytest) (19.1.0)
Requirement already satisfied: packaging in ./venv/lib/python3.6/site-packages (from pytest) (19.0)
Requirement already satisfied: more-itertools>=4.0.0 in ./venv/lib/python3.6/site-packages (from pytest) (7.2.0)
Requirement already satisfied: atomicwrites>=1.0 in ./venv/lib/python3.6/site-packages (from pytest) (1.3.0)
Requirement already satisfied: zipp>=0.5 in ./venv/lib/python3.6/site-packages (from importlib-metadata>=0.12->pytest) (0.5.2)
Requirement already satisfied: six in ./venv/lib/python3.6/site-packages (from packaging->pytest) (1.12.0)
Requirement already satisfied: pyparsing>=2.0.2 in ./venv/lib/python3.6/site-packages (from packaging->pytest) (2.4.1)
$ pytest t.py -n auto
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: /tmp/x
plugins: forked-1.0.2, xdist-1.29.0
gw0 [1] / gw1 [1] / gw2 [1] / gw3 [1] / gw4 [1]
.                                                                        [100%]
=========================== 1 passed in 0.40 seconds ===========================

@zzzeek
Copy link

zzzeek commented Jul 21, 2019

shrugs maybe it was an old one, dunno

@asottile
Copy link
Member

the closest I can get is

$ PYTHONPATH=venv/lib/python3.6/site-packages ./venv/bin/pytest t.py
...
TypeError: __init__() got an unexpected keyword argument 'allow_abbrev'

but that's pretty nonsense :S

@Karel-van-de-Plassche
Copy link

Might be a specific interaction between argparse-1.4.0 and pytest-5.0.1, got this when downgrading pytest with pip.

@asottile
Copy link
Member

@Karel-van-de-Plassche can you provide reproduction steps?

@pbaughman
Copy link

IDK if it helps or not, but I encountered this problem with pytest 5.2.0 when I pip3 installed an unrelated package which (probably erroneously) installed the 1.4.0 version of argparse over the one installed in the standard library.

@asottile
Copy link
Member

asottile commented Oct 7, 2019

unless you're doing weird things with PYTHONPATH even that shouldn't matter -- site-packages is after the stdlib in sys.path ordering (unless something wacky / incorrect is going on)

@jnguyen19
Copy link

the closest I can get is

$ PYTHONPATH=venv/lib/python3.6/site-packages ./venv/bin/pytest t.py
...
TypeError: __init__() got an unexpected keyword argument 'allow_abbrev'

but that's pretty nonsense :S

i saw a similar problem, when i had set my PYTHONPATH to the site-packages: like so:
PYTHONPATH=/usr/local/bin/python3.7:/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages

@asottile
Copy link
Member

you should not set PYTHONPATH to site-packages, it'll be included automatically

@jnguyen19

This comment has been minimized.

@asottile

This comment has been minimized.

@jnguyen19

This comment has been minimized.

@asottile

This comment has been minimized.

@jnguyen19

This comment has been minimized.

@dmwelch
Copy link

dmwelch commented Apr 16, 2021

I had this issue using virtualenvwrapper because my shell was setting:

PYTHONPATH=/usr/local/lib/python3.9/site-packages

Unsetting it fixed the issue in my case

@denis-bz
Copy link

pytest --version -->

# from: pytest --version
# run: 31 May 2021 11:53  in ~bz/py/etc/issues  Denis-iMac  mac 10.10.5 i386 
Python 3.7.6

/opt/local/py3/lib/python3.7/site-packages/_pytest/_version.py
version = '6.2.4'
version_tuple = (6, 2, 4)

 273365 Apr  7  2020 /opt/local/py3/lib/python3.7/site-packages/pyparsing.py
__version__ = "2.4.7"
__versionTime__ = "30 Mar 2020 00:43 UTC"

Traceback (most recent call last):
  File "/opt/local/py3/lib/python3.7/site-packages/bin/pytest", line 8, in <module>
    sys.exit(console_main())
  File "/opt/local/py3/lib/python3.7/site-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/opt/local/py3/lib/python3.7/site-packages/_pytest/config/__init__.py", line 143, in main
    config = _prepareconfig(args, plugins)
  File "/opt/local/py3/lib/python3.7/site-packages/_pytest/config/__init__.py", line 319, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/opt/local/py3/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/opt/local/py3/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/opt/local/py3/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/opt/local/py3/lib/python3.7/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/opt/local/py3/lib/python3.7/site-packages/_pytest/helpconfig.py", line 100, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/opt/local/py3/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/opt/local/py3/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/opt/local/py3/lib/python3.7/site-packages/_pytest/config/__init__.py", line 1003, in pytest_cmdline_parse
    self.parse(args)
  File "/opt/local/py3/lib/python3.7/site-packages/_pytest/config/__init__.py", line 1283, in parse
    self._preparse(args, addopts=addopts)
  File "/opt/local/py3/lib/python3.7/site-packages/_pytest/config/__init__.py", line 1157, in _preparse
    self._initini(args)
  File "/opt/local/py3/lib/python3.7/site-packages/_pytest/config/__init__.py", line 1074, in _initini
    args, namespace=copy.copy(self.option)
  File "/opt/local/py3/lib/python3.7/site-packages/_pytest/config/argparsing.py", line 155, in parse_known_and_unknown_args
    optparser = self._getparser()
  File "/opt/local/py3/lib/python3.7/site-packages/_pytest/config/argparsing.py", line 113, in _getparser
    optparser = MyOptionParser(self, self.extra_info, prog=self.prog)
  File "/opt/local/py3/lib/python3.7/site-packages/_pytest/config/argparsing.py", line 387, in __init__
    allow_abbrev=False,
TypeError: __init__() got an unexpected keyword argument 'allow_abbrev'

Commenting out line 387 allow_abbrev=False in argparsing.py fixes this:

pytest --version
pytest 6.2.4

also self.allow_abbrev line 441 ? dunno

@asottile
Copy link
Member

@denis-bz as stated above, that's impossible unless you're doing hacks with PYTHONPATH or sys.path and have the deprecated argparse "backport" (for python<=2.6) installed

@denis-bz
Copy link

@asottile, it's a pip versionitis (version disease) problem

pip3 install --upgrade --prefix /opt/local/py3/lib/python3.7/site-packages argparse
Collecting argparse
  Downloading argparse-1.4.0-py2.py3-none-any.whl (23 kB)
Installing collected packages: argparse
Successfully installed argparse-1.4.0

confused pip because 1.4.0 > 1.1 !! I'll raise it there.

(I keep all pip installs in PYTHONPATH = /opt/local/py3/lib/python3.7/site-packages
ahead of /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7 in sys.path
sound reasonable ?

@asottile
Copy link
Member

no it isn't a pip problem, argparse shouldn't be installed. it has been in the standard library since 2009

using PYTHONPATH in that way is not recommended and is what is contributing to your problem

@denis-bz
Copy link

@asottile, would you agree that keeping all pip installs in a separate dir
and /Library/.../python3.7 readonly is reasonable ?
and that old old version 1.4.0, newest version 1.1 is a bug -- whose ?

@RonnyPfannschmidt
Copy link
Member

@denis-bz on python 3.7 NEVER install argparse, it breaks the world there

as for the python layout for installs - i beleive the general suggestion is to use virtualenv, but personally im unfamiliar with osx

@denis-bz
Copy link

@RonnyPfannschmidt, the old argparse got installed by pip somethingelse long ago,
I installed again just to reproduce the error.

Repeat, old old version 1.4.0, newest version 1.1 is a bug -- whose ?

@RonnyPfannschmidt
Copy link
Member

@denis-bz under python 3.7 installing argparse is always a error - argparse on pypi is not up2date with python 3.6+ and will break your python if you make it importable before the stdlib

@asottile
Copy link
Member

@RonnyPfannschmidt, the old argparse got installed by pip somethingelse long ago,
I installed again just to reproduce the error.

Repeat, old old version 1.4.0, newest version 1.1 is a bug -- whose ?

repeat, yours: don't install it

would you agree that keeping all pip installs in a separate dir
and /Library/.../python3.7 readonly is reasonable ?

no I don't agree, that's what virtualenvs are for and your PYTHONPATH hack is going to cause problems (such as the one you're encountering)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: needs information reporter needs to provide more information; can be closed after 2 or more weeks of inactivity
Projects
None yet
Development

No branches or pull requests

9 participants