Tests are broken on master for pypy / pypy3

[asottile]

seems we don't test them in CI any more? perhaps an oversight?

appears to have regressed in #1310

The core of the problem is that tox -e py27 might match a python2.7 that is pypy2 which fails the interpreter matching code (python != pypy)

These are the failing tests:

FAILED tests/unit/interpreters/test_interpreters.py::test_tox_get_python_executable - assert None
FAILED tests/unit/config/test_config.py::test_config_current_py - AssertionError: 1
FAILED tests/integration/test_package_int.py::test_package_setuptools - AssertionError: 1
FAILED tests/unit/test_z_cmdline.py::test_notest - AssertionError: 1

They each fail similar to:

E       AssertionError: 1
E       Traceback (most recent call last):
E         File "/tmp/tox/.tox/pypy/site-packages/tox/_pytestplugin.py", line 126, in run
E           tox.session.main([str(x) for x in argv])
E         File "/tmp/tox/.tox/pypy/site-packages/tox/session/__init__.py", line 68, in main
E           exit_code = session.runcommand()
E         File "/tmp/tox/.tox/pypy/site-packages/tox/session/__init__.py", line 192, in runcommand
E           return self.subcommand_test()
E         File "/tmp/tox/.tox/pypy/site-packages/tox/session/__init__.py", line 208, in subcommand_test
E           venv.package = self.hook.tox_package(session=self, venv=venv)
E         File "/tmp/tox/.tox/pypy/site-packages/pluggy/hooks.py", line 289, in __call__
E           return self._hookexec(self, self.get_hookimpls(), kwargs)
E         File "/tmp/tox/.tox/pypy/site-packages/pluggy/manager.py", line 87, in _hookexec
E           return self._inner_hookexec(hook, methods, kwargs)
E         File "/tmp/tox/.tox/pypy/site-packages/pluggy/manager.py", line 81, in <lambda>
E           firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
E         File "/tmp/tox/.tox/pypy/site-packages/pluggy/callers.py", line 208, in _multicall
E           return outcome.get_result()
E         File "/tmp/tox/.tox/pypy/site-packages/pluggy/callers.py", line 81, in get_result
E           _reraise(*ex)  # noqa
E         File "/tmp/tox/.tox/pypy/site-packages/pluggy/callers.py", line 187, in _multicall
E           res = hook_impl.function(*args)
E         File "/tmp/tox/.tox/pypy/site-packages/tox/package/__init__.py", line 16, in tox_package
E           session.package, session.dist = get_package(session)
E         File "/tmp/tox/.tox/pypy/site-packages/tox/package/__init__.py", line 29, in get_package
E           package = acquire_package(config, session)
E         File "/tmp/tox/.tox/pypy/site-packages/tox/package/__init__.py", line 40, in acquire_package
E           path = build_package(config, session)
E         File "/tmp/tox/.tox/pypy/site-packages/tox/package/builder/__init__.py", line 9, in build_package
E           return build(config, session)
E         File "/tmp/tox/.tox/pypy/site-packages/tox/package/builder/isolated.py", line 31, in build
E           raise package_venv.status
E       InterpreterNotFound: InterpreterNotFound: python
E       
E       using tox.ini: /tmp/pytest-of-asottile/pytest-56/popen-gw6/test_package_setuptools0/magic/tox.ini (pid 6031)
E       using tox-3.13.3.dev6+g1e6430c from /tmp/tox/.tox/pypy/site-packages/tox/__init__.pyc (pid 6031)
E       .package start: getenv /tmp/pytest-of-asottile/pytest-56/popen-gw6/test_package_setuptools0/magic/.tox/.package
E       .package cannot reuse: no previous config /tmp/pytest-of-asottile/pytest-56/popen-gw6/test_package_setuptools0/magic/.tox/.package/.tox-config1
E       .package create: /tmp/pytest-of-asottile/pytest-56/popen-gw6/test_package_setuptools0/magic/.tox/.package
E       .package uses None
E       ERROR: InterpreterNotFound: python
E       .package finish: getenv /tmp/pytest-of-asottile/pytest-56/popen-gw6/test_package_setuptools0/magic/.tox/.package after 0.00 seconds

[asottile]

A (perhaps?) wrong fix would be to adjust https://github.com/tox-dev/tox/blob/master/src/tox/_pytestplugin.py#L576-L580 to return pypy## for pypy

Another fix would be to allow python to match any interpreter type (restoring the previous behaviour I think)

@gaborbernat thoughts?

GitHub

tox-dev/tox

Command line driven CI frontend and development task automation tool - tox-dev/tox

[gaborbernat]

Why would the wrong fix be to return pypy there instead? I'd prefer that I think 🤔 but I might be missing something here.

[asottile]

it would hide the regression -- unless that change was intentional

[gaborbernat]

The change was intentional.

[asottile]

so this should InterpreterNotFound when run with pypy?

[tox]

[testenv]
basepython=python

[gaborbernat]

Yes 👍 Or maybe not 🤔 Actually it should work if python on the path points to pypy.

[asottile]

ok I'll fix the suite according to that and add pypy back to our matrix