py33 environment uses python2.7 virtualenv.py?

[pytoxbot]

• Bitbucket: https://bitbucket.org/hpk42/tox/issue/129
• Originally reported by: jorgenschaefer
• Originally created at: 2013-10-21T10:11:19.428

I'm using tox in a pretty simple setup, and can't get it to run with the py33 environment. It is quite likely that this is a bad setup on my local system, but I can't figure out why it is doing what it is doing.

$ cat setup.py 
#!/usr/bin/env python

from setuptools import setup, find_packages

setup(name="test")

$ cat tox.ini
[tox]
envlist = py33

[testenv]
commands = ls

$ tox
GLOB sdist-make: /tmp/tox-test/setup.py
py33 create: /tmp/tox-test/.tox/py33
ERROR: invocation failed, logfile: /tmp/tox-test/.tox/py33/log/py33-0.log
ERROR: actionid=py33
msg=getenv
cmdargs=['/home/forcer/bin/python3.3', '/home/forcer/.local/lib/python2.7/site-packages/virtualenv.py', '--setuptools', '--python', '/home/forcer/bin/python3.3', 'py33']
env=[...]
Already using interpreter /home/forcer/bin/python3.3
Traceback (most recent call last):
  File "/home/forcer/.local/lib/python2.7/site-packages/virtualenv.py", line 2308, in <module>
    main()
  File "/home/forcer/.local/lib/python2.7/site-packages/virtualenv.py", line 821, in main
    symlink=options.symlink)
  File "/home/forcer/.local/lib/python2.7/site-packages/virtualenv.py", line 952, in create_environment
    home_dir, lib_dir, inc_dir, bin_dir = path_locations(home_dir)
  File "/home/forcer/.local/lib/python2.7/site-packages/virtualenv.py", line 1017, in path_locations
    inc_dir = join(home_dir, 'include', stdout.strip(), py_version + abiflags)
  File "/home/forcer/Programs/Python/python3.3/lib/python3.3/posixpath.py", line 92, in join
    "components.") from None
TypeError: Can't mix strings and bytes in path components.
ERROR: InvocationError: /home/forcer/bin/python3.3 /home/forcer/.local/lib/python2.7/site-packages/virtualenv.py --setuptools --python /home/forcer/bin/python3.3 py33 (see /tmp/tox-test/.tox/py33/log/py33-0.log)
___________________________________ summary ____________________________________
ERROR:   py33: InvocationError: /home/forcer/bin/python3.3 /home/forcer/.local/lib/python2.7/site-packages/virtualenv.py --setuptools --python /home/forcer/bin/python3.3 py33 (see /tmp/tox-test/.tox/py33/log/py33-0.log)

I can't figure out why it is using the virtualenv.py from python2.7. Shouldn't it be using venv.py from python3.3?

[pytoxbot]

Original comment by @hpk42

You can install the dev version with pip install --pre -U -i http://devpi.net/hpk/dev/+simple/ tox, btw.

[pytoxbot]

Original comment by jorgenschaefer

Thanks a lot! :-)

[pytoxbot]

Original comment by @hpk42

fix issue129: tox now uses Popen(..., universal_newlines=True) to force
creation of unicode stdout/stderr streams. fixes a problem on specific
platform configs when creating virtualenvs with Python3.3. Thanks Jorgen Schäfer
or investigation and solution sketch.

→ <>

[pytoxbot]

Original comment by jorgenschaefer

stdout does:

{'abiflags': 'm',
 'stdout': b'multiarch-x86_64-linux',
 'py_version': 'python3.3',
 'home_dir': 'py33'}

Indeed, according to the documentation, if subprocess.Popen is called without universal_newlines, stdin, stdout and stderr are opened as binary streams with no conversion: http://docs.python.org/3.3/library/subprocess.html#index-1

Looks like this only happens when /usr/bin/multiarch-platform exists. I can confirm all works fine if I remove that file; it's installed on this host by the rpmbuild package.

Edit: … and yes, adding universal_newlines=True also fixes it.

[pytoxbot]

Original comment by @hpk42

Using the python2.7 virtualenv is fine, expected behaviour. After all, before creating the py33 env, there is no "virtualenv" installed with python3.3.

I can't reproduce the problem.

Could you try to find out which of the join arguments (virtualenv.py line 1017) actually contains bytes?