py_test not respecting virtualenv (using python2 instead of python3)

[kgreenek]

Description of the problem / feature request:

After upgrading from bazel 0.19 to 0.21, I am no longer able to run my python tests, which require python3.

We enforce python3 by using a virtual environment. However, even though the virtual env is activated, when I print out the python version, it is 2.7. When I print the python interpreter path, it is using /usr/bin/python instead of the python interpreter from my virtualenv.

As of right now, this is preventing us from being able to upgrade from bazel 0.19 because we can't run our tests.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Set up and activate a virtualenv with python3:

python3 -m venv ~/pyenv
source ~/pyenv/bin/activate

Verify python3 is the default python and its path is correct:

$ which python
/home/kevin/pyenv/bin/python
$ python --version
Python 3.5.2

python3_test.py

#!/usr/bin/env python3
import sys
import unittest


class Python3Test(unittest.TestCase):
    def test_python3(self):
        self.assertTrue(sys.version.startswith("3"))


if __name__ == "__main__":
    print(sys.version)
    print(sys.executable)
    unittest.main()

BUILD.bazel

py_test(
    name = "python3_test",
    size = "small",
    srcs = ["python3_test.py"],
)

When the above test is run, it fails and prints:

======================================================================
FAIL: test_python3 (__main__.Python3Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "[...]", line 8, in test_python3
    self.assertTrue(sys.version.startswith("3"))
AssertionError: False is not true

----------------------------------------------------------------------
Ran 1 test in 0.000s

FAILED (failures=1)
2.7.12 (default, Nov 12 2018, 14:36:49) 
[GCC 5.4.0 20160609]
/usr/bin/python

What operating system are you running Bazel on?

Ubuntu 16.04

What's the output of bazel info release?

release 0.21.0

[kgreenek]

As a workaround, I was able to get it to work by adding the following line to my project's .bazelrc:

test --python_top=//:my_python

And adding a py_runtime rule to the top-level BUILD.bazel file as follows:

py_runtime(
    name = "my_python",
    files = [],
    interpreter = "pyenv/bin/python",
    visibility = ["//visibility:public"],
)

Also note: this works because I have a script in my repository that initializes the virtualenv and creates a symlink to it at the top level of my workspace. I just add pyenv to our .gitignore. That way the path can be relative to the workspace and not an absolute path.

[kgreenek]

The above work-around worked for my simple example, but when I tried to use it for real tests that depend on pip-installed packages, it could not find them.

We currently pip-install external dependencies in the virtualenv. So the virtualenv is how we control the python version as well as external deps.

I tried setting the files attribute for my py_runtime as follows, with no success:

py_runtime(
    name = "my_python",
    files = glob(["pyenv/**"]),
    interpreter = "pyenv/bin/python",
    visibility = ["//visibility:public"],
)

That throws a bunch of errors that make me think that's not what the files attribute is for.

So I'm stuck again. I need py_test to respect the PYTHONPATH so I can use a virtualenv.

[irengrig]

/cc @brandjon

[brandjon]

See #4080 and #2867 -- virtualenv works by manipulating PATH, and PATH behaves differently in py_binary vs py_test.

You should use py_runtime (and soon, toolchains; see #7375) to declare the specific interpreter you want Bazel to use, and ensure that this interpreter is the one that has your pip-installed packages. The easiest way to do this is to setup the virtualenv outside of Bazel, on the target platform, and point to it via the interpreter_path attribute (not interpreter) of py_runtime.

If you really want to invoke virtualenv from within your build, you should be able to do that, but instead of having the script emit a symlink in the source root, you should have it produce a symlink or wrapper script as its generated declared output, and reference the location of that output (via $(location) expansion) in the interpreter attribute. Manipulating the contents of the source root is not supported, and if you don't declare the symlink/wrapper as an output then Bazel doesn't know about it for sandboxing and incremental correctness purposes.

[brandjon]

Closing since I believe this issue is represented by #4080 and #2867, but please reopen if the py_runtime solution I described above has problems.