Windows support and exposing interpreters

[spietras]

I changed a lot actually. So let's just get into it:

• pyenv-win has a slightly different repository structure than pyenv. The difference is that pyenv-win is in a directory called pyenv-win, while pyenv has everything in root. So we can just add that directory to strip_prefix.

• There are some differences between Linux and Windows installations:

  • pyenv executable: pyenv.bat on Windows, pyenv on Linux
  • pyenv options: pyenv is quiet by default, for pyenv-win to be quiet we need to pass the -q option
  • Directory structure: on Windows you can find the interpreter in versions/{version}/python.exe, while on Linux it's versions/{version}/bin/python.
  • Interpreter name: python.exe on Windows, python on Linux

  I added configurations as a dictionary to the top of the .bzl file, so it can be easily updated if something changes. The logic uses values from the configuration of the current system (we can get the system from repository_ctx).

• Exposed the interpreters as files. This can be useful because some repository rules could use the interpreter for some things. For example, see this PR from rules_python. After that rules_pyenv can be used alongside rules_python for completely hermetic Python workflow (excluding the dependencies to build Python itself and, unfortunately, any other system Python on Linux).

  We register the interpreters as a toolchain, but a toolchain can't be used by repository rules. But we can just expose the interpreter files. This is a little problematic, because of a different directory structure, different names. Also, we would not like users to refer to the file by the full path with the python version (versions/3.7.5/bin/python). We can solve all these problems with symlinks.

  We have two different versions of python, so we can't just copy files to the root (of pyenv workspace), because there would be conflicts. So each version should be in at least one separate directory. We could create two symlinks at the root, pointing to the interpreters, but it doesn't work. For some reason, if the interpreter is executed as a symlink in the root directory, it doesn't see all the other necessary files in the true directory. I have tested a lot of different approaches and I present what I think is the optimal one.

  First, we should create a common name for python interpreters. python is good and it already exists on Linux. For Windows, a symlink called python is created in the interpreter directory, pointing to python.exe. For clarity: versions/{version}/python -> versions/{version}/python.exe

  Then, we need to create two symlinks (called py2 and py3 by default) in the root directory, pointing to corresponding true interpreter directories. That way the interpreter will see all the files in the true directory. For clarity: py3 -> versions/{version} on Windows, py3 -> versions/{version}/bin on Linux.

  Now we can expose the interpreters in the generated BUILD file and users can refer to them for example as @pyenv//:py3/python.

  On Windows, you need to have symlinks enabled for this to work.

• Added the ability to specify in pyenv_install:

  • interpreters workspace name (default: pyenv)
  • symlink names (default: py2 and py3)
  • pyenv and pyenv-win repositories (defaults are provided by us and by the way, I updated the pyenv-win repo to the newest version)

• Updated README with new information.

I have tested it on Windows and on Linux. I must admit that this together with python_rules (or python_rules_external) allows for a pretty nice workflow.

And of course, thank you for this repository. It made a lot of things easier. It's awesome to be finally able to be somewhat hermetic with Python and Bazel 💚

[ddlees]

I'm glad hear you're finding this to be as useful as I have; The non-hermeticity of Bazel + python was driving me bonkers. Thank you for your contribution and apologies for taking this long to review and merge it. 😄