PEP 518 build dependency installs don't respect --cert command line option

[skylerbunny]

Environment

• pip version: 10.0.1
• Python version: Python 2.7.13
• OS: Debian Stretch (latest)

Description
pip ignores provided --cert options on the command line when it makes an installation attempt on a library that has a pyproject.toml file, meaning PEP 518 build dependencies kick in.

(Note: Whether or not attrs has a valid toml file because there's no [build-system] section present is technically irrelevant, because the failure I'm reporting is related to how pip tries to pull what it derives, not what pip derives to pull.)

Expected behavior
When pip attempts to install PEP 518 build dependencies, it should use any provided --cert or --client-cert option. I tested that it doesn't use --cert. I would guess it probably doesn't use the related --client-cert either.

How to Reproduce

1. Create a virtualenv to isolate what you're going to be doing.
2. Create/use a devpi repository with a self-signed certificate in front of it, so the problem can be seen. (Alternatively, I suppose, you could probably purposely move your verify bundle to a nonstandard location, then explicitly specify 'that file' with the --cert command line option even if you used something like https://pypi.python.org/.)
3. Run pip download --no-binary :all: --index-url https://my-devpi-server/myindex/+simple/ --cert /etc/ssl/certs/ca-certificates.crt attrs
4. The source file will successfully download. Then it will attempt to install build dependencies, which fails on an CERTIFICATE_VERIFY_FAILED error.

Output

(Note that my-index has root/pypi as an index parent, so that redirection would be expected.)

(temp) wheel-builder@build1:~/temp$ pip download --no-binary :all: --index-url https://my-devpi-server/my-index/+simple/ --cert /etc/ssl/certs/ca-certificates.crt attrs
Looking in indexes: https://my-devpi-server/my-index/+simple/
Collecting attrs
  Downloading https://my-devpi-server/root/pypi/+f/e0d/0eb91441a3b53/attrs-18.1.0.tar.gz (106kB)
  Saved ./attrs-18.1.0.tar.gz
  Installing build dependencies ... error
  Complete output from command /var/lib/wheel-builder/temp/bin/python2 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-QsBWnb https://my-devpi-server/root/pypi/+f/8fc/a9275c89964f1/setuptools-39.2.0-py2.py3-none-any.whl#sha256=8fca9275c89964f13da985c3656cb00ba029d7f3916b37990927ffdf264e7926 https://my-devpi-server/root/pypi/+f/800/44e51ec5bbf6c/wheel-0.31.1-py2.py3-none-any.whl#sha256=80044e51ec5bbf6c894ba0bc48d26a8c20a9ba629f4ca19ea26ecfcf87685f5f:
  Collecting setuptools==39.2.0 from https://my-devpi-server/root/pypi/+f/8fc/a9275c89964f1/setuptools-39.2.0-py2.py3-none-any.whl#sha256=8fca9275c89964f13da985c3656cb00ba029d7f3916b37990927ffdf264e7926
    Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)'),)': /root/pypi/+f/8fc/a9275c89964f1/setuptools-39.2.0-py2.py3-none-any.whl
(((Repeats four more times)))
  Could not install packages due to an EnvironmentError: HTTPSConnectionPool(host='my-devpi-server', port=443): Max retries exceeded with url: /root/pypi/+f/8fc/a9275c89964f1/setuptools-39.2.0-py2.py3-none-any.whl (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)'),))


  ----------------------------------------
Command "/var/lib/wheel-builder/temp/bin/python2 -m pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-QsBWnb https://my-devpi-server/root/pypi/+f/8fc/a9275c89964f1/setuptools-39.2.0-py2.py3-none-any.whl#sha256=8fca9275c89964f13da985c3656cb00ba029d7f3916b37990927ffdf264e7926 https://my-devpi-server/root/pypi/+f/800/44e51ec5bbf6c/wheel-0.31.1-py2.py3-none-any.whl#sha256=80044e51ec5bbf6c894ba0bc48d26a8c20a9ba629f4ca19ea26ecfcf87685f5f" failed with error code 1 in None

[skylerbunny]

I'm guessing this problem is here:

pip/src/pip/_internal/build_env.py

Line 79 in 77f6ac6

def install_requirements(self, finder, requirements, message):

(Which is to say that --cert nor --client-cert are checked for nor appended in this use case.)

[benoit-pierre]

Indeed, session specific options are not being forwarded.

[adamgreg]

This is still the case in 19.2.2

[onlineque]

Still the case with 20.0.2. Is there a chance that this bug will be fixed ? It effectively prevents me using local PYPI repository with HTTPS due to that and I do not want to workaround this bug with ugly --trusted-host

[uranusjr]

IIUC the fix to this issue should not be too technically involved—locate where the PEP 517 hook installs build dependencies, and pass the flags to them (likely via environment variables for better compatibility). The biggest problem would be to find someone interested enough in this to do the necessary code-tracing and patching, which is still significant but boring work. Feel free to give it a shot if you’re inclined to.

[chrahunt]

Anyone that needs this behavior now should be able to work around it by specifying their cert path in the PIP_CERT environment variable or specifying cert in one of the pip configuration files.

[thibmonsel]

This seems to reappear in pip==23.0.1 with the opened issue : jax-ml/jax#15387.

Solution :
By adding --no-build-isolation in the CLI did the trick for me.
eg
python3 -m pip install --proxy=MYPROXY MYPACKAGE --no-build-isolation
(I had python3.8 pip==23.0.1)

[SeaQuench]

Anyone that needs this behavior now should be able to work around it by specifying their cert path in the PIP_CERT environment variable or specifying cert in one of the pip configuration files.

This appears to have been inverted for build 23.1.2 (likely earlier):
PIP_CERT is now ignored but --cert is working! 🙃

[jle-pass]

+1
Hi, dependence with #11476 ?

[ichard26]

Hi all,

I have a proposed PR to fix this: #13063. It's ready to land, but I'd like to have some real world testing before I press merge. I'd appreciate anyone with a real world scenario that hits this bug to test my changes. It'd be helpful in ensuring that A) it works, and B) it doesn't break in whatever weird enterprise TLS setups that exist in the wild. You can install pip from the PR branch using pip install https://github.com/ichard26/pip/archive/pass-certs-down.zip.

Thanks!