pip install doesn't carry-on index-url certificate - failing build dependencies installation

[awaizman1]

Environment

• pip version: 20.2.3
• Python version: 3.6.8
• OS: Windows 10

Description

I'm using pip to install a package (iniconfig==1.1.0) from an internal secured repository, thus providing a certificate via command line args (--cert=...).
The package is distributed as a source dist (not a wheel) and pip tries first to install the package build dependencies (Installing build dependencies ...).
when it tries doing so, pip invokes a nested pip install command, carrying on the index-url but not the certificate, thus the installation fails.
Note how in the sub pip install command the index is carried on but the certificate is not:

ERROR: Command errored out with exit status 1:
   command: 'bug_venv\Scripts\python.exe' 'bug_venv\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\AWAIZM~1\AppData\Local\Temp\pip-build-env-cmo7_f0v\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://il-app-61/repository/pypi_group/simple -- 'setuptools>=41.2.0' wheel 'setuptools_scm>3'

Expected behavior

certificate argument should be carried on to sub pip install commands

How to Reproduce

1. create a new venv with pip 20.2.3, setuptools 50.3.1 and wheel 0.35.1

python -m venv bug_venv
bug_venv\Scripts\python.exe -m pip install pip==20.2.3
bug_venv\Scripts\python.exe -m pip install -U setuptools==50.3.1 wheel==0.35.1

2. run pip install command with internal repo and certificate

bug_venv\Scripts\python.exe -m pip install --cert=<certificate file> --index-url=<https repo url> iniconfig==1.1.0 

3. install should fail with "certificate verify failed"

Output

bug_venv\Scripts\python -m pip install --cert=F:\views\g\qprism\QBuildSystemCore\python\build_infra\src\amat_build_system\resources\amat.pem.cer --index-url=https://il-app-61/repository/pypi_group/simple iniconfig==1.1.0
Looking in indexes: https://il-app-61/repository/pypi_group/simple
Collecting iniconfig==1.1.0
  Downloading https://il-app-61/repository/pypi_group/packages/iniconfig/1.1.0/iniconfig-1.1.0.tar.gz (8.1 kB)
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: 'bug_venv\Scripts\python.exe' 'bug_venv\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\AWAIZM~1\AppData\Local\Temp\pip-build-env-cmo7_f0v\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://il-app-61/repository/pypi_group/simple -- 'setuptools>=41.2.0' wheel 'setuptools_scm>3'
       cwd: None
  Complete output (10 lines):
  Looking in indexes: https://il-app-61/repository/pypi_group/simple
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /repository/pypi_group/simple/setuptools/
  WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /repository/pypi_group/simple/setuptools/
  WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /repository/pypi_group/simple/setuptools/
  WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /repository/pypi_group/simple/setuptools/
  WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)': /repository/pypi_group/simple/setuptools/
  Could not fetch URL https://il-app-61/repository/pypi_group/simple/setuptools/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='il-app-61', port=443): Max retries exceeded with url: /repository/pypi_group/simple/setuptools/ (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)) - skipping
  ERROR: Could not find a version that satisfies the requirement setuptools>=41.2.0 (from versions: none)
  ERROR: No matching distribution found for setuptools>=41.2.0
  Could not fetch URL https://il-app-61/repository/pypi_group/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='il-app-61', port=443): Max retries exceeded with url: /repository/pypi_group/simple/pip/ (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)'),)) - skipping
  ----------------------------------------
ERROR: Command errored out with exit status 1: 'bug_venv\Scripts\python.exe' 'bug_venv\lib\site-packages\pip' install --ignore-installed --no-user --prefix 'C:\Users\AWAIZM~1\AppData\Local\Temp\pip-build-env-cmo7_f0v\overlay' --no-warn-script-location --no-binary :none: --only-binary :none: -i https://il-app-61/repository/pypi_group/simple -- 'setuptools>=41.2.0' wheel 'setuptools_scm>3' Check the logs for full command output.

[pradyunsg]

Is this related to pypi/warehouse#8706?

[awaizman1]

@pradyunsg it is not related to 8706 issue

[pfmoore]

This sounds more like some pip options not being carried over to the isolated build environment.

[awaizman1]

Hi @pradyunsg

This issue happens because only parts of repo-related arguments are carry over to pip subprocess during the creation of build environment.
when a user passes an index-url with a certificate both should be used by any subcall to pip.
Currently only the index-url is carry over but not he cert so it doomed to fail.

I guess the network connectivity tag is misleading here

thanks.

[pfmoore]

I guess the network connectivity tag is misleading here

Agreed. I've changed it to "PEP 517 impact" which isn't entirely accurate, I thought we had a "Build isolation" tag, but either I'm misremembering or it's been removed. PEP 517 isn't specifically build isolation, but it's the closest I could find.

[awaizman1]

Hi @pfmoore @pradyunsg ,

Kindly would like to know if there is some fix planned for this or some workaround I can use.

Thanks

[pfmoore]

No fix currently planned/available, I'm afraid. As usual PRs would be welcome (but it's not a simple fix!)

As a workaround, you could disable build isolation and manage the build environment manually.

[ichard26]

This is a duplicate of #5502 which was fixed by #13063.