Update pip to 20.1.1 #1005
Assignees
Comments
edmorley
added a commit
that referenced
this issue
Jul 23, 2020
The versions installed by the buildpack have been updated as follows: * pip: - If using Python 3.4: No change (already using the last to support 3.4) - If using pipenv: No change (need to update to a newer pipenv first) - For everything else: `20.0.2` -> `20.1.1` * setuptools: - If using Python 3.4: `39.0.1` -> `43.0.0` (latest for 3.4) - If using Python 2.7: `39.0.1` -> `44.1.1` (latest for 2.7) - For everything else: `39.0.1` -> `47.1.1` (until #1006 fixed) * wheel: - If using Python 3.4: `unpinned` -> `0.33.6` - For everything else: `unpinned` -> `0.34.2` This fixes #949 and fixes #1005, and means packages that rely on newer setuptools will now install successfully. Changelogs: https://pip.pypa.io/en/stable/news/ https://setuptools.readthedocs.io/en/latest/history.html#v47-1-1 https://wheel.readthedocs.io/en/latest/news.html In addition: * Installed versions are now deterministic (fixes #1000, fixes #1003) * The build output now includes the versions used, making it easier to debug future upgrades (closes #939) * Errors during pip/setuptools/wheel install now correctly fail the build, and stderr is no longer sent to `/dev/null` (fixes #1002) * Setuptools is no longer installed twice (fixes #1001) * Everything that is downloaded is now used (fixes #999) * `--no-cache` and `--disable-version-check` are now used, saving unnecessary work and preventing creation of unwanted files in `/app` * The `PIP_UPDATE` env var no longer leaks into subprocesses. As part of fixing version pinning, we now use pip itself to determine whether the installed packages are up to date, since parsing pip's output is fragile (eg #1003). This means `pip install` is now called every time, however this is a no-op for repeat builds where the versions have not changed, since unless `--upgrade` is specified pip does not hit the index (PyPI) if requirements are satisfied. For the installation itself `get-pip.py` is no longer used, since: - It uses `--force-reinstall`, which is unnecessary here and would slow down repeat builds (given we call pip install every time now). Trying to work around this by using `get-pip.py` only for the initial install, and real pip for subsequent updates would mean we lose protection against cached broken installs, plus significantly increase the version combinations test matrix. - It means downloading pip twice (once embedded in `get-pip.py`, and again during the install, since `get-pip.py` can't install the embedded version directly) - We would still have to manage several versions of get-pip.py, to support older Pythons. We don't use `ensurepip` since: - Not all of the previously generated Python runtimes on S3 include it - We would still have to upgrade pip afterwards - The versions of pip/setuptools bundled with ensurepip differ greatly depending on Python version, and we could easily start using a CLI flag for the first pip install before upgrade that isn't supported on all versions, without even knowing it (unless we test against hundreds of Python archives). The new pip wheel assets on S3 were generated using: ``` $ pip download --no-cache pip==19.1.1 Collecting pip==19.1.1 Downloading pip-19.1.1-py2.py3-none-any.whl (1.4 MB) Saved ./pip-19.1.1-py2.py3-none-any.whl Successfully downloaded pip $ pip download --no-cache pip==20.1.1 Collecting pip==20.1.1 Downloading pip-20.1.1-py2.py3-none-any.whl (1.5 MB) Saved ./pip-20.1.1-py2.py3-none-any.whl Successfully downloaded pip $ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read --dryrun (dryrun) upload: ./pip-19.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-19.1.1-py2.py3-none-any.whl (dryrun) upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl $ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read upload: ./pip-19.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-19.1.1-py2.py3-none-any.whl upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl ```
edmorley
added a commit
that referenced
this issue
Aug 3, 2020
Updates pip from 20.0.2 to 20.1.1 for Python 2.7 and Python 3.5+: https://pip.pypa.io/en/stable/news/#id40 The version used for Python 3.4 remains unchanged at 19.1.1, since it's the last version of pip that supports it. Pip has been updated to 20.1.1 rather than the recently released 20.2, since the latter has a few regressions and even though these will be fixed shortly in 20.2.1, we should let the changes soak for longer before picking them up. The `PIP_NO_PYTHON_VERSION_WARNING` environment variable has been set (equivalent to passing `--no-python-version-warning`) to prevent the Python 2.7 EOL warnings added in pip 20.1 from spamming the build log: https://github.com/pypa/pip/blob/20.1.1/src/pip/_internal/cli/base_command.py#L139-L154 This was set via environment variable rather than CLI flag, since: * otherwise we'd have to pass it to every pip invocation * older pip (such as the 19.1.1 used by Python 3.4) doesn't support this option and would error out due to an unknown CLI flag being passed, unless we added conditional flags throughout. The new pip wheel was uploaded to S3 using: ``` $ pip download --no-cache pip==20.1.1 Collecting pip==20.1.1 Downloading pip-20.1.1-py2.py3-none-any.whl (1.5 MB) Saved ./pip-20.1.1-py2.py3-none-any.whl Successfully downloaded pip $ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read --dryrun (dryrun) upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl $ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl ``` Fixes #1005. @W-7659489@
Merged
edmorley
added a commit
that referenced
this issue
Aug 3, 2020
Updates pip from 20.0.2 to 20.1.1 for Python 2.7 and Python 3.5+: https://pip.pypa.io/en/stable/news/#id40 The version used for Python 3.4 remains unchanged at 19.1.1, since it's the last version of pip that supports it. Pip has been updated to 20.1.1 rather than the recently released 20.2, since the latter has a few regressions and even though these will be fixed shortly in 20.2.1, we should let the changes soak for longer before picking them up. The `PIP_NO_PYTHON_VERSION_WARNING` environment variable has been set (equivalent to passing `--no-python-version-warning`) to prevent the Python 2.7 EOL warnings added in pip 20.1 from spamming the build log: https://github.com/pypa/pip/blob/20.1.1/src/pip/_internal/cli/base_command.py#L139-L154 This was set via environment variable rather than CLI flag, since: * otherwise we'd have to pass it to every pip invocation * older pip (such as the 19.1.1 used by Python 3.4) doesn't support this option and would error out due to an unknown CLI flag being passed, unless we added conditional flags throughout. The new pip wheel was uploaded to S3 using: ``` $ pip download --no-cache pip==20.1.1 Collecting pip==20.1.1 Downloading pip-20.1.1-py2.py3-none-any.whl (1.5 MB) Saved ./pip-20.1.1-py2.py3-none-any.whl Successfully downloaded pip $ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read --dryrun (dryrun) upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl $ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl ``` Fixes #1005. @W-7659489@
dryan
pushed a commit
to dryan/heroku-buildpack-python
that referenced
this issue
Nov 19, 2020
Updates pip from 20.0.2 to 20.1.1 for Python 2.7 and Python 3.5+: https://pip.pypa.io/en/stable/news/#id40 The version used for Python 3.4 remains unchanged at 19.1.1, since it's the last version of pip that supports it. Pip has been updated to 20.1.1 rather than the recently released 20.2, since the latter has a few regressions and even though these will be fixed shortly in 20.2.1, we should let the changes soak for longer before picking them up. The `PIP_NO_PYTHON_VERSION_WARNING` environment variable has been set (equivalent to passing `--no-python-version-warning`) to prevent the Python 2.7 EOL warnings added in pip 20.1 from spamming the build log: https://github.com/pypa/pip/blob/20.1.1/src/pip/_internal/cli/base_command.py#L139-L154 This was set via environment variable rather than CLI flag, since: * otherwise we'd have to pass it to every pip invocation * older pip (such as the 19.1.1 used by Python 3.4) doesn't support this option and would error out due to an unknown CLI flag being passed, unless we added conditional flags throughout. The new pip wheel was uploaded to S3 using: ``` $ pip download --no-cache pip==20.1.1 Collecting pip==20.1.1 Downloading pip-20.1.1-py2.py3-none-any.whl (1.5 MB) Saved ./pip-20.1.1-py2.py3-none-any.whl Successfully downloaded pip $ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read --dryrun (dryrun) upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl $ aws s3 sync . s3://lang-python/common/ --exclude "*" --include "*.whl" --acl public-read upload: ./pip-20.1.1-py2.py3-none-any.whl to s3://lang-python/common/pip-20.1.1-py2.py3-none-any.whl ``` Fixes heroku#1005. @W-7659489@
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
For standard (non-pipenv) Python 2.7 and 3.5+ installs the buildpack currently defaults to pip 20.0.2:
heroku-buildpack-python/bin/compile
Line 67 in 156b07c
The latest pip is 20.1.1 - we should update to pick up the fixes/features since 20.0.2:
https://pip.pypa.io/en/stable/news/#id1
The text was updated successfully, but these errors were encountered: