Split system packages into two sections: required to build python vs required to build users' python packages #7877
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
99c04c0 to
28ea367
Compare
28ea367 to
9775313
Compare
|
Nice work! Could the test failure be related? I've never seen it before. |
55d01b5 to
3413284
Compare
|
Hmm, that's tough to know, normally I'd assume yes, but the error message looks more like a flake network connection... lemme try running it again. For posterity, here's the failing run: https://github.com/dependabot/dependabot-core/actions/runs/5946994350/job/16128413270?pr=7877 |
jeffwidman
commented
Aug 23, 2023
Base automatically changed from
DRY-up-apt-installs-in-python-dockerfile
to
main
August 23, 2023 08:35
9775313 to
363999f
Compare
|
Looks like the CI failure was indeed a network flake, it worked fine the second time around. |
363999f to
736bc21
Compare
deivid-rodriguez
approved these changes
Aug 23, 2023
1. Reorder the list of package lexicographically 2. Assume the only packages needed for `pyenv` to install `python` are the ones listed in https://github.com/pyenv/pyenv/wiki#suggested-build-environment 3. Move the remaining packages to a separate section of packages that are installed in the final image, not the builder image (they're not needed for building the various python's) 4. Go through the list of packages that are _potentially_ used for user packages, and search the repo issues/PR's/commit log for any mentions. Also google to see if any common python libraries require them. If so, document why. 5. Delete the remaining packages, they're probably not needed. This should slightly shrink our build times/image size, but there is a risk that one or more of the removed libraries are actually needed. If so, we'll probably see it in Sentry error logs once we deploy and jobs start running against user repos, and then we can re-add the ones that are still needed, except this time we'll actually be able to document _why_ they're needed. I was a little surprised how many packages I was able to delete... some of them may be part of the existing base image, but many of them looked like things that may have been needed for other parts of the monolithic docker image, but are no longer needed for just the Python bits.
736bc21 to
1695861
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
pyenvto installpythonarethe ones listed in https://github.com/pyenv/pyenv/wiki#suggested-build-environment
are installed in the final image, not the builder image (they're not
needed for building the various python's)
packages, and search the repo issues/PR's/commit log for any
mentions. Also google to see if any common python libraries require
them. If so, document why.
This should slightly shrink our build times/image size, but there is a
risk that one or more of the removed libraries are actually needed. If
so, we'll probably see it in Sentry error logs once we deploy and jobs
start running against user repos, and then we can re-add the ones that
are still needed, except this time we'll actually be able to document
why they're needed.
I was a little surprised how many packages I was able to delete... 10 packages are gone!
Some of them may be part of the existing base image, but many of them looked
like things that may have been needed for other parts of the monolithic
docker image, but are no longer needed for just the Python bits.
Stacked on top of: