Skip to content

Commit

Permalink
Split packages into "building python" vs "building user packages" (#7877
Browse files Browse the repository at this point in the history
)

1. Reorder the list of package lexicographically
2. Assume the only packages needed for `pyenv` to install `python` are
   the ones listed in https://github.com/pyenv/pyenv/wiki#suggested-build-environment
3. Move the remaining packages to a separate section of packages that
   are installed in the final image, not the builder image (they're not
   needed for building the various python's)
4. Go through the list of packages that are _potentially_ used for user
   packages, and search the repo issues/PR's/commit log for any
   mentions. Also google to see if any common python libraries require
   them. If so, document why.
5. Delete the remaining packages, they're probably not needed.

This should slightly shrink our build times/image size, but there is a
risk that one or more of the removed libraries are actually needed. If
so, we'll probably see it in Sentry error logs once we deploy and jobs
start running against user repos, and then we can re-add the ones that
are still needed, except this time we'll actually be able to document
_why_ they're needed.

I was a little surprised how many packages I was able to delete... some
of them may be part of the existing base image, but many of them looked
like things that may have been needed for other parts of the monolithic
docker image, but are no longer needed for just the Python bits.
  • Loading branch information
jeffwidman authored Aug 23, 2023
1 parent 1e17026 commit 9cb48a8
Showing 1 changed file with 23 additions and 20 deletions.
43 changes: 23 additions & 20 deletions python/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -14,36 +14,24 @@ ARG PY_3_8
ARG PYENV_VERSION
USER root

# Install *only* the apt packages required for this builder image to build Python.
# C-libs needed by users to build their Python packages should be installed down below in the final docker image.
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y --no-install-recommends \
dirmngr \
zlib1g-dev \
liblzma-dev \
libgdbm-dev \
bison \
tzdata \
zip \
openssh-client \
build-essential \
make \
libpq-dev \
libssl-dev \
libbz2-dev \
libffi-dev \
liblzma-dev \
libncursesw5-dev \
libreadline-dev \
libsqlite3-dev \
libcurl4-openssl-dev \
llvm \
libncurses5-dev \
libncursesw5-dev \
libmysqlclient-dev \
xz-utils \
tk-dev \
libssl-dev \
libxml2-dev \
libxmlsec1-dev \
libgeos-dev \
python3-enchant \
tk-dev \
xz-utils \
zlib1g-dev \
&& rm -rf /var/lib/apt/lists/*

COPY --chown=dependabot:dependabot python/helpers /opt/python/helpers
Expand Down Expand Up @@ -82,6 +70,21 @@ COPY --from=python-3.10 /usr/local/.pyenv/3.10.tar.gz /usr/local/.pyenv/3.10.tar
COPY --from=python-3.9 /usr/local/.pyenv/3.9.tar.gz /usr/local/.pyenv/3.9.tar.gz
COPY --from=python-3.8 /usr/local/.pyenv/3.8.tar.gz /usr/local/.pyenv/3.8.tar.gz

# Install C-libs needed to build users' Python packages. Please document why each package is needed.
USER root
RUN apt-get update \
&& apt-get upgrade -y \
&& apt-get install -y --no-install-recommends \
# Used by pycurl
libcurl4-openssl-dev \
# Used by mysqlclient
libmysqlclient-dev \
# Used by psycopg Postgres Client
libpq-dev \
# Used by python zoneinfo core lib
tzdata \
&& rm -rf /var/lib/apt/lists/*

USER dependabot

COPY --chown=dependabot:dependabot python $DEPENDABOT_HOME/python
Expand Down

0 comments on commit 9cb48a8

Please sign in to comment.