Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust compiler options used to build Python #1566

Merged
merged 1 commit into from
Apr 16, 2024
Merged

Adjust compiler options used to build Python #1566

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## [Unreleased]

- Adjusted compiler options used to build Python for improved parity with the Docker Hub Python images. ([#1566](https://github.com/heroku/heroku-buildpack-python/pull/1566))
- Excluded `LD_LIBRARY_PATH` and `PYTHONHOME` app config vars when invoking subprocesses during the build. ([#1565](https://github.com/heroku/heroku-buildpack-python/pull/1565))

## [v248] - 2024-04-09
Expand Down
30 changes: 26 additions & 4 deletions builds/build_python_runtime.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@ set -euo pipefail
PYTHON_VERSION="${1:?"Error: The Python version to build must be specified as the first argument."}"
PYTHON_MAJOR_VERSION="${PYTHON_VERSION%.*}"

INSTALL_DIR="/app/.heroku/python"
# Python is relocated to different locations by the classic buildpack and CNB (which works since we
# set `LD_LIBRARY_PATH` and `PYTHONHOME` appropriately at build/run-time), so for packaging purposes
# we install Python into an arbitrary location that intentionally matches neither location.
INSTALL_DIR="/tmp/python"
SRC_DIR="/tmp/src"
UPLOAD_DIR="/tmp/upload/${STACK}/runtimes"

Expand Down Expand Up @@ -82,6 +85,10 @@ cd "${SRC_DIR}"
# for maximum compatibility / most battle-tested build configuration:
# https://github.com/docker-library/python
CONFIGURE_OPTS=(
# Explicitly set the target architecture rather than auto-detecting based on the host CPU.
# This only affects targets like i386 (for which we don't build), but we pass it anyway for
# completeness and parity with the Python Docker image builds.
"--build=$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"
# Support loadable extensions in the `_sqlite` extension module.
"--enable-loadable-sqlite-extensions"
# Enable recommended release build performance optimisations such as PGO.
Expand Down Expand Up @@ -133,11 +140,26 @@ fi

./configure "${CONFIGURE_OPTS[@]}"

# Using LDFLAGS we instruct the linker to omit all symbol information from the final binary
# `-Wl,--strip-all` instructs the linker to omit all symbol information from the final binary
# and shared libraries, to reduce the size of the build. We have to use `--strip-all` and
# not `--strip-unneeded` since `ld` only understands the former (unlike the `strip` command),
# however it's safe to use since these options don't apply to static libraries.
make -j "$(nproc)" LDFLAGS='-Wl,--strip-all'
# however, `--strip-all` is safe to use since LDFLAGS doesn't apply to static libraries.
# `dpkg-buildflags` returns the distro's default compiler/linker options, which enable various
# security/hardening best practices. See:
# - https://wiki.ubuntu.com/ToolChain/CompilerFlags
# - https://wiki.debian.org/Hardening
# - https://github.com/docker-library/python/issues/810
# We only use `dpkg-buildflags` for Python versions where we build in shared mode (Python 3.9+),
# since some of the options it enables interferes with the stripping of static libraries.
if [[ "${PYTHON_MAJOR_VERSION}" == 3.[8-9] ]]; then
EXTRA_CFLAGS=''
LDFLAGS='-Wl,--strip-all'
else
EXTRA_CFLAGS="$(dpkg-buildflags --get CFLAGS)"
LDFLAGS="$(dpkg-buildflags --get LDFLAGS) -Wl,--strip-all"
fi

make -j "$(nproc)" "EXTRA_CFLAGS=${EXTRA_CFLAGS}" "LDFLAGS=${LDFLAGS}"
make install

if [[ "${PYTHON_MAJOR_VERSION}" == 3.[8-9] ]]; then
Expand Down