Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Python 3.14 on macOS and GNU Linux #443

Merged
merged 21 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
3985547
Add support for Python 3.14 on GNU Linux
zanieb Dec 18, 2024
cd9dbfd
Only apply Apple cross-compile patches on macOS
zanieb Dec 18, 2024
dc6a50f
Apply 3.13 apple-cross patch to 3.14
zanieb Dec 18, 2024
6ec00cf
Fix "runshared" patch on 3.14
zanieb Dec 18, 2024
2094ced
Drop `_blake2` from expected extension modules
zanieb Dec 18, 2024
0e5f479
Use `meets_python_minimum_version` for free-threaded safety check
zanieb Dec 18, 2024
d5131a5
Add 3.14 to validation
zanieb Dec 18, 2024
871c190
Enable 3.14 for macOS builds
zanieb Dec 18, 2024
7869c0a
Fix libc filtering in the matrix generator
zanieb Dec 19, 2024
de2aea3
Add multiple path support to `includes-conditional` and `sources-cond…
zanieb Dec 19, 2024
76d9da1
Add min/max-python-version support to `links-conditional`
zanieb Dec 19, 2024
dd15340
Split `_blake` extension configuratoin for 3.14
zanieb Dec 19, 2024
33b1bee
Bump minimum macOS version from 10.9 -> 10.15 on x86_64
zanieb Dec 19, 2024
504468a
Add `_BSD_SOURCE` and `_DEFAULT_SOURCE`
zanieb Dec 19, 2024
9dd6bd5
Set `ac_cv_func_explicit_bzero=no`
zanieb Dec 19, 2024
eaa5ab9
Fix cross-compilation for macOS x86_64
zanieb Dec 19, 2024
e1c604b
Set `LINUX_NO_EXPLICIT_BZERO` for `_blake` separately
zanieb Dec 19, 2024
9e61d1f
Update expected macosx target and fix patch
zanieb Dec 19, 2024
5fdcd3d
Ignore `test_strftime_y2k` when cross-compiling on Linux for 3.14+
zanieb Dec 19, 2024
2f4fa03
Remove 3.14 cross-path — it matches 3.13
zanieb Dec 19, 2024
7055071
Restore `_blake2` to global expected extension modules
zanieb Dec 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion ci-matrix.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,11 @@ def should_include_entry(entry: dict[str, str], filters: dict[str, set[str]]) ->
if filters.get("arch") and entry["arch"] not in filters["arch"]:
return False

if filters.get("libc") and entry.get("libc") not in filters["libc"]:
if (
filters.get("libc")
and entry.get("libc")
and entry["libc"] not in filters["libc"]
):
Comment on lines +74 to +78
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a bug that disabled the macOS builds in the matrix if libc labels were requested.

return False

if filters.get("build"):
Expand Down
11 changes: 11 additions & 0 deletions ci-targets.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ darwin:
- "3.11"
- "3.12"
- "3.13"
- "3.14"
build_options:
- debug
- pgo+lto
Expand All @@ -26,6 +27,7 @@ darwin:
- "3.11"
- "3.12"
- "3.13"
- "3.14"
build_options:
- debug
- pgo+lto
Expand All @@ -45,6 +47,7 @@ linux:
- "3.11"
- "3.12"
- "3.13"
- "3.14"
build_options:
- debug
- noopt
Expand All @@ -65,6 +68,7 @@ linux:
- "3.11"
- "3.12"
- "3.13"
- "3.14"
build_options:
- debug
- noopt
Expand All @@ -85,6 +89,7 @@ linux:
- "3.11"
- "3.12"
- "3.13"
- "3.14"
build_options:
- debug
- noopt
Expand All @@ -105,6 +110,7 @@ linux:
- "3.11"
- "3.12"
- "3.13"
- "3.14"
build_options:
- debug
- noopt
Expand All @@ -125,6 +131,7 @@ linux:
- "3.11"
- "3.12"
- "3.13"
- "3.14"
build_options:
- debug
- noopt
Expand All @@ -145,6 +152,7 @@ linux:
- "3.11"
- "3.12"
- "3.13"
- "3.14"
build_options:
- debug
- pgo+lto
Expand All @@ -165,6 +173,7 @@ linux:
- "3.11"
- "3.12"
- "3.13"
- "3.14"
build_options:
- debug
- pgo+lto
Expand All @@ -185,6 +194,7 @@ linux:
- "3.11"
- "3.12"
- "3.13"
- "3.14"
build_options:
- debug
- pgo+lto
Expand All @@ -205,6 +215,7 @@ linux:
- "3.11"
- "3.12"
- "3.13"
- "3.14"
build_options:
- debug
- noopt
Expand Down
2 changes: 1 addition & 1 deletion cpython-unix/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ BUILD := $(HERE)/build.py
NULL :=
SPACE := $(subst ,, )

ALL_PYTHON_VERSIONS := 3.9 3.10 3.11 3.12 3.13
ALL_PYTHON_VERSIONS := 3.9 3.10 3.11 3.12 3.13 3.14

ifndef PYBUILD_TARGET_TRIPLE
$(error PYBUILD_TARGET_TRIPLE not defined)
Expand Down
30 changes: 23 additions & 7 deletions cpython-unix/build-cpython.sh
Original file line number Diff line number Diff line change
Expand Up @@ -70,12 +70,14 @@ cat Makefile.extra
pushd Python-${PYTHON_VERSION}

# configure doesn't support cross-compiling on Apple. Teach it.
if [ "${PYTHON_MAJMIN_VERSION}" = "3.13" ]; then
patch -p1 -i ${ROOT}/patch-apple-cross-3.13.patch
elif [ "${PYTHON_MAJMIN_VERSION}" = "3.12" ]; then
patch -p1 -i ${ROOT}/patch-apple-cross-3.12.patch
else
patch -p1 -i ${ROOT}/patch-apple-cross.patch
if [ "${PYBUILD_PLATFORM}" = "macos" ]; then
if [ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_13}" ]; then
patch -p1 -i ${ROOT}/patch-apple-cross-3.13.patch
elif [ "${PYTHON_MAJMIN_VERSION}" = "3.12" ]; then
patch -p1 -i ${ROOT}/patch-apple-cross-3.12.patch
else
patch -p1 -i ${ROOT}/patch-apple-cross.patch
fi
fi

# This patch is slightly different on Python 3.10+.
Expand All @@ -94,7 +96,9 @@ fi
# Configure nerfs RUNSHARED when cross-compiling, which prevents PGO from running when
# we can in fact run the target binaries (e.g. x86_64 host and i686 target). Undo that.
if [ -n "${CROSS_COMPILING}" ]; then
if [ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_13}" ]; then
if [ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_14}" ]; then
patch -p1 -i ${ROOT}/patch-dont-clear-runshared-14.patch
elif [ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_13}" ]; then
patch -p1 -i ${ROOT}/patch-dont-clear-runshared-13.patch
elif [ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_11}" ]; then
patch -p1 -i ${ROOT}/patch-dont-clear-runshared.patch
Expand Down Expand Up @@ -471,6 +475,18 @@ if [ "${PYBUILD_PLATFORM}" = "macos" ]; then
CONFIGURE_FLAGS="${CONFIGURE_FLAGS} ac_cv_func_ptsname_r=no"
fi

# explicit_bzero is only available in glibc 2.25+, but we target a lower version for compatibility.
# it's only needed for the HACL Blake2 implementation in Python 3.14+
if [ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_14}" ]; then
CONFIGURE_FLAGS="${CONFIGURE_FLAGS} ac_cv_func_explicit_bzero=no"
fi

# On 3.14+ `test_strftime_y2k` fails when cross-compiling for `x86_64_v2` and `x86_64_v3` targets on
# Linux, so we ignore it. See https://github.com/python/cpython/issues/128104
if [[ -n "${PYTHON_MEETS_MINIMUM_VERSION_3_14}" && -n "${CROSS_COMPILING}" && "${PYBUILD_PLATFORM}" != "macos" ]]; then
export PROFILE_TASK='-m test --pgo --ignore test_strftime_y2k'
fi

# We use ndbm on macOS and BerkeleyDB elsewhere.
if [ "${PYBUILD_PLATFORM}" = "macos" ]; then
CONFIGURE_FLAGS="${CONFIGURE_FLAGS} --with-dbmliborder=ndbm"
Expand Down
6 changes: 5 additions & 1 deletion cpython-unix/build-main.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
import subprocess
import sys

from pythonbuild.cpython import meets_python_minimum_version
from pythonbuild.downloads import DOWNLOADS
from pythonbuild.utils import (
compress_python_archive,
Expand Down Expand Up @@ -68,6 +69,7 @@ def main():
"cpython-3.11",
"cpython-3.12",
"cpython-3.13",
"cpython-3.14",
},
default="cpython-3.11",
help="Python distribution to build",
Expand Down Expand Up @@ -164,7 +166,9 @@ def main():
release_tag = release_tag_from_git()

# Guard against accidental misuse of the free-threaded flag with older versions
if "freethreaded" in args.options and python_majmin not in ("3.13",):
zanieb marked this conversation as resolved.
Show resolved Hide resolved
if "freethreaded" in args.options and not meets_python_minimum_version(
python_majmin, "3.13"
):
print(
"Invalid build option: 'freethreaded' is only compatible with CPython 3.13+ (got %s)"
% cpython_version
Expand Down
5 changes: 3 additions & 2 deletions cpython-unix/build.py
Original file line number Diff line number Diff line change
Expand Up @@ -465,7 +465,7 @@ def build_cpython_host(

# Set environment variables allowing convenient testing for Python
# version ranges.
for v in ("3.9", "3.10", "3.11", "3.12", "3.13"):
for v in ("3.9", "3.10", "3.11", "3.12", "3.13", "3.14"):
normal_version = v.replace(".", "_")

if meets_python_minimum_version(python_version, v):
Expand Down Expand Up @@ -801,7 +801,7 @@ def build_cpython(

# Set environment variables allowing convenient testing for Python
# version ranges.
for v in ("3.9", "3.10", "3.11", "3.12", "3.13"):
for v in ("3.9", "3.10", "3.11", "3.12", "3.13", "3.14"):
normal_version = v.replace(".", "_")

if meets_python_minimum_version(python_version, v):
Expand Down Expand Up @@ -1223,6 +1223,7 @@ def main():
"cpython-3.11",
"cpython-3.12",
"cpython-3.13",
"cpython-3.14",
):
build_cpython(
settings,
Expand Down
37 changes: 33 additions & 4 deletions cpython-unix/extension-modules.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,36 @@ _bisect:
- _bisectmodule.c

_blake2:
sources:
- _blake2/blake2module.c
- _blake2/blake2b_impl.c
- _blake2/blake2s_impl.c
# In 3.14+, Blake2 is provided by Hacl*
sources-conditional:
- sources:
- _blake2/blake2module.c
- _blake2/blake2b_impl.c
- _blake2/blake2s_impl.c
maximum-python-version: "3.13"
- sources:
- blake2module.c
- _hacl/Hacl_Hash_Blake2s.c
- _hacl/Hacl_Hash_Blake2b.c
- _hacl/Lib_Memzero0.c
minimum-python-version: "3.14"
includes-conditional:
- includes:
- Modules/_hacl
- Modules/_hacl/include
- Modules/_hacl/internal
minimum-python-version: "3.14"
links-conditional:
- name: :libHacl_Hash_Blake2.a
minimum-python-version: "3.14"
defines-conditional:
- define: _BSD_SOURCE
minimum-python-version: "3.14"
- define: _DEFAULT_SOURCE
minimum-python-version: "3.14"
# Disable `explicit_bzero`, it requires glib 2.25+
- define: LINUX_NO_EXPLICIT_BZERO
minimum-python-version: "3.14"

_bz2:
sources:
Expand Down Expand Up @@ -355,6 +381,9 @@ _multiprocessing:
_opcode:
sources:
- _opcode.c
setup-enabled-conditional:
- enabled: true
minimum-python-version: "3.14"

_operator:
setup-enabled: true
Expand Down
15 changes: 15 additions & 0 deletions cpython-unix/patch-dont-clear-runshared-14.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
diff --git a/configure.ac b/configure.ac
index bd0221481c5..f2fb52c1efc 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1605,10 +1605,6 @@ else # shared is disabled
fi
AC_MSG_RESULT([$LDLIBRARY])

-if test "$cross_compiling" = yes; then
- RUNSHARED=
-fi
-
# HOSTRUNNER - Program to run CPython for the host platform
AC_MSG_CHECKING([HOSTRUNNER])
if test -z "$HOSTRUNNER"
Loading
Loading