Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

numpy 2.0.0rc migration #5790

Merged
merged 10 commits into from
May 4, 2024
57 changes: 57 additions & 0 deletions recipe/migrations/numpy2.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
__migrator:
build_number: 1
kind: version
commit_message: |
Rebuild for numpy 2.0

As of version 2.0, the way we build against numpy has changed. In particular,
we no longer need to use the oldest available numpy version at build time
in order to support old numpy version at runtime - numpy will by default
use a compatible ABI for the oldest still-supported numpy versions.

Additionally, we no longer need to use `{{ pin_compatible("numpy") }}` as a
run requirement - this has been handled for more than two years now by a
run-export on the numpy package itself. The migrator will therefore remove
any occurrences of this.

However, by default, building against numpy 2.0 will assume that the package
is compatible with numpy 2.0, which is not necessarily the case. You should
check that the upstream package explicitly supports numpy 2.0, otherwise you
need to add a `- numpy <2` run requirement until that happens (check numpy
issue 26191 for an overview of the most important packages).

h-vetinari marked this conversation as resolved.
Show resolved Hide resolved
Finally, numpy 2.0 has not yet been released, but there is a release candidate
that promises to be ABI-compatible with the final release. This means we can
build against numpy 2.0.0rc1 (which can only be installed using a special channel
to opt in, i.e. `conda install -c conda-forge/label/numpy_rc numpy=2.0`), and yet
produce packages that are compatible with older numpy versions and therefore can
be published to our main channels.
h-vetinari marked this conversation as resolved.
Show resolved Hide resolved

migration_number: 1
ordering:
# prefer channels including numpy_rc (otherwise smithy doesn't
# know which of the two values should be taken on merge)
channel_sources:
- conda-forge
- conda-forge/label/numpy_rc,conda-forge

# needs to fully reproduce zip of {python, python_impl, numpy}
# in order to override it
numpy:
- 1.22 # no py38 support for numpy 2.0
- 2.0
- 2.0
- 2.0
python:
- 3.8.* *_cpython
- 3.9.* *_cpython
- 3.10.* *_cpython
- 3.11.* *_cpython
Comment on lines +66 to +70
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I just realized that having this bit in the migrator is a problem, because it considers all python-feedstocks as up for migration. I'll try to reduce that.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 1aef67b (after testing that the rerender in scipy didn't change)

python_impl:
- cpython
- cpython
- cpython
- cpython
channel_sources:
- conda-forge/label/numpy_rc,conda-forge
migrator_ts: 1713572489.295986
4 changes: 3 additions & 1 deletion recipe/migrations/pypy38.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ python:
- 3.9.* *_73_pypy # [not (osx and arm64)]
numpy:
# part of a zip_keys: python, python_impl, numpy
- 1.22 # [not (osx and arm64)]
- 2.0 # [not (osx and arm64)]
h-vetinari marked this conversation as resolved.
Show resolved Hide resolved
python_impl:
- pypy # [not (osx and arm64)]
channel_sources:
- conda-forge/label/numpy_rc,conda-forge # [not (osx and arm64)]
4 changes: 3 additions & 1 deletion recipe/migrations/python312.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ python:
- 3.12.* *_cpython
# additional entries to add for zip_keys
numpy:
- 1.26
- 2.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@h-vetinari This will break all feedstocks with Python 3.12. Please revert.

Cc @hmaarrfk

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How so? I've tested this with a bunch of feedstocks building for 3.12 successfully.

I'm happy to revert of course, but can someone describe (or link to) the breakage?

Copy link
Member Author

@h-vetinari h-vetinari May 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah nevermind, I understand what you mean. It changes py312 builds even before the migrator comes by. 😑

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's an interesting wrinkle. I'm not sure how we can then do the numpy migration for 3.12...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: 09f6263

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've continued the discussion how to fix this in conda-forge/conda-forge.github.io#1997

python_impl:
- cpython
channel_sources:
- conda-forge/label/numpy_rc,conda-forge