numba 0.55.0 on defaults doesn't constrain numpy as expected

[charlesbluca]

When using Conda to create an environment with both Numba and NumPy, I notice that the linux-64 numba 0.55.0 package hosted on defaults doesn't respect the max version constraint for NumPy like the conda-forge package does; this means it is possible to create an environment with numba 0.55 and numpy 1.22, which results in an ImportError when trying to import numba:

$ mamba create -n test numba=0.55 numpy=1.22
...
  + numba                0.55.0  py310h00e6091_0      pkgs/main/linux-64       Cached
  + numpy                1.22.1  py310h454958d_0      conda-forge/linux-64     Cached

If we limit the environment creation to only use conda-forge, this constraint is properly respected:

$ mamba create --override-channels -c conda-forge -n test numba=0.55 numpy=1.22
...
Encountered problems while solving:
  - package numba-0.55.0-py37h2d894fd_0 has constraint numpy >=1.18,<1.22 conflicting with numpy-1.22.0-py310h454958d_0

Is this different behavior between the two packages expected? I notice that before, the defaults hosted packages seemed to respect this constraint:

$ mamba create --override-channels -c defaults -n test numba=0.54 numpy=1.21
...
Encountered problems while solving:
  - package numba-0.54.1-py37h51133e4_0 has constraint numpy >=1.17,<1.21 conflicting with numpy-1.21.2-py310h20f2e39_0

For context, I observed this issue in dask-contrib/dask-sql#383 - here is an example of potential downstream failures that can occur as a result of this.

[esc]

@charlesbluca thank you, I can confirm this, but I do need to add -c conda-forge for it to resolve as 1.22 isn't available from defaults at the time of writing:

conda create -c conda-forge -n testxyz numba=0.55 numpy=1.22

[esc]

Interestingly enough the build from the numba channel may be correct. Here is what I get when I try to force it:

 💣 zsh» conda create -c conda-forge -n testxyz numba::numba=0.55 numpy=1.22                         :(
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: |
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

Package pypy3.7 conflicts for:
numba::numba=0.55 -> llvmlite[version='>=0.38.0rc1,<0.39'] -> pypy3.7[version='7.3.3.*|7.3.4.*|7.3.5.*|7.3.7.*|>=7.3.3|>=7.3.4|>=7.3.5|>=7.3.7']
pip -> python[version='>=3.6'] -> pypy3.7[version='7.3.3.*|7.3.4.*|7.3.5.*|7.3.7.*|>=7.3.7|>=7.3.5|>=7.3.3']

Package numpy conflicts for:
numpy=1.22
numba::numba=0.55 -> numpy[version='>=1.18,<1.22']

Package pip conflicts for:
numba::numba=0.55 -> python[version='>=3.7,<3.8.0a0'] -> pip
pip

  70.96s user 2.29s system 89% cpu 1:21.71 total

The error message is somewhat confusing however...

[charlesbluca]

I do need to add -c conda-forge for it to resolve as 1.22 isn't available from defaults

Yeah this really only becomes a problem when environments are created using both conda-forge and defaults - using one or the other shouldn't be an issue until numpy 1.22 becomes available in defaults.

The error message is somewhat confusing however...

It is - I wonder why mamba's solver doesn't catch the pypy3.7 conflicts?

$ mamba create --override-channels -c conda-forge -n test numba::numba=0.55 numpy=1.22
...
Encountered problems while solving:
  - package numba-0.55.0rc1-np1.11py3.7hc13618b_gf2a673cd0_0 requires numpy >=1.18,<1.22, but none of the providers can be installed

[esc]

@charlesbluca I have forwarded this to the team that does defaults.

[stuartarchibald]

@esc should this be closed?

[esc]

@stuartarchibald only when the package has been fixed, I would say?

[github-actions]

This issue is marked as stale as it has had no activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with any updates and confirm that this issue still needs to be addressed.

[esc]

I just tried this and the package from defaults will happily install alongside a Numpy 1.22 which we don't support:

  numba              pkgs/main/osx-64::numba-0.55.0-py310hc081a56_0
  numpy              conda-forge/osx-64::numpy-1.22.2-py310hfbbbacf_0

[esc]

During an OOB conversation with a colleague, I was pointed towards:

https://github.com/AnacondaRecipes/repodata-hotfixes

This may be a good way to constrain the package data.

[stuartarchibald]

During an OOB conversation with a colleague, I was pointed towards:

https://github.com/AnacondaRecipes/repodata-hotfixes

This may be a good way to constrain the package data.

xref conda-forge/numba-feedstock#90