pylibcugraph: declare cupy and numpy hard dependencies

[jameslamb]

While testing stuff for #4804, I found that pylibcugraph has a hard runtime dependency on cupy and numpy, but isn't declaring them

docker run \
    --rm \
    --gpus 0 \
    -it rapidsai/ci-wheel:latest \
    bash

python -m pip install 'pylibcugraph-cu12==25.2.*,>=0.0.0a0'
python -c "import pylibcugraph"

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/pyenv/versions/3.12.7/lib/python3.12/site-packages/pylibcugraph/__init__.py", line 18, in <module>
    from pylibcugraph.graphs import SGGraph, MGGraph
  File "graphs.pyx", line 1, in init pylibcugraph.graphs
  File "utils.pyx", line 20, in init pylibcugraph.utils
ModuleNotFoundError: No module named 'cupy'

This declares those dependencies.

It also promotes cugraph-service-server's numpy dependency to a hard runtime dependency.

Notes for Reviewers

Evidence that pylibcugraph already has a hard dependency on these libraries

They're used unconditionally here:

cugraph/python/pylibcugraph/pylibcugraph/utils.pyx

Lines 19 to 20 in cddd69e

	import numpy
	import cupy

But have import guards in other places:

cugraph/python/pylibcugraph/pylibcugraph/sssp.pyx

Lines 127 to 139 in cddd69e

	# FIXME: import these modules here for now until a better pattern can be
	# used for optional imports (perhaps 'import_optional()' from cugraph), or
	# these are made hard dependencies.
	try:
	import cupy
	except ModuleNotFoundError:
	raise RuntimeError("sssp requires the cupy package, which could not "
	"be imported")
	try:
	import numpy
	except ModuleNotFoundError:
	raise RuntimeError("sssp requires the numpy package, which could not "
	"be imported")

So this PR doesn't introduce new hard dependencies... it just makes them explicit, to make it easier to install and run pylibcugraph.

How was this not caught in CI?

Import tests aren't run here for conda packages, because conda builds happen on CPU-only nodes.

cugraph/ci/build_python.sh

Lines 27 to 30 in cddd69e

	rapids-conda-retry mambabuild \
	--no-test \
	--channel "${CPP_CHANNEL}" \
	conda/recipes/pylibcugraph

And numpy and cupy are probably getting pulled in by some of the wheels' test dependencies, like cudf, here:

cugraph/ci/test_wheel.sh

Line 17 in cddd69e

python -m pip install $(ls ./dist/${python_package_name}*.whl)[test]

Should we just make the other unconditional cases conditional with try-catching?

No. Talked with @rlratzel, @ChuckHastings, and @eriknw offline, and we agreed to declare these as hard runtime dependencies (and remove the try-catching in places that had it).

[jameslamb]

cugraph-service-server should have a hard runtime dependency on numpy, because of this:

cugraph/python/cugraph-service/server/cugraph_service_server/cugraph_handler.py

Line 30 in a6eea2f

import numpy as np

its conda recipe does:

cugraph/conda/recipes/cugraph-service/meta.yaml

Line 66 in a6eea2f

- numpy >=1.23,<3.0a0

[rlratzel]

agreed, thanks for being so thorough!

[rlratzel]

LGTM, thanks!

[rlratzel]

agreed, thanks for being so thorough!

[bdice]

Since numpy doesn’t have any matrices for CUDA version or any extra index URLs, I would treat it as a normal dependency and add it to the corresponding lists. We don’t use depends_on_numpy in other repos because it doesn’t reduce complexity. You could use a YAML anchor for deduplication of the pinning declaration if desired.

[bdice]

Example: https://github.com/rapidsai/cudf/blob/dc2a75cba40d38f4a6ba66e652764e96fa6b593d/dependencies.yaml#L661

[jameslamb]

Ok sure.

That is not how I've been thinking about when to do these depends_on_* lists. At https://github.com/rapidsai/build-planning/blob/d9e3c606d95c835ee384ac6480a4af0ac6cb024a/docs/docs/packaging.md#L181 we have this:

Dependencies appearing in several lists should be in their own standalone depends_on_{project} lists.

I can update those docs with this added nuance, I don't disagree with you.

Pushed 6a83f20 here removing depends_on_numpy in favor of anchors. That revealed that there were some other issues in cugrpah-service-server's [test] extra... it was pulling through all of cugraph's testing dependencies, which it doesn't need based on my read of https://github.com/rapidsai/cugraph/tree/branch-25.02/python/cugraph-service/server/cugraph_service_server/testing

[bdice]

Yes, I think those docs should be updated. I think it should be something like:

Dependencies appearing in several lists can be handled in two ways. Dependencies with complex requirements such as extra index URLs or package names that vary by CUDA version (like -cuXX suffixes) or format (different conda and PyPI names) should be in their own standalone depends_on_{project} lists. Simpler dependencies can use YAML anchors to avoid duplication of the pinning specs.

I'll re-review and approve based on this. Feel free to update those docs as you see fit.