Databricks ML Runtime 15 fails to import cudf

[jacobtomlinson]

Reproducer:

• Follow the docs here
  • Create a startup script (see below)
  • Create single node compute instance
  • Choose the 15.4 LTS ML (includes Apache Spark 3.5.0, GPU, Scala 2.12)
  • Choose a GPU VM type
  • Under advanced add the startup script

#!/bin/bash
set -e

# Install RAPIDS libraries
pip install \
    --extra-index-url=https://pypi.anaconda.org/rapidsai-wheels-nightly/simple \
    "cudf-cu11>=25.02.*,>=0.0.0a0" "cuml-cu11>=25.02.*,>=0.0.0a0" \
    "dask-cuda>=25.02.*,>=0.0.0a0"

Importing cudf fails

[jacobtomlinson]

Switching to *-cu12 should resolve this.