Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changes to Cargo easyblock break installation of polars #3567

Open
boegel opened this issue Jan 25, 2025 · 0 comments
Open

changes to Cargo easyblock break installation of polars #3567

boegel opened this issue Jan 25, 2025 · 0 comments

Comments

@boegel
Copy link
Member

boegel commented Jan 25, 2025

The installation of polars-0.20.2-gfbf-2023a.eb is failing with current develop (and when using eb --ignore-checksums) with:

== 2025-01-25 19:43:23,905 build_log.py:171 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:126 in __init__): cmd " /software/Python/3.11.3-GCCcore-12.3.0/bin/python -m pip install --prefix=/software/polars/0.20.2-gfbf-2023a  --verbose  --no-deps  --ignore-installed  --no-index  --no-build-isolation  .[deltalake,matplotlib,numpy,openpyxl,pandas,pyarrow]" exited with exit code 1 and output:
Using pip 23.1.2 from /software/Python/3.11.3-GCCcore-12.3.0/lib/python3.11/site-packages/pip (python 3.11)
Processing /tmp/easybuild/polars/0.20.2/gfbf-2023a/polars-0.20.2
  Preparing metadata (pyproject.toml): started
  Running command Preparing metadata (pyproject.toml)
  error: failed to get `jsonpath_lib` as a dependency of package `polars-ops v0.35.4 (/tmp/easybuild/polars/0.20.2/gfbf-2023a/polars-0.20.2/crates/polars-ops)`

  Caused by:
    failed to load source for dependency `jsonpath_lib`

  Caused by:
    Unable to update https://github.com/ritchie46/jsonpath?branch=improve_compiled#24eaf0b4

  Caused by:
    can't checkout from 'https://github.com/ritchie46/jsonpath': you are in the offline mode (--offline)
   maturin failed
    Caused by: Cargo metadata failed. Does your crate compile with `cargo build`?
    Caused by: `cargo metadata` exited with an error:
  Error running maturin: Command '['maturin', 'pep517', 'write-dist-info', '--metadata-directory', '/tmp/eb-p76aywjz/pip-modern-metadata-5wesklv0', '--interpreter', '/software/Python/3.11.3-GCCcore-12.3.0/bin/python']' returned non-zero exit status 1.
  Checking for Rust toolchain....
  Running `maturin pep517 write-dist-info --metadata-directory /tmp/eb-p76aywjz/pip-modern-metadata-5wesklv0 --interpreter /software/Python/3.11.3-GCCcore-12.3.0/bin/python`  error: subprocess-exited-with-error

To get to this point, I had to first remove the checksums list from the polars easyconfig, otherwise I hit an earlier error:

== FAILED: Installation ended unsuccessfully (build directory: /tmp/vsc40003/easybuild/polars/0.20.2/gfbf-2023a): build failed (first 300 chars): Missing checksum for
jsonpath_lib-0.3.0-24eaf0b4416edff38a4d1b6b17bc4b9f3f047b4b.tar.gz in {'jsonpath_lib-0.3.0.tar.gz': 'dfb5eb7f47b5b5a7b35262f7b41787a785958001a23ff039cdae926396c6af96'}

It seems like this problem was introduced in #3483, which changed how crates entries that use a Git repo, as is the case for jsonpath_lib in the polars easyconfig:

crates = [
    ...
    ('jsonpath_lib', '0.3.0', 'https://github.com/ritchie46/jsonpath', '24eaf0b4416edff38a4d1b6b17bc4b9f3f047b4b'),

One key difference seems to be the name of the source file: in the working installation it's jsonpath_lib-0.3.0.tar.gz, while with current develop I'm seeing jsonpath_lib-0.3.0-24eaf0b4416edff38a4d1b6b17bc4b9f3f047b4b.tar.gz being used.

After unpacking, these tarballs are exactly the same though.
And while jsonpath_lib-0.3.0-24eaf0b4416edff38a4d1b6b17bc4b9f3f047b4b.tar.gz is being unpacked to /tmp/easybuild/polars/0.20.2/gfbf-2023a/easybuild_vendor_git/jsonpath, it gets moved to /tmp/easybuild/polars/0.20.2/gfbf-2023a/easybuild_vendor/jsonpath, which is the same path that jsonpath_lib-0.3.0.tar.gz gets unpacked into in the working installation.
So I'm not entirely sure what actually causes the " failed to load source for dependency" problem...

@Micket Any ideas here?

@boegel boegel added this to the release after 4.9.4 milestone Jan 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant