maturin fails to package .so files in target directory

[messense]

Discussed in #1087

^{Originally posted by jdiggans-twist September 8, 2022}
I have a PyO3 project with python and Rust code. One of the Rust crate dependencies is catboost, which has its own build.rs script and compiles an rlib and an .so.1 library file.

The catboost .so.1 file is correctly build by maturin (triggering cargo build) and is present deep in a target/build subdirectory. Adding this via LD_LIBRARY_PATH does not seem to help maturin find the file. I am using quay.io/pypa/manylinux_2_28_x86_64 as my base image, so everything around the manylinux stuff seems to work fine.

When trying to build a manylinux_2_28 wheel using maturin, however, I get the following:

💥 maturin failed
 Caused by: Cannot repair wheel, because required library libcatboostmodel.so.1 could not be located.

I can build the wheel using --skip-auditwheel and then try repairing but again get failure:

ValueError: Cannot repair wheel, because required library "libcatboostmodel.so.1" could not be located

If I unzip the wheel and look at the output of ldd on the binary I see:

linux-vdso.so.1 (0x00007ffc6d1a8000)
libcatboostmodel.so.1 => not found
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f1a413eb000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1a411cb000)
libm.so.6 => /lib64/libm.so.6 (0x00007f1a40e49000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f1a40c45000)
libc.so.6 => /lib64/libc.so.6 (0x00007f1a40880000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1a41cfc000)

How can I help maturin find and include this file? Why is it ignoring the library?

[jdiggans-twist]

Many thanks for fast assist @messense!

[messense]

Taking a note here:

cargo rustc --message-format=json output for catboost-sys is

{
  "reason": "build-script-executed",
  "package_id": "catboost-sys 0.1.0 (git+https://github.com/catboost/catboost#073ae73fc49a3aebf6c9290b686a5582ff2dea68)",
  "linked_libs": [
    "stdc++",
    "dylib=catboostmodel"
  ],
  "linked_paths": [
    "/root/code/dylib-target/target/debug/build/catboost-sys-26c2bd1bc1bd5310/out/catboost/libs/model_interface"
  ],
  "cfgs": [],
  "env": [],
  "out_dir": "/root/code/dylib-target/target/debug/build/catboost-sys-26c2bd1bc1bd5310/out"
}

Documentation for build-script-executed: https://doc.rust-lang.org/cargo/reference/external-tools.html#build-script-output

Files in the mentions directories:

root@ubuntu:~/code/dylib-target# ls -l target/debug/build/catboost-sys-26c2bd1bc1bd5310/out/catboost/libs/model_interface
total 252048
-rwxr-xr-x 6 root root 129047272 Sep  8 04:53 libcatboostmodel.so
-rwxr-xr-x 6 root root 129047272 Sep  8 04:53 libcatboostmodel.so.1
root@ubuntu:~/code/dylib-target# ls -l target/debug/build/catboost-sys-26c2bd1bc1bd5310/out
total 24
-rw-r--r-- 1 root root 17641 Sep  8 04:53 bindings.rs
drwxr-xr-x 3 root root  4096 Sep  8 04:53 catboost

[jdiggans-twist]

Thanks @messense - I cloned maturin HEAD and updated its Cargo.toml to point to lddtree's github HEAD. This still resulted in:

💥 maturin failed
  Caused by: Cannot repair wheel, because required library libcatboostmodel.so.1 could not be located.

while the relevant files/output from the catboost model look good to me:

> ls target/debug/build/catboost-sys-55fa36f12777dd6b/out/catboost/libs/model_interface/
target/debug/build/catboost-sys-55fa36f12777dd6b/out/catboost/libs/model_interface/libcatboostmodel.so.1

and

> more target/debug/build/catboost-sys-55fa36f12777dd6b/output
cargo:rustc-link-search=/<my-crate-name>/target/debug/build/catboost-sys-55fa36f12777dd6b/out/catboost/libs/model_interface
cargo:rustc-link-lib=stdc++
cargo:rustc-link-lib=dylib=catboostmodel

[messense]

The new API in lddtree has not been integrated into maturin yet, I'll look into it soon.

$ cargo install lddtree
$ root@ubuntu:~/code/dylib-target# lddtree target/debug/libdylib_target.so / target/debug/build/catboost-sys-26c2bd1bc1bd5310/out/catboost/libs/model_interface
libcatboostmodel.so.1 => /root/code/dylib-target/target/debug/build/catboost-sys-26c2bd1bc1bd5310/out/catboost/libs/model_interface/libcatboostmodel.so.1
    libdl.so.2 => /usr/lib/x86_64-linux-gnu/libdl.so.2
        libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6
            ld-linux-x86-64.so.2 => /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
    librt.so.1 => /usr/lib/x86_64-linux-gnu/librt.so.1
        libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6
            ld-linux-x86-64.so.2 => /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
    libpthread.so.0 => /usr/lib/x86_64-linux-gnu/libpthread.so.0
        libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6
            ld-linux-x86-64.so.2 => /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
    libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6
        ld-linux-x86-64.so.2 => /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
    ld-linux-x86-64.so.2 => /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
    libm.so.6 => /usr/lib/x86_64-linux-gnu/libm.so.6
        libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6
            ld-linux-x86-64.so.2 => /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
        ld-linux-x86-64.so.2 => /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
libgcc_s.so.1 => /usr/lib/x86_64-linux-gnu/libgcc_s.so.1
    libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6
        ld-linux-x86-64.so.2 => /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6
    ld-linux-x86-64.so.2 => /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
ld-linux-x86-64.so.2 => /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2

The lddtree command line can find it now so it's just a matter of speeding the time to add the support in maturin.

---

Note that I have limited time to work on open source projects after my day job, it can take a while sometimes. I'm seeking for sponsorship to be able to allocate more time on open source projects ☺️.

[messense]

Please try v0.13.3-beta.3, you can install it by running pip install maturin==0.13.3b3.

[jdiggans-twist]

This worked beautifully - thank you!

[jdiggans-twist]

Actually maybe I spoke too soon - maturin build --release builds a wheel that properly includes the library and installs just fine. maturin develop however does not seem to move the library to wherever python is looking for it. Was this fix expected to address develop too?

[messense]

@jdiggans-twist Opened #1093, for now maturin build then pip install can be used as a workaround I think.