cross 0.2.4 failed to compile detect-wasi on aarch64-unknown-linux-gnu

[NobodyXu]

Checklist

• I've looked through the issues and pull requests for similar reports

Describe your issue

Running cmd

cross build --target aarch64-unknown-linux-gnu --profile release --no-default-features --features zlib-ng,static,rustls,trust-dns,fancy-no-backtrace,log_release_max_level_debug -Z build-std=std,panic_abort -Z build-std-features=panic_immediate_abort

on cargo-binstall v0.15.0 failed with error:

  = note: /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::thread::local::fast::Key$LT$T$GT$::try_initialize::hd96642440da283f9':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std6thread5local4fast12Key$LT$T$GT$14try_initialize17hd96642440da283f9E+0xe0): undefined reference to `__aarch64_ldadd8_rel'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::sync::once::Once::call_inner::h060aa2b0d3f5a68e':
          detect_wasi.a3009cc8-cgu.0:(.text.unlikely._ZN3std4sync4once4Once10call_inner17h060aa2b0d3f5a68eE+0x80): undefined reference to `__aarch64_cas8_rel'
          detect_wasi.a3009cc8-cgu.0:(.text.unlikely._ZN3std4sync4once4Once10call_inner17h060aa2b0d3f5a68eE+0xb4): undefined reference to `__aarch64_ldadd4_acq'
          detect_wasi.a3009cc8-cgu.0:(.text.unlikely._ZN3std4sync4once4Once10call_inner17h060aa2b0d3f5a68eE+0xd8): undefined reference to `__aarch64_cas4_acq'
          detect_wasi.a3009cc8-cgu.0:(.text.unlikely._ZN3std4sync4once4Once10call_inner17h060aa2b0d3f5a68eE+0x114): undefined reference to `__aarch64_cas8_acq'
          detect_wasi.a3009cc8-cgu.0:(.text.unlikely._ZN3std4sync4once4Once10call_inner17h060aa2b0d3f5a68eE+0x14c): undefined reference to `__aarch64_swp8_acq_rel'
          detect_wasi.a3009cc8-cgu.0:(.text.unlikely._ZN3std4sync4once4Once10call_inner17h060aa2b0d3f5a68eE+0x18c): undefined reference to `__aarch64_swp4_rel'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::thread::current::h6376ef4dd963a383':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std6thread7current17h6376ef4dd963a383E+0x54): undefined reference to `__aarch64_ldadd8_relax'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::thread::Thread::new::h83b0ac90303c4f44':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std6thread6Thread3new17h83b0ac90303c4f44E+0x9c): undefined reference to `__aarch64_cas8_relax'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::sys_common::thread_local_key::StaticKey::key::h23535b24a0a27e8e':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std10sys_common16thread_local_key9StaticKey3key17h23535b24a0a27e8eE+0x58): undefined reference to `__aarch64_cas8_acq_rel'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::sync::once::Once::call_once::_$u7b$$u7b$closure$u7d$$u7d$::he53d694758cb088e':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std4sync4once4Once9call_once28_$u7b$$u7b$closure$u7d$$u7d$17he53d694758cb088eE+0x88): undefined reference to `__aarch64_cas4_acq'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::sys::unix::locks::futex_mutex::Mutex::unlock::haa69c46c5ac213ae':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std3sys4unix5locks11futex_mutex5Mutex6unlock17haa69c46c5ac213aeE+0x10): undefined reference to `__aarch64_swp4_rel'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::sys::unix::locks::futex_mutex::Mutex::lock::h6e75f17ed18423bc':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std3sys4unix5locks11futex_mutex5Mutex4lock17h6e75f17ed18423bcE+0x58): undefined reference to `__aarch64_swp4_acq'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::sys::unix::os::env_read_lock::hdf4a0560f8167376':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std3sys4unix2os13env_read_lock17hdf4a0560f8167376E+0x38): undefined reference to `__aarch64_cas4_acq'
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std3sys4unix2os13env_read_lock17hdf4a0560f8167376E+0x8c): undefined reference to `__aarch64_cas4_acq'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::sys_common::rwlock::MovableRwLock::read_unlock::h6a75b81985194117':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std10sys_common6rwlock13MovableRwLock11read_unlock17h6a75b81985194117E+0x10): undefined reference to `__aarch64_ldadd4_rel'
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std10sys_common6rwlock13MovableRwLock11read_unlock17h6a75b81985194117E+0xac): undefined reference to `__aarch64_cas4_relax'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::sys::unix::locks::futex_rwlock::RwLock::wake_writer::h895883f366df70bd':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std3sys4unix5locks12futex_rwlock6RwLock11wake_writer17h895883f366df70bdE+0x10): undefined reference to `__aarch64_ldadd4_rel'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::process::Command::output::h701327146e947caf':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std7process7Command6output17h701327146e947cafE+0x15d4): undefined reference to `__aarch64_ldset8_relax'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `std::sys::unix::fs::remove_dir_impl::remove_dir_all_recursive::h93d2e878300c4293':
          detect_wasi.a3009cc8-cgu.0:(.text._ZN3std3sys4unix2fs15remove_dir_impl24remove_dir_all_recursive17h93d2e878300c4293E+0xf8): undefined reference to `__aarch64_ldadd8_relax'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `OUTLINED_FUNCTION_41':
          detect_wasi.a3009cc8-cgu.0:(.text.OUTLINED_FUNCTION_41+0x4): undefined reference to `__aarch64_ldadd8_rel'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `OUTLINED_FUNCTION_55':
          detect_wasi.a3009cc8-cgu.0:(.text.OUTLINED_FUNCTION_55+0x8): undefined reference to `__aarch64_cas4_acq'
          /target/aarch64-unknown-linux-gnu/release/deps/detect_wasi-c76bdb245350245c.detect_wasi.a3009cc8-cgu.0.rcgu.o: In function `OUTLINED_FUNCTION_71':
          detect_wasi.a3009cc8-cgu.0:(.text.OUTLINED_FUNCTION_71+0x4): undefined reference to `__aarch64_cas4_relax'
          collect2: error: ld returned 1 exit status

The error originates from std lib synchronization APIs (thread key, mutex, rwlock Once), std::thread::Thead::new, std::process::Command::output and std::sys::unix::fs::remove_dir_impl::remove_dir_all_recursive.

This might be related to #1066

I've tried setting export RUSTFLAGS="-C link-arg=-lgcc -Clink-arg=-static-libgcc" according to https://github.com/cross-rs/cross/wiki/FAQ#undefined-reference-with-build-std but still doesn't work.

Compiling cargo-binstall worked before #1046

What target(s) are you cross-compiling for?

aarch64-unknown-linux-gnu

Which operating system is the host (e.g computer cross is on) running?

• macOS
• Windows
• Linux / BSD
• other OS (specify in description)

What architecture is the host?

• x86_64 / AMD64
• arm32
• arm64 (including Mac M1)

What container engine is cross using?

• docker
• podman
• other container engine (specify in description)

cross version

cross 0.2.4

Example

No response

Additional information / notes

No response

[Alexhuszagh]

Pretty certain this is invalid as well, as you need to disable outline-atomics (-Ctarget-feature=-outline-atomics) for this, which are enabled by default:

$ rustc +nightly -Z unstable-options --print target-spec-json --target aarch64-unknown-linux-gnu
{
  "arch": "aarch64",
  "crt-static-respected": true,
  "data-layout": "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128",
  "dynamic-linking": true,
  "env": "gnu",
  "features": "+outline-atomics",
  "has-rpath": true,
  "has-thread-local": true,
  "is-builtin": true,
  "llvm-target": "aarch64-unknown-linux-gnu",
  "max-atomic-width": 128,
  "os": "linux",
  "position-independent-executables": true,
  "relro-level": "full",
  "supported-sanitizers": [
    "address",
    "cfi",
    "leak",
    "memory",
    "memtag",
    "thread",
    "hwaddress"
  ],
  "target-family": [
    "unix"
  ],
  "target-mcount": "\u0001_mcount",
  "target-pointer-width": "64"
}

Not closing this yet because I need to do a few small tests first to ensure this is the case. It seems there's some issues with feature detection in zlib-ng, however. See also japaric/xargo#329.

[NobodyXu]

@Alexhuszagh This error is caused by detect-wasi, not libz-ng.

To reproduce it, you need to clone cargo-bins/cargo-binstall then run the cross command in it.

[Alexhuszagh]

To reproduce it, you need to clone cargo-bins/cargo-binstall then run the cross command in it.

Ah good to know. By any chance can you get a minimal package so I can test this locally?

[NobodyXu]

To reproduce it, you need to clone cargo-bins/cargo-binstall then run the cross command in it.

Ah good to know. By any chance can you get a minimal package so I can test this locally?

detect-wasi is quite small (only a few lines of code) and you may be able to reproduce it compiling only detect-wasi since the error is from this crate.

[NobodyXu]

@Alexhuszagh https://github.com/cargo-bins/cargo-binstall/tree/main/crates/detect-wasi it is inside the cargo-binstall repository as a workspace.

[Alexhuszagh]

@Alexhuszagh https://github.com/cargo-bins/cargo-binstall/tree/main/crates/detect-wasi it is inside the cargo-binstall repository as a workspace.

Thank you, it's getting late here so I'll get to this tomorrow, but I'll email myself to do it first thing tomorrow.

[Alexhuszagh]

This isn't reproducing for me with the last main of cross and rustc 1.66.0-nightly (0b84a35c2 2022-10-03)

$ cross --version
cross 0.2.4 (0fcfc0b 2022-09-29)
$ cargo +nightly --version
cargo 1.66.0-nightly (0b84a35c2 2022-10-03)
$ cross +nightly build --target aarch64-unknown-linux-gnu --profile release --no-default-features --features zlib-ng,static,rustls,trust-dns,fancy-no-backtrace,log_release_max_level_debug -Z build-std=std,panic_abort -Z build-std-features=panic_immediate_abort

It does reproduces on cross 0.2.4, and I'm guessing #591 (bumping the toolchain versions and base distro) patched the issue (since we use the official Ubuntu toolchain).

It seems like the move to GCC 9.4.0 and glibc 2.31 (from GCC 5.4.0 and 2.23) fixes the issue.

[NobodyXu]

Good to hear that, I'm looking forward to the 0.3.0 release

[Alexhuszagh]

If you want stability but also a recent distro, you can always pin to a specific cross version before the official release. We're getting there, there's just some work on the backburner we'd like to solve first. For example, cargo install cross --git https://github.com/cross-rs/cross --rev a11fd20cf96a40d35906f06eca9a0d95f1c29ebf. And that way you can use know versions and just bump the toolchain whenever. This will, however, have the image versions change based on the latest main.

[NobodyXu]

Thanks for the advice, I would like to stick with using point release though.

Using specific git rev would add a lot of trouble and either makes the CI more expensive or make the code for the CI more complex by adding more caching.

Currently we uses taikie-e/install-actions which uses pre-built binaries and I'd like to keep that.