tests from crates.io tarball are failing due to missing files

[ignatenkobrain]

---- elements::module::integration_tests::const_ stdout ----
thread 'elements::module::integration_tests::const_' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5
note: Run with `RUST_BACKTRACE=1` for a backtrace.

---- elements::module::integration_tests::hello stdout ----
thread 'elements::module::integration_tests::hello' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::module::integration_tests::peek stdout ----
thread 'elements::module::integration_tests::peek' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::module::integration_tests::memory_space stdout ----
thread 'elements::module::integration_tests::memory_space' panicked at 'failed to deserialize: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::module::integration_tests::names stdout ----
thread 'elements::module::integration_tests::names' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::module::integration_tests::serde stdout ----
thread 'elements::module::integration_tests::serde' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::module::integration_tests::peek_3 stdout ----
thread 'elements::module::integration_tests::peek_3' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::module::integration_tests::peek_2 stdout ----
thread 'elements::module::integration_tests::peek_2' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::module::integration_tests::serde_code stdout ----
thread 'elements::module::integration_tests::serde_code' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::module::integration_tests::serde_import stdout ----
thread 'elements::module::integration_tests::serde_import' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::module::integration_tests::serde_type stdout ----
thread 'elements::module::integration_tests::serde_type' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::module::integration_tests::store stdout ----
thread 'elements::module::integration_tests::store' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::module::integration_tests::varuint1_case stdout ----
thread 'elements::module::integration_tests::varuint1_case' panicked at 'Maybe shouldn't be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::reloc_section::tests::reloc_section stdout ----
thread 'elements::reloc_section::tests::reloc_section' panicked at 'Module should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

---- elements::section::tests::import_section stdout ----
thread 'elements::section::tests::import_section' panicked at 'Should be deserialized: HeapOther("Can\'t read from the file: Os { code: 2, kind: NotFound, message: \"No such file or directory\" }")', libcore/result.rs:945:5

Please either include them or somehow disable those tests by default. Thanks!

[NikVolf]

Those tests rely on submodule, so download it before (git submodule update —init in the directory)

[ignatenkobrain]

note that crates.io tarballs do not contain any git submodule information

[NikVolf]

Is there any way to specify it?

[ignatenkobrain]

@NikVolf I believe that those files are excluded here: https://github.com/paritytech/parity-wasm/blob/master/Cargo.toml#L13

[NikVolf]

But if I remove it, every user will have to download the test suite, which is not at all needed

[pepyakin]

Hi @ignatenkobrain ! sorry for chiming in so late.

Cargo testing usually assumes that you're working with crate locally and crates.io packages don't always contain all necesarry files for testing.

I wonder why do you have such requirements?

[ignatenkobrain]

We try to test all crates before we ship them to users when packaging in Fedora.

This works for most of the crates, but for those which do not, upstream usually add some flag which disables testing of data-dependent tests.

[ignatenkobrain]

by doing this we catched a lot of different issues related to endianess and/or some specific versions of crates.

[ignatenkobrain]

rust-lang/rust-bindgen#1412
dimforge/nalgebra#383
BurntSushi/rust-pcre2#3
dtolnay/ryu#2
sfackler/rust-openssl#964
stratis-storage/devicemapper-rs#343
rust-cli/config-rs#75
hyperium/mime#85
sunng87/handlebars-rust#219
serde-deprecated/test#19
alexcrichton/filetime#23
serde-rs/serde#1230
servo/rust-cssparser#214
tailhook/humantime#2

Just some links for the references.

[pepyakin]

We try to test all crates before we ship them to users when packaging in Fedora.

Interesting! This is totally a legit case, but as I've said it's not officially supported... I would recommend to file an issue to cargo/crates.io repo to ask about officially supporting this use-case.

In the meantime, I think it should be possible to download the tests dynamically. Will look into that.

[silwol]

We stumbled over the same problem when packaging the crate for Debian. Unfortunately, adding a method to dynamically download the files also doesn't help, because in Debian all packages must be buildable without a network connection. In this case, I also think it's not worth distributing additional files of 30MB size, so I'll disable the tests for now. Even though running tests has surfaced several real issues in the past, similar to Fedora's experience.