cargo build on workspaces does not interact well with vendored dependencies

[froydnj]

I'm not sure if this is a problem with vendored dependencies, or whether it's a problem with how the dependencies of workspaces get computed. Apologies in advance for the long-winded details.

Firefox has some crates that get built separately:

• gkrust: https://hg.mozilla.org/mozilla-central/file/tip/toolkit/library/rust/Cargo.toml
• gkrust-gtest: https://hg.mozilla.org/mozilla-central/file/tip/toolkit/library/gtest/rust/Cargo.toml

Both of these crates depend on a shared crate:

• gkrust-shared: https://hg.mozilla.org/mozilla-central/file/tip/toolkit/library/rust/shared/Cargo.toml

Building both gkrust and gkrust-gtest require separate builds of gkrust-shared, which is unnecessary work; we would like to use workspaces to solve this problem.

Firefox also vendors all of its Rust dependencies:

https://hg.mozilla.org/mozilla-central/file/tip/third_party/rust

The crates currently there permit all the Rust code in tree to be built without a problem. In particular. the mp4parse crate:

https://hg.mozilla.org/mozilla-central/file/tip/media/libstagefright/binding/mp4parse/Cargo.toml

has a dev-dependencies of test-assembler, but we do not have test-assembler vendored, as we don't compile tests, examples, or benchmarks (as the documentation behind dev-dependencies indicates).

If we try to tie gkrust and gkrust-gtest into a workspace, however, by introducing a top-level Cargo.toml:

[workspace]
members = [
    "toolkit/library/rust",
    "toolkit/library/gtest/rust",
]

# Explicitly specify what our profiles use.
[profile.dev]
opt-level = 1
debug = true
rpath = false
lto = false
debug-assertions = true
codegen-units = 1
panic = "abort"

[profile.release]
opt-level = 2
debug = true
rpath = false
lto = true
debug-assertions = false
panic = "abort"

when we try to build the gkrust crate via cargo build, Cargo says:

error: no matching package named `test-assembler` found (required by `mp4parse`)
location searched: registry https://github.com/rust-lang/crates.io-index
version required: = 0.1.5

But Cargo shouldn't need that crate for cargo build, since building gkrust standalone without workspaces works just fine.

[froydnj]

@mbrubeck informs me that this behavior was implemented in #3125. I claim there are still some bugs to be worked out. :)

[alexcrichton]

I believe that this is intended behavior of Cargo today, and I believe that #3125 is unrelated to this issue.

On all Cargo invocations it will always generate a Cargo.lock, and for a workspace this lock file describes the entire workspace. Lock files contain dev dependencies to support cargo test (e.g. everything is resolved ahead of time), and cargo test is supported on all crates in a workspace. That means that all member crates of a workspace will have all their dev-dependencies included in the lock file.

A solution here would be to exclude mp4parse from the workspace, but that would likely require #3192 to be implemented. Conversely the test-assembler could also be vendored for now in the vendor directory.

Does that make sense?

[alexcrichton]

I'm going to close this as intended behavior, but let me know if there's still something here!