Support bzlmod

[cameron-martin]

This makes rules_rust load dependencies via bzlmod. Currently only basic functionality is completed, such as registering rustc toolchains and compiling crates. Note that it cannot interact with cargo, wasm or load any other rust toolchains such as rustfmt.

There is one new module, examples/bzlmod/hello_world, that depends on the root rules_rust module. This example can be built and run using:

cd examples/bzlmod/hello_world
bazel run //:hello_world

To register toolchains in an ergonomic way, it defines a new "hub" repository that contains all the toolchain proxies, so they can all be registered like so:

register_toolchains("@rust_toolchains//:all")

closes #1493

[UebelAndre]

re #1493 (comment)

@scentini do you or anyone you have contact with at Google know anything about bzlmod?

[scentini]

I'm not familiar with bzlmod, in case we're doing something silly here it's not obvious to me. @Wyverald could you PTAL?

[scentini]

cc @meteorcloudy

[meteorcloudy]

Bzlmod may have some bugs in Bazel 5.3.0, it's recommended to test with Bazel@HEAD, you can use Bazelisk:

export USE_BAZEL_VERSION=last_green
bazelisk build //:hello_world

I got the following toolchain error instead of the visibility error:

ERROR: /Users/pcloudy/workspace/rules_rust/examples/bzlmod/hello_world/BUILD.bazel:5:12: While resolving toolchains for target //:hello_world: No matching toolchains found for types @rules_rust~override//rust:toolchain_type.
To debug, rerun with --toolchain_resolution_debug='@rules_rust~override//rust:toolchain_type'
If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro.

[cameron-martin]

I've updated the PR with a rough proof of concept that can compile and run a rust binary.

[meteorcloudy]

Nice, can you also fix the buildifier errors?

[benbrittain]

Is there any work I can take on here to help this get landed?

[UebelAndre]

@cameron-martin hey, any updates? 😄

[cameron-martin]

@cameron-martin hey, any updates? smile

Sorry I've been busy with other things. I may be able to get back to this in the next week or so.

[opicaud]

hello @cameron-martin, do you have any update ?
Thanks!

[matts1]

I've got a prototype working for a module extension for bzlmod (changes stacked on top of this PR). It's still very much a WIP, and needs a lot of cleaning up, but you can at least build examples/bzlmod/crates_repository.

https://github.com/matts1/rules_rust/tree/module_extension_prototype

TLDR on how to use:

crates_vendor = use_extension("@rules_rust//:extensions.bzl", "crates_vendor")
crates_vendor.crates_vendor(
    manifests = ["//:Cargo.toml"],
    cargo_lockfile = "//:Cargo.lock",
    lockfile = "//:cargo-bazel-lock.json",
)
use_repo(
    crates_vendor,
    crates = "crates_crates_repository",
)

[cameron-martin]

I've rebased this PR. Here are the main things that need finishing:

• Support all options when registering toolchains via bzlmod.
• Work out how to run the examples on CI

[cameron-martin]

We also have to consider the API for the module extensions. How many module extensions do we want? Just a single one, called "rust" for everything including crates_universe, etc? One per top-level package, e.g. rust, crates_universe, cargo?

[cameron-martin]

This should now be ready to review.

[UebelAndre]

cc @scentini @krasimirgg Hoping one of you two know anything about bzlmod and would be able to review this 😅

[scentini]

I read up a bit on bzlmod and unless someone with in-depth knowledge offers to do the review, I'll claim the 'expert' title. This PR LGTM, and aside from rebasing and updating the version in MODULE.bazel, I'd try merging.

[Wyverald]

had a brief pass, mostly looks good! I didn't read the actual implementation very deeply.

[scentini]

Thank you @Wyverald !

[meteorcloudy]

Nice! Will there be the next step to integrate with cargo to actually fetch rust dependencies?

[meteorcloudy]

Sorry for being late here. Do we want rules_rust to register a default toolchain so that users don't have to do toolchain registration in their MODULE.bazel file at all? (of course, they can still do it to use their desired toolchain version).

[cameron-martin]

Currently only the toolchains declared in the top level module are registered. There was an unresolved discussion on whether this was the correct thing to do here, but currently the toolchain that rules_rust defines would not get registered.

[meteorcloudy]

Got it, thanks!

[cameron-martin]

Nice! Will there be the next step to integrate with cargo to actually fetch rust dependencies?

@matts1 currently has a PR for this here: #1910