Put info about Rust toolchain into recipe and use it in cargo chef cook

[haaawk]

Problem

1. Currently cargo chef cook uses default Rust toolchain.
2. The project can define a Rust toolchain it wants to be built with using rust-toolchain.toml.
3. This may lead to dependencies being built using different toolchain than the actual project. As a result, cargo build will compile the dependencies again using the toolchain defined in rust-toolchain.toml.

The solution would be to include info about the toolchain into the recipe and use that info in cargo chef cook to make sure it compiles dependencies using the correct toolchain.

Problem reproducer:

1. cargo new --lib dummy
2. cd dummy
3. Create rust-toolchain.toml with following content:

[toolchain]
profile = "default"
channel = "1.70.0"

4. Add tokio as a dependency in Cargo.toml to be able to observe it being compiled twice:

[package]
name = "dummy"
version = "0.1.0"
edition = "2021"

[dependencies]
tokio = { version = "1", features = ["full"] }

5. Create a Dockerfile:

FROM rust:slim-bullseye AS chef
RUN cargo install cargo-chef

FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM chef AS builder
COPY --from=planner /recipe.json recipe.json
RUN cargo chef cook --recipe-path recipe.json
COPY . .
RUN cargo build

6. Run docker build --progress=plain --target builder --no-cache . 2>&1 | tee build-wrong.log
7. Observe dependencies being compiled both by cargo chef cook and cargo build:

...

#10 [builder 2/4] RUN cargo chef cook --recipe-path recipe.json
#10 0.334     Updating crates.io index
#10 0.699  Downloading crates ...
#10 1.053   Downloaded parking_lot_core v0.9.8
#10 1.058   Downloaded bytes v1.4.0
#10 1.064   Downloaded signal-hook-registry v1.4.1
#10 1.065   Downloaded num_cpus v1.16.0
#10 1.072   Downloaded lock_api v0.4.10
#10 1.076   Downloaded tokio-macros v2.1.0
#10 1.080   Downloaded scopeguard v1.2.0
#10 1.083   Downloaded pin-project-lite v0.2.10
#10 1.095   Downloaded autocfg v1.1.0
#10 1.097   Downloaded socket2 v0.4.9
#10 1.100   Downloaded parking_lot v0.12.1
#10 1.103   Downloaded smallvec v1.11.0
#10 1.106   Downloaded mio v0.8.8
#10 1.128   Downloaded tokio v1.29.1
#10 1.189    Compiling libc v0.2.147
#10 1.189    Compiling autocfg v1.1.0
#10 1.189    Compiling proc-macro2 v1.0.66
#10 1.192    Compiling unicode-ident v1.0.11
#10 1.192    Compiling parking_lot_core v0.9.8
#10 1.192    Compiling scopeguard v1.2.0
#10 1.194    Compiling smallvec v1.11.0
#10 1.194    Compiling cfg-if v1.0.0
#10 1.230    Compiling pin-project-lite v0.2.10
#10 1.245    Compiling bytes v1.4.0
#10 1.430    Compiling lock_api v0.4.10
#10 1.430    Compiling tokio v1.29.1
#10 1.850    Compiling quote v1.0.32
#10 1.972    Compiling num_cpus v1.16.0
#10 1.972    Compiling mio v0.8.8
#10 1.974    Compiling socket2 v0.4.9
#10 1.974    Compiling signal-hook-registry v1.4.1
#10 1.980    Compiling syn v2.0.27
#10 2.121    Compiling parking_lot v0.12.1
#10 4.373    Compiling tokio-macros v2.1.0
#10 7.876    Compiling dummy v0.0.1 (/)
#10 9.263     Finished dev [unoptimized + debuginfo] target(s) in 8.96s
#10 DONE 9.4s

...

#12 [builder 4/4] RUN cargo build
#12 0.232 info: syncing channel updates for '1.70.0-x86_64-unknown-linux-gnu'
#12 0.472 info: latest update on 2023-06-01, rust version 1.70.0 (90c541806 2023-05-31)
#12 0.472 info: downloading component 'cargo'
#12 0.705 info: downloading component 'clippy'
#12 0.790 info: downloading component 'rust-docs'
#12 1.259 info: downloading component 'rust-std'
#12 2.162 info: downloading component 'rustc'
#12 4.139 info: downloading component 'rustfmt'
#12 4.230 info: installing component 'cargo'
#12 4.788 info: installing component 'clippy'
#12 5.059 info: installing component 'rust-docs'
#12 7.217 info: installing component 'rust-std'
#12 9.429 info: installing component 'rustc'
#12 14.03 info: installing component 'rustfmt'
#12 14.53    Compiling libc v0.2.147
#12 14.53    Compiling autocfg v1.1.0
#12 14.53    Compiling proc-macro2 v1.0.66
#12 14.53    Compiling unicode-ident v1.0.11
#12 14.53    Compiling parking_lot_core v0.9.8
#12 14.53    Compiling cfg-if v1.0.0
#12 14.53    Compiling smallvec v1.11.0
#12 14.53    Compiling scopeguard v1.2.0
#12 14.61    Compiling pin-project-lite v0.2.10
#12 14.64    Compiling bytes v1.4.0
#12 14.85    Compiling lock_api v0.4.10
#12 14.85    Compiling tokio v1.29.1
#12 15.34    Compiling quote v1.0.32
#12 15.49    Compiling mio v0.8.8
#12 15.49    Compiling socket2 v0.4.9
#12 15.49    Compiling signal-hook-registry v1.4.1
#12 15.49    Compiling num_cpus v1.16.0
#12 15.55    Compiling syn v2.0.27
#12 15.69    Compiling parking_lot v0.12.1
#12 18.01    Compiling tokio-macros v2.1.0
#12 21.67    Compiling dummy v0.1.0 (/)
#12 23.05     Finished dev [unoptimized + debuginfo] target(s) in 8.70s
#12 DONE 23.2s

...

Workaround

If we copy rust-toolchain.toml into chef image, it will make cargo chef cook to use the same Rust toolchain as the project being compiled:

FROM rust:slim-bullseye AS chef
RUN cargo install cargo-chef
COPY rust-toolchain.toml rust-toolchain.toml # <=== THIS IS ADDED

FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM chef AS builder
COPY --from=planner /recipe.json recipe.json
RUN cargo chef cook --recipe-path recipe.json
COPY . .
RUN cargo build

Now, when we build it with docker build --progress=plain --target builder --no-cache . 2>&1 | tee build-correct.log we will see only cargo chef cook compiling dependencies and cargo build not doing that.

Both logs attached
build-correct.log
build-wrong.log

[LukeMathWalker]

I agree that this would be an improvement.

The only caveat is that you need to be careful in how you define your layers, otherwise you'll end up re-installing the toolchain at each stage. But at least it'll be the right one.

[LukeMathWalker]

This has been solved by #254.

[LtdSauce]

Hi,

could it be that this has not been fully solved or that there is a regression making this broken again? #271 looks like it is this issue.