Feature: support dockerized builds of current source-code #2030
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The initial docker builds takes a while with QEMU, while the built docker image caches makes building locally very fast. If this is for the purpose of reuse, one may just pull the latest version from the docker registry, making it more efficient than just building from the sources (without having experience with rust/cargo). Docker builds can be performed as follows: $ docker buildx build --platform linux/amd64 -t marcellodesales/mdbook -o type=docker . [+] Building 2.8s (23/23) FINISHED => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 2.17kB 0.0s => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load metadata for docker.io/library/alpine:3.16.0 2.0s => [internal] load metadata for docker.io/rustlang/rust:nightly-buster-slim 1.4s => [auth] library/alpine:pull token for registry-1.docker.io 0.0s => [auth] rustlang/rust:pull token for registry-1.docker.io 0.0s => [runtime 1/3] FROM docker.io/library/alpine:3.16.0@sha256:686d8c9dfa6f3ccfc8230bc3178d23f84eeaf7e457f36f271ab1acc53015037c 0.0s => => resolve docker.io/library/alpine:3.16.0@sha256:686d8c9dfa6f3ccfc8230bc3178d23f84eeaf7e457f36f271ab1acc53015037c 0.0s => [builder 1/11] FROM docker.io/rustlang/rust:nightly-buster-slim@sha256:609c65daad3c69f9a37717e45d794e2eab99ad488dc5d492b8fc85c97c1df531 0.0s => => resolve docker.io/rustlang/rust:nightly-buster-slim@sha256:609c65daad3c69f9a37717e45d794e2eab99ad488dc5d492b8fc85c97c1df531 0.0s => [internal] load build context 0.0s => => transferring context: 4.94kB 0.0s => CACHED [builder 2/11] RUN apt-get update && apt-get install -y git 0.0s => CACHED [builder 3/11] WORKDIR /usr/src/github.com/rust-lang 0.0s => CACHED [builder 4/11] RUN USER=root cargo new mdBook 0.0s => CACHED [builder 5/11] COPY Cargo.toml Cargo.lock /usr/src/github.com/rust-lang/mdBook 0.0s => CACHED [builder 6/11] COPY examples /usr/src/github.com/rust-lang/mdBook/examples 0.0s => CACHED [builder 7/11] WORKDIR /usr/src/github.com/rust-lang/mdBook 0.0s => CACHED [builder 8/11] RUN rustup target add x86_64-unknown-linux-musl 0.0s => CACHED [builder 9/11] RUN cargo build -vv --config "net.git-fetch-with-cli=true" --target x86_64-unknown-linux-musl --release 0.0s => CACHED [builder 10/11] COPY src /usr/src/github.com/rust-lang/mdBook/src 0.0s => CACHED [builder 11/11] RUN cargo build --locked --bin mdbook --release --target x86_64-unknown-linux-musl 0.0s => CACHED [runtime 2/3] COPY --from=builder /usr/src/github.com/rust-lang/mdBook/target/x86_64-unknown-linux-musl/release/mdbook /usr/local/bin/mdbook 0.0s => CACHED [runtime 3/3] RUN echo "I am running on linux/arm64, building for linux/amd64" > /etc/build.log 0.0s => exporting to oci image format 0.7s => => exporting layers 0.0s => => exporting manifest sha256:f658cf0db29cb5655960164f745fbd841fb0dbba53f2b387e60c5e23f360459b 0.0s => => exporting config sha256:55c34204ee9e1fea12a11af808615f46f89cb428a3c6ff1e46c1bade9282a26a 0.0s => => sending tarball 0.7s => importing to docker Running the server is a matter of the following: * Specifying a container to run exposing a given port number * Specifying the source-code to load mounted as docker volume At this point, just specifying the current dir of sources with md files and the local port number the same as the server, here's what we have: $ ls -la src total 24 drwxr-xr-x 13 mdesales staff 416 Feb 22 14:06 . drwxr-xr-x 12 mdesales staff 384 Feb 22 14:06 .. -rw-r--r-- 1 mdesales staff 980 Feb 22 14:06 README.md -rw-r--r-- 1 mdesales staff 4764 Feb 22 14:06 SUMMARY.md drwxr-xr-x 10 mdesales staff 320 Feb 22 14:06 assets drwxr-xr-x 7 mdesales staff 224 Feb 22 14:06 data drwxr-xr-x 19 mdesales staff 608 Feb 22 14:06 infra drwxr-xr-x 7 mdesales staff 224 Feb 22 14:06 monitoring drwxr-xr-x 7 mdesales staff 224 Feb 22 14:06 onboard drwxr-xr-x 17 mdesales staff 544 Feb 22 14:06 services drwxr-xr-x 4 mdesales staff 128 Feb 22 14:06 structure drwxr-xr-x 43 mdesales staff 1376 Feb 22 14:06 support drwxr-xr-x 17 mdesales staff 544 Feb 22 14:06 tips $ docker run -ti -v $(pwd)/src:/src -p 3000:3000 marcellodesales/mdbook WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested 2023-02-23 03:08:29 [INFO] (mdbook::book): Book building has started 2023-02-23 03:08:29 [INFO] (mdbook::book): Running the html backend 2023-02-23 03:08:34 [INFO] (mdbook::cmd::serve): Serving on: http://0.0.0.0:3000 2023-02-23 03:08:34 [INFO] (warp::server): Server::run; addr=0.0.0.0:3000 2023-02-23 03:08:34 [INFO] (warp::server): listening on http://0.0.0.0:3000 2023-02-23 03:08:34 [INFO] (mdbook::cmd::watch): Listening for changes... At this point, one may just check the service locally as follows: $ curl -I http://localhost:3000/ HTTP/1.1 200 OK content-length: 26343 content-type: text/html accept-ranges: bytes last-modified: Thu, 23 Feb 2023 03:08:29 GMT date: Thu, 23 Feb 2023 03:08:35 GMT $ curl http://localhost:3000/ | grep mdBook % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 <!-- Book generated using mdBook --> 100 26343 100 26343 0 0 366k 0 --:--:-- --:--:-- --:--:-- 467k
KFearsoff
suggested changes
Nov 18, 2023
|
@rustbot label +A-Installation +C-enhancement -S-waiting-on-review +S-waiting-on-author |
|
Thank you @KFearsoff for the awesome review!!! I will resolve the conversations and send additional PR changes! |
The sequence now guarantees the caches aren't broken when Cargo.* files change for the dependencies.
$ docker compose build [+] Building 1.9s (24/24) FINISHED docker-container:mac-linux-builder => [mdbook internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 2.47kB 0.0s => [mdbook internal] load metadata for docker.io/rustlang/rust:nightly-buster-slim 1.4s => [mdbook internal] load metadata for docker.io/library/alpine:3.18.4 1.3s => [mdbook auth] library/alpine:pull token for registry-1.docker.io 0.0s => [mdbook auth] rustlang/rust:pull token for registry-1.docker.io 0.0s => [mdbook internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [mdbook internal] load build context 0.0s => => transferring context: 4.94kB 0.0s => [mdbook runtime 1/2] FROM docker.io/library/alpine:3.18.4@sha256:eece025e432126ce23f223450a0326fbebde39cdf496a85d8c016293fc851978 0.0s => => resolve docker.io/library/alpine:3.18.4@sha256:eece025e432126ce23f223450a0326fbebde39cdf496a85d8c016293fc851978 0.0s => [mdbook builder 1/13] FROM docker.io/rustlang/rust:nightly-buster-slim@sha256:328863752ff0c44a09be7c19d612de90e2a59fc2547496317cc2b01b0c1d1bd0 0.0s => => resolve docker.io/rustlang/rust:nightly-buster-slim@sha256:328863752ff0c44a09be7c19d612de90e2a59fc2547496317cc2b01b0c1d1bd0 0.0s => CACHED [mdbook builder 2/13] RUN apt-get update && apt-get install -y git 0.0s => CACHED [mdbook builder 3/13] WORKDIR /usr/src/github.com/rust-lang 0.0s => CACHED [mdbook builder 4/13] RUN USER=root cargo new mdBook 0.0s => CACHED [mdbook builder 5/13] WORKDIR /usr/src/github.com/rust-lang/mdBook 0.0s => CACHED [mdbook builder 6/13] RUN rustup default stable 0.0s => CACHED [mdbook builder 7/13] RUN rustup target add x86_64-unknown-linux-musl 0.0s => CACHED [mdbook builder 8/13] COPY Cargo.toml Cargo.lock /usr/src/github.com/rust-lang/mdBook 0.0s => CACHED [mdbook builder 9/13] COPY examples /usr/src/github.com/rust-lang/mdBook/examples 0.0s => CACHED [mdbook builder 10/13] RUN cargo build -vv --config "net.git-fetch-with-cli=true" --target x86_64-unknown-linux-musl --release 0.0s => CACHED [mdbook builder 11/13] WORKDIR /usr/src/github.com/rust-lang/mdBook 0.0s => CACHED [mdbook builder 12/13] COPY src /usr/src/github.com/rust-lang/mdBook/src 0.0s => CACHED [mdbook builder 13/13] RUN cargo build --locked --bin mdbook --release --target x86_64-unknown-linux-musl 0.0s => CACHED [mdbook runtime 2/2] COPY --from=builder /usr/src/github.com/rust-lang/mdBook/target/x86_64-unknown-linux-musl/release/mdbook /usr/local/bin/mdbook 0.0s => [mdbook] exporting to docker image format 0.3s => => exporting layers 0.0s => => exporting manifest sha256:f4bff2d67ccffb4741203fb90e532154b0e1714343d361ebc1ca682de293fdc9 0.0s => => exporting config sha256:d222b963c70cb9ac09fd3ae7c4c51e4605340c6d5a0930cf6b9c85ce145a5b74 0.0s => => sending tarball 0.3s => [mdbook mdbook] importing to docker 0.0s
|
@KFearsoff Everything is better now... Please review it and let me know! |
KFearsoff
approved these changes
Nov 23, 2023
|
@rustbot label -S-waiting-on-author |
According to the spec, we need to end the command with a "/"
|
@KFearsoff Just added the "/" to the last conversation needed...! Hope this is it!!! π |
|
Happy new year! When will this be merged? @rustbot ready |
|
Do we have anyone to review this? |
|
Thanks for the PR! Unfortunately I don't really understand what this PR is for, the description seems to be missing an explanation of why it is being proposed. There are precompiled binaries available for several platforms, which should be rather easy to download and use. If this is for development of mdbook itself, I think installing Rust shouldn't be too much of a hurdle to get started, and there shouldn't be anything else besides that. There are official docker images if that is your preferred method of using it. I think if you want, you can put a page on the wiki with your example workflow. Otherwise, I probably don't want to maintain a docker image. |
|
Would be pretty amazing to have this docker based set-up. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
π Support containers
25MBbuilt from the current codeproddocker images to the official Docker Registry.ποΈ Docker image build
The build below is performed from the source-code, and not from an external installation. It is building a single platform version for simplicity.
π Running a server
π³ Docker compose option
docker-compose.yamlfor users to run locallyπ Testing in container
docker run -ti --network mdbook_default alpine/lynx mdbook:3000