Skip to content
This repository was archived by the owner on Nov 15, 2023. It is now read-only.

[CI] Docker image updates #1799

Merged
merged 19 commits into from
Oct 19, 2020
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .github/workflows/publish-docker-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Publish Docker image for new releases

on:
release:
types:
- published

jobs:
main:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Cache Docker layers
uses: actions/cache@v2
- name: Login to Dockerhub
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push
id: docker_build
uses: docker/build-push-action@v2
with:
push: true
file: scripts/docker/Dockerfile.release
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can actually cache layers in docker builds with Buildx, which gives a significant performance boost here.

First you'd need to set up caching, with a separate step (prior build-push-action):

      - name: Cache Docker layers
        uses: actions/cache@v2
        with:
          path: /tmp/.buildx-cache
          key: ${{ runner.os }}-buildx-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-buildx-

And then in your build-push-action itself you just add

          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,dest=/tmp/.buildx-cache

properties, and everything seems to work. (At least on that silly side-project of mine.)

tags: |
parity/polkadot:latest
parity/polkadot:${{ github.event.release.tag_name }}
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache
- name: Image digest
run: echo ${{ steps.docker_build.outputs.digest }}
6 changes: 5 additions & 1 deletion .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -212,12 +212,16 @@ generate-impl-guide:
- EXTRATAG="$(cat ./artifacts/EXTRATAG)"
- echo "Polkadot version = ${VERSION} (EXTRATAG ${EXTRATAG})"

publish-docker-release:
publish-docker:
<<: *publish-build
image: docker:stable
services:
- docker:dind
<<: *collect-artifacts
# Don't run on releases - this is handled by the Github Action here:
# .github/workflows/publish-docker-release.yml
rules:
- if: $CI_COMMIT_REF_NAME !~ /^v[0-9]+\.[0-9]+\.[0-9]+$/ # i.e., v1.2.3 but NOT v1.2.3-rc1
variables:
DOCKER_HOST: tcp://localhost:2375
DOCKER_DRIVER: overlay2
Expand Down
18 changes: 10 additions & 8 deletions doc/docker.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,38 @@
### The easiest way

The easiest/faster option is to use the latest image.
The easiest/faster option to run Polkadot in docker is to use the latest
release images. These are small images that use the latest official release of
the polkadot binary, pulled from our package repository.

Let´s first check the version we have. The first time you run this command, the polkadot docker image will be downloaded. This takes a bit of time and bandwidth, be patient:

```bash
docker run --rm -it chevdor/polkadot:latest polkadot --version
docker run --rm -it parity/polkadot:latest polkadot --version
```

You can also pass any argument/flag that polkadot supports:

```bash
docker run --rm -it chevdor/polkadot:latest polkadot --chain westend --name "PolkaDocker"
docker run --rm -it parity/polkadot:latest polkadot --chain westend --name "PolkaDocker"
```

Once you are done experimenting and picking the best node name :) you can start polkadot as daemon, exposes the polkadot ports and mount a volume that will keep your blockchain data locally:

```bash
docker run -d -p 30333:30333 -p 9933:9933 -v /my/local/folder:/data chevdor/polkadot:latest polkadot --chain westend
docker run -d -p 30333:30333 -p 9933:9933 -v /my/local/folder:/data parity/polkadot:latest polkadot --chain westend
```

Additionally if you want to have custom node name you can add the `--name "YourName"` at the end

```bash
docker run -d -p 30333:30333 -p 9933:9933 -v /my/local/folder:/data chevdor/polkadot:latest polkadot --chain westend --name "PolkaDocker"
docker run -d -p 30333:30333 -p 9933:9933 -v /my/local/folder:/data parity/polkadot:latest polkadot --chain westend --name "PolkaDocker"
```

```bash
docker run -d -p 30333:30333 -p 9933:9933 -v /my/local/folder:/data chevdor/polkadot:latest polkadot --rpc-external --chain westend
docker run -d -p 30333:30333 -p 9933:9933 -v /my/local/folder:/data parity/polkadot:latest polkadot --rpc-external --chain westend
```

if you want to connect to rpc port 9933, then must add polkadot startup parameter: `--rpc-external`.
If you want to connect to rpc port 9933, then must add polkadot startup parameter: `--rpc-external`.

**Note:** The `--chain westend` argument is important and you need to add it to the command line. If you are running older node versions (pre 0.3) you don't need it.

Expand Down Expand Up @@ -68,7 +70,7 @@ If you run into issues with polkadot when using docker, please run the following
(replace the tag with the appropriate one if you do not use latest):

```bash
docker run --rm -it chevdor/polkadot:latest polkadot --version
docker run --rm -it parity/polkadot:latest polkadot --version
```

This will show you the polkadot version as well as the git commit ref that was used to build your container.
Expand Down
20 changes: 4 additions & 16 deletions docker/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,33 +1,21 @@
FROM phusion/baseimage:0.11 as builder
LABEL maintainer "chevdor@gmail.com"
FROM paritytech/ci-linux:production as builder
LABEL description="This is the build stage for Polkadot. Here we create the binary."

ARG PROFILE=release
WORKDIR /polkadot

COPY . /polkadot

RUN apt-get update && \
apt-get upgrade -y && \
apt-get install -y cmake pkg-config libssl-dev git clang
RUN curl https://sh.rustup.rs -sSf | sh -s -- -y && \
export PATH=$PATH:$HOME/.cargo/bin && \
scripts/init.sh && \
cargo build --$PROFILE
RUN cargo build --$PROFILE

# ===== SECOND STAGE ======

FROM phusion/baseimage:0.11
LABEL maintainer "chevdor@gmail.com"
FROM debian:buster-slim
LABEL description="This is the 2nd stage: a very small image where we copy the Polkadot binary."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could use the same Label as in release.Dockerfile below

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The intention of this Dockerfile is to be used by users - i.e., for dev purposes etc., so it doesn't make sense to have vendor info and references to variables (like $VCS_REF) that won't be present

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OCI recommends filling labels as a best practice. At least it can be handy to know where is the code of this Dockerfile, this could be in label.

ARG PROFILE=release
COPY --from=builder /polkadot/target/$PROFILE/polkadot /usr/local/bin

RUN mv /usr/share/ca* /tmp && \
rm -rf /usr/share/* && \
mv /tmp/ca-certificates /usr/share/ && \
rm -rf /usr/lib/python* && \
useradd -m -u 1000 -U -s /bin/sh -d /polkadot polkadot && \
RUN useradd -m -u 1000 -U -s /bin/sh -d /polkadot polkadot && \
mkdir -p /polkadot/.local/share/polkadot && \
chown -R polkadot:polkadot /polkadot/.local && \
ln -s /polkadot/.local/share/polkadot /data && \
Expand Down
4 changes: 2 additions & 2 deletions docker/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ cd $PROJECT_ROOT

# Find the current version from Cargo.toml
VERSION=`grep "^version" ./Cargo.toml | egrep -o "([0-9\.]+)"`
GITUSER=chevdor
GITUSER=parity
GITREPO=polkadot

# Build the image
echo "Building ${GITUSER}/${GITREPO}:latest docker image, hang on!"
time docker build -f ./docker/Dockerfile --build-arg PROFILE=release -t ${GITUSER}/${GITREPO}:latest .
time docker build -f ./docker/Dockerfile --build-arg RUSTC_WRAPPER= --build-arg PROFILE=release -t ${GITUSER}/${GITREPO}:latest .

# Show the list of available images for this repo
echo "Image is ready"
Expand Down
46 changes: 46 additions & 0 deletions scripts/docker/release.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
FROM debian:buster-slim

# metadata
ARG VCS_REF
ARG BUILD_DATE

LABEL io.parity.image.authors="devops-team@parity.io" \
io.parity.image.vendor="Parity Technologies" \
io.parity.image.title="parity/polkadot" \
io.parity.image.description="polkadot: a platform for web3" \
io.parity.image.source="https://github.com/paritytech/polkadot/blob/${VCS_REF}/scripts/docker/Dockerfile" \
io.parity.image.revision="${VCS_REF}" \
io.parity.image.created="${BUILD_DATE}" \
io.parity.image.documentation="https://github.com/paritytech/polkadot/"

# show backtraces
ENV RUST_BACKTRACE 1

# install tools and dependencies
RUN apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get upgrade -y && \
DEBIAN_FRONTEND=noninteractive apt-get install -y \
libssl1.1 \
ca-certificates \
curl \
gnupg && \
gpg --recv-keys --keyserver hkps://keys.mailvelope.com 9D4B2B6EB8F97156D19669A9FF0812D491B96798 && \
gpg --export 9D4B2B6EB8F97156D19669A9FF0812D491B96798 > /usr/share/keyrings/parity.gpg && \
echo 'deb [signed-by=/usr/share/keyrings/parity.gpg] https://releases.parity.io/deb release main' > /etc/apt/sources.list.d/parity.list && \
apt update && \
apt install polkadot && \
# apt cleanup
apt-get autoremove -y && \
apt-get clean && \
find /var/lib/apt/lists/ -type f -not -name lock -delete

USER polkadot

# check if executable works in this container
RUN /usr/bin/polkadot --version

EXPOSE 30333 9933 9944
VOLUME ["/polkadot"]

ENTRYPOINT ["/usr/bin/polkadot"]