Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reworked CI Build #322

Merged
merged 17 commits into from
Feb 22, 2025
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
142 changes: 112 additions & 30 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,6 @@ on:
push:
pull_request_target:
types: [labeled]
workflow_dispatch:
inputs:
tag:
description: 'Image Tag'
required: false

env:
NODE_VERSION: 20
Expand All @@ -20,7 +15,7 @@ defaults:

jobs:
test:
name: Build and Test
name: Run Tests
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
Expand Down Expand Up @@ -68,37 +63,124 @@ jobs:
- name: Build and test backend
working-directory: backend
run: >
mvn -B clean verify
./mvnw -B clean verify
org.sonarsource.scanner.maven:sonar-maven-plugin:sonar
-Dsonar.projectKey=cryptomator_hub_backend
-Dsonar.organization=cryptomator
-Dsonar.host.url=https://sonarcloud.io
--no-transfer-progress
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
SONAR_TOKEN: ${{ secrets.SONAR_TOKEN }}
- id: get_tag
if: inputs.tag != '' || github.ref_type == 'tag' || contains(github.event.head_commit.message, '[build image]')
run: |
if [[ ! -z "${{ inputs.tag }}" ]]; then
TAG="${{ inputs.tag }}"
elif [[ ${{ github.ref_type }} == 'tag' || ${{ github.ref_name }} == 'develop' ]]; then
TAG="${{ github.ref_name }}"
else
TAG="commit-${{ github.sha }}"
fi
echo tag=${TAG} >> "$GITHUB_OUTPUT"

build-native-image:
name: Build and Push ${{ matrix.arch }} Image
needs: test
if: startsWith(github.ref, 'refs/tags/') || contains(github.event.head_commit.message, '[build image]')
strategy:
fail-fast: false
matrix:
include:
- os: ubuntu-latest
platform: linux/amd64
arch: amd64
- os: ubuntu-24.04-arm
platform: linux/arm64
arch: arm64
runs-on: ${{ matrix.os }}
outputs:
digest_amd64: ${{ steps.digest.outputs.digest_amd64 }}
digest_arm64: ${{ steps.digest.outputs.digest_arm64 }}
permissions:
contents: read
packages: write
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}
cache: 'npm'
cache-dependency-path: frontend/package-lock.json
- name: NPM install
working-directory: frontend
run: npm ci --ignore-scripts
- name: Deploy frontend
working-directory: frontend
run: npm run dist
- name: Ensure to use tagged version
if: startsWith(github.ref, 'refs/tags/')
run: mvn versions:set --file ./backend/pom.xml -DnewVersion=${GITHUB_REF##*/}
- name: Build and push container image
if: github.event.inputs.tag != '' || startsWith(github.ref, 'refs/tags/') || contains(github.event.head_commit.message, '[build image]')
working-directory: backend
run: mvn -B clean package -DskipTests
run: ./mvnw versions:set --file pom.xml -DnewVersion=${GITHUB_REF##*/}
- name: Docker metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ghcr.io/cryptomator/hub
tags: |
type=sha,prefix=,format=short
flavor: |
suffix=-${{ matrix.arch }}
labels: |
org.opencontainers.image.title=Cryptomator Hub
org.opencontainers.image.vendor=Skymatic GmbH
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and Push Container Image
id: push
uses: docker/build-push-action@v6
with:
context: backend
file: backend/src/main/docker/Dockerfile.native
platforms: ${{ matrix.platform }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
push: true
- name: Export Digest
id: digest
run: |
echo "digest_${{ matrix.arch }}=${{ steps.push.outputs.digest }}" >> "$GITHUB_OUTPUT"

multi-arch-image:
name: Build and Push Multi-Arch Image
needs: build-native-image
runs-on: ubuntu-latest
permissions:
id-token: write
contents: read
attestations: write
packages: write
steps:
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Determine short Commit SHA
id: sha
run: echo "short_sha=${LONG_SHA:0:7}" >> "$GITHUB_OUTPUT"
env:
QUARKUS_JIB_PLATFORMS: linux/amd64,linux/arm64/v8
QUARKUS_CONTAINER_IMAGE_TAG: ${{ steps.get_tag.outputs.tag }}
QUARKUS_CONTAINER_IMAGE_BUILD: true
QUARKUS_CONTAINER_IMAGE_PUSH: true
QUARKUS_CONTAINER_IMAGE_REGISTRY: ghcr.io
QUARKUS_CONTAINER_IMAGE_USERNAME: ${{ github.actor }}
QUARKUS_CONTAINER_IMAGE_PASSWORD: ${{ secrets.GITHUB_TOKEN }}
LONG_SHA: ${{ github.sha }}
- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Create Multi-Arch Manifest for ghcr.io/cryptomator/hub:${{ steps.sha.outputs.short_sha }}
run: >
docker buildx imagetools create --tag ghcr.io/cryptomator/hub:${{ steps.sha.outputs.short_sha }}
ghcr.io/cryptomator/hub@${{ needs.build-native-image.outputs.digest_amd64 }}
ghcr.io/cryptomator/hub@${{ needs.build-native-image.outputs.digest_arm64 }}
- name: Retrieve Multi-Arch Digest
id: inspect
run: |
DIGEST=$(docker buildx imagetools inspect ghcr.io/cryptomator/hub:${{ steps.sha.outputs.short_sha }} --format "{{json .Manifest}}" | jq -r .digest)
echo "digest_multiarch=${DIGEST}" >> "$GITHUB_OUTPUT"
- name: Generate artifact attestation
uses: actions/attest-build-provenance@v2
with:
subject-name: ghcr.io/cryptomator/hub
subject-digest: ${{ steps.inspect.outputs.digest_multiarch }}
push-to-registry: true
1 change: 1 addition & 0 deletions backend/.mvn/wrapper/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
maven-wrapper.jar
19 changes: 19 additions & 0 deletions backend/.mvn/wrapper/maven-wrapper.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
wrapperVersion=3.3.2
distributionType=only-script
distributionUrl=https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.9.9/apache-maven-3.9.9-bin.zip
14 changes: 3 additions & 11 deletions backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ If you want to learn more about Quarkus, please visit its website: https://quark

You can run your application in dev mode that enables live coding using:
```shell script
mvn clean quarkus:dev
./mvnw clean quarkus:dev
```

> **_NOTE:_** Quarkus now ships with a Dev UI, which is available in dev mode only at http://localhost:8080/q/dev/.
Expand Down Expand Up @@ -48,20 +48,12 @@ Make sure a container engine is running (required to register the built image lo
Then run this command to build the image:

```shell script
mvn clean package -Dquarkus.container-image.build=true -Dquarkus.container-image.tag=latest
```

### Using containerd or podman

Tell JIB which executable to use (replace `nerctl` with `podman` etc):

```shell script
-Dquarkus.jib.docker-executable-name=$(which nerdctl)
docker build -f src/main/docker/Dockerfile.jvm -t ghcr.io/cryptomator/hub .
```

### Building native images

3x smaller but takes longer to build. Docker VM requires sufficient memory during the build:
```shell script
mvn clean package -Pnative -Dquarkus.container-image.build=true -Dquarkus.native.container-build=true -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-mandrel-builder-image:23.1-java21 -Dquarkus.container-image.tag=latest
docker build -f src/main/docker/Dockerfile.native -t ghcr.io/cryptomator/hub .
```
Loading
Loading