Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: remove nix from gh workflows #21572

Merged
merged 4 commits into from
Sep 11, 2024
Merged

ci: remove nix from gh workflows #21572

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
02efb47
feat: remove nix from ci
auricom Sep 6, 2024
3f07bf0
Merge branch 'main' into claude/gomod2nix
auricom Sep 9, 2024
22c9201
chore: coderabbitai suggestions
auricom Sep 9, 2024
010137b
Merge branch 'main' into claude/gomod2nix
auricom Sep 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions .github/scripts/install-rocksdb.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/usr/bin/env bash
set -Eeuo pipefail

if [ -z "$ROCKSDB_VERSION" ]; then
echo "ROCKSDB_VERSION is not set."
exit 1
fi

# Update and install dependencies
sudo apt update && sudo apt-get install -y libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev libzstd-dev build-essential

# Clone RocksDB repository
git clone https://github.com/facebook/rocksdb.git /home/runner/rocksdb
cd /home/runner/rocksdb || exit 1
git checkout "v${ROCKSDB_VERSION}"

auricom marked this conversation as resolved.
Show resolved Hide resolved
# Build shared library
sudo make -j "$(nproc --all)" shared_lib
sudo cp --preserve=links ./librocksdb.* /usr/local/lib/
sudo cp -r ./include/rocksdb/ /usr/local/include/
auricom marked this conversation as resolved.
Show resolved Hide resolved
34 changes: 28 additions & 6 deletions .github/workflows/build.yml
View file
Open in desktop
julienrbrt marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ concurrency:
group: ci-${{ github.ref }}-build
cancel-in-progress: true

env:
ROCKSDB_VERSION: 8.11.3

jobs:
build:
runs-on: ubuntu-latest
Expand All @@ -21,12 +24,33 @@ jobs:
go-arch: ["amd64", "arm", "arm64"]
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- uses: actions/setup-go@v5
with:
go-version: "1.23"
check-latest: true
- name: Fix permissions for cache
run: sudo chown $(whoami) /usr/local/lib /usr/local/include
- name: Restore rocksdb libraries cache
id: cache-rocksdb
if: matrix.go-arch == 'amd64'
uses: actions/cache/restore@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-${{ matrix.go-arch }}
- name: Install rocksdb
if: matrix.go-arch == 'amd64' && steps.cache-rocksdb.outputs.cache-hit != 'true'
id: install_rocksdb
run: ./.github/scripts/install-rocksdb.sh
- name: Saves rocksdb libraries cache
if: matrix.go-arch == 'amd64' && steps.install_rocksdb.outcome == 'success'
uses: actions/cache/restore@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-${{ matrix.go-arch }}
###################
#### Build App ####
###################
Expand All @@ -35,10 +59,8 @@ jobs:
- name: Build Legacy
run: GOARCH=${{ matrix.go-arch }} COSMOS_BUILD_OPTIONS=legacy make build
- name: Build with rocksdb backend
if: |
env.GIT_DIFF &&
julienrbrt marked this conversation as resolved.
Show resolved Hide resolved
matrix.go-arch == 'amd64'
run: nix run . -- version --long
if: matrix.go-arch == 'amd64'
run: GOARCH=${{ matrix.go-arch }} COSMOS_BUILD_OPTIONS="rocksdb" make build
###################
## Build Tooling ##
###################
Expand Down
36 changes: 30 additions & 6 deletions .github/workflows/lint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,16 @@ on:
merge_group:
permissions:
contents: read

env:
ROCKSDB_VERSION: 8.11.3

jobs:
golangci:
name: golangci-lint
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

YES, thank you!!
We would be able to remove so much boilerplate from scripts, ci etc..

- uses: DeterminateSystems/magic-nix-cache-action@main
- uses: actions/setup-go@v5
with:
go-version: "1.23"
Expand All @@ -28,13 +30,35 @@ jobs:
Makefile
**/Makefile
.golangci.yml
- name: Fix permissions for cache
run: sudo chown $(whoami) /usr/local/lib /usr/local/include
- name: Restore rocksdb libraries cache
id: cache-rocksdb
uses: actions/cache/restore@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: Install rocksdb
if: env.GIT_DIFF && steps.cache-rocksdb.outputs.cache-hit != 'true'
id: install_rocksdb
run: ./.github/scripts/install-rocksdb.sh
- name: Saves rocksdb libraries cache
if: steps.install_rocksdb.outcome == 'success'
uses: actions/cache/restore@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: run linting (long)
if: env.GIT_DIFF
id: lint_long
run: |
nix develop -c make lint
make lint
env:
NIX: 1
ROCKSDB: 1
- uses: technote-space/get-diff-action@v6.1.2
if: steps.lint_long.outcome == 'skipped'
with:
Expand All @@ -57,8 +81,8 @@ jobs:
- name: run linting (short)
if: steps.lint_long.outcome == 'skipped' && env.GIT_DIFF
run: |
nix develop -c make lint
make lint
env:
GIT_DIFF: ${{ env.GIT_DIFF }}
LINT_DIFF: 1
NIX: 1
ROCKSDB: 1
57 changes: 50 additions & 7 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ concurrency:
group: ci-${{ github.ref }}-tests
cancel-in-progress: true

env:
ROCKSDB_VERSION: 8.11.3

jobs:
split-test-files:
runs-on: ubuntu-latest
Expand Down Expand Up @@ -774,11 +777,9 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- uses: actions/setup-go@v5
with:
go-version: "1.20"
go-version: "1.23"
check-latest: true
cache: true
cache-dependency-path: store/go.sum
Expand All @@ -789,11 +790,33 @@ jobs:
store/**/*.go
store/go.mod
store/go.sum
- name: Fix permissions for cache
run: sudo chown $(whoami) /usr/local/lib /usr/local/include
- name: Restore rocksdb libraries cache
id: cache-rocksdb
uses: actions/cache/restore@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: Install rocksdb
if: env.GIT_DIFF && steps.cache-rocksdb.outputs.cache-hit != 'true'
id: install_rocksdb
run: ./.github/scripts/install-rocksdb.sh
- name: Saves rocksdb libraries cache
if: steps.install_rocksdb.outcome == 'success'
uses: actions/cache/restore@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: tests
if: env.GIT_DIFF
run: |
cd store
nix develop .. -c go test -mod=readonly -timeout 30m -coverprofile=coverage.out -covermode=atomic -tags='norace ledger test_ledger_mock rocksdb' ./...
go test -mod=readonly -timeout 30m -coverprofile=coverage.out -covermode=atomic -tags='norace ledger test_ledger_mock rocksdb' ./...
- name: sonarcloud
if: ${{ env.GIT_DIFF && !github.event.pull_request.draft && env.SONAR_TOKEN != null }}
uses: SonarSource/sonarcloud-github-action@master
Expand All @@ -809,8 +832,6 @@ jobs:
fail-fast: false
steps:
- uses: actions/checkout@v4
- uses: DeterminateSystems/nix-installer-action@main
- uses: DeterminateSystems/magic-nix-cache-action@main
- uses: actions/setup-go@v5
with:
go-version: "1.23"
Expand All @@ -824,11 +845,33 @@ jobs:
store/v2/**/*.go
store/v2/go.mod
store/v2/go.sum
- name: Fix permissions for cache
run: sudo chown $(whoami) /usr/local/lib /usr/local/include
- name: Restore rocksdb libraries cache
id: cache-rocksdb
uses: actions/cache/restore@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: Install rocksdb
if: env.GIT_DIFF && steps.cache-rocksdb.outputs.cache-hit != 'true'
id: install_rocksdb
run: ./.github/scripts/install-rocksdb.sh
- name: Saves rocksdb libraries cache
if: steps.install_rocksdb.outcome == 'success'
uses: actions/cache/restore@v4
with:
path: |
/usr/local/lib/librocksdb.*
/usr/local/include/rocksdb
key: ${{ runner.os }}-rocksdb-${{ env.ROCKSDB_VERSION }}-amd64
- name: test & coverage report creation
if: env.GIT_DIFF
run: |
cd store/v2
nix develop ../.. -c go test -mod=readonly -timeout 30m -coverprofile=coverage.out -covermode=atomic -tags='norace ledger test_ledger_mock rocksdb' ./...
go test -mod=readonly -timeout 30m -coverprofile=coverage.out -covermode=atomic -tags='norace ledger test_ledger_mock rocksdb' ./...
- name: sonarcloud
if: ${{ env.GIT_DIFF && !github.event.pull_request.draft && env.SONAR_TOKEN != null }}
uses: SonarSource/sonarcloud-github-action@master
Expand Down
86 changes: 0 additions & 86 deletions flake.lock
View file
Open in desktop

This file was deleted.

Loading
Loading