Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move LLVS to EE #126

Merged
merged 49 commits into from
Feb 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
e3ea932
Move LLVS to EE
astef Dec 19, 2024
8bfb31b
also move snapshots restore to EE
astef Dec 20, 2024
22210fe
fix linter
astef Dec 20, 2024
ef104f4
[docs] Add labels list for BlockDevice (#121)
duckhawk Jan 23, 2025
8df78d8
[github] CI: add editions
krpsh123 Feb 3, 2025
d216988
include llvs always
astef Feb 6, 2025
589735a
added webhook
NikolayDemchuk Feb 6, 2025
a31a695
fixes
NikolayDemchuk Feb 6, 2025
4fde4d2
change
NikolayDemchuk Feb 6, 2025
5fef574
fixes
NikolayDemchuk Feb 6, 2025
ad96530
fixes
NikolayDemchuk Feb 6, 2025
91f82be
changed go version
NikolayDemchuk Feb 6, 2025
7a1c568
fix
NikolayDemchuk Feb 6, 2025
b180e15
try
astef Feb 6, 2025
39a5d15
[webhooks] fix werf
krpsh123 Feb 6, 2025
7c7cab8
bump go version to 1.23.5
krpsh123 Feb 6, 2025
d0724ff
update go
astef Feb 6, 2025
3d5d082
fix yamls
astef Feb 7, 2025
92377bc
fix caBundle
astef Feb 7, 2025
8c8937a
better message
astef Feb 7, 2025
ca24d46
optionally disable validating webhook when featureLLVSEnabled
astef Feb 7, 2025
ba5fad8
fix overwriting openapi folder in bundle
astef Feb 7, 2025
67ca9a1
fix bundle problem; change build trigger
astef Feb 7, 2025
c5e682f
fix typo
astef Feb 7, 2025
5c692a1
fix reversed logic of featureLLVSEnabled
astef Feb 7, 2025
dec45c6
upgrade Go in workflows
astef Feb 7, 2025
9a764f2
fix linter
astef Feb 7, 2025
1632ff2
some changes
AleksZimin Feb 9, 2025
280f6d8
another fix
AleksZimin Feb 9, 2025
d2c8893
another fixes2
AleksZimin Feb 9, 2025
62fe597
fixes
AleksZimin Feb 9, 2025
766f00e
fixes
AleksZimin Feb 9, 2025
9f72440
some fixes
AleksZimin Feb 9, 2025
4bb050e
change workflows
AleksZimin Feb 9, 2025
6413fe0
some changes in choice edition
AleksZimin Feb 9, 2025
4724b5f
change werf
AleksZimin Feb 9, 2025
9c36325
fix
AleksZimin Feb 10, 2025
c25ac75
fix default MODULE_EDITION
krpsh123 Feb 10, 2025
6bef14e
leave conditional build only in feature flags
astef Feb 10, 2025
b202326
revert "leave conditional build only in feature flags"
astef Feb 12, 2025
09261c8
temp
astef Feb 12, 2025
cf9d419
remove redundand validation logic
astef Feb 13, 2025
769e9fc
fix webhok, remove choice-edition.yaml
astef Feb 13, 2025
14308dc
fix merge conflict
astef Feb 13, 2025
8b7ee5f
fix edition selection algorithm
astef Feb 13, 2025
e4b486b
avoid non-zero status
astef Feb 13, 2025
59b76da
avoid non-zero status 2
astef Feb 13, 2025
9089dc5
minor fixes after self-review
astef Feb 15, 2025
6eaa8e3
pass DEFAULT_EDITION from vars
astef Feb 15, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 44 additions & 9 deletions .github/workflows/build_dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,45 +6,80 @@ env:
MODULES_MODULE_NAME: ${{ vars.MODULE_NAME }}
MODULES_MODULE_SOURCE: ${{ vars.DEV_MODULE_SOURCE }}
MODULES_REGISTRY_LOGIN: ${{ vars.DEV_MODULES_REGISTRY_LOGIN }}
DEFAULT_EDITION: ${{ vars.DEFAULT_EDITION }}
MODULES_REGISTRY_PASSWORD: ${{ secrets.DEV_MODULES_REGISTRY_PASSWORD }}
SOURCE_REPO: "${{ secrets.SOURCE_REPO }}"



on:
#pull_request:
pull_request:
types: [opened, reopened, labeled, unlabeled, synchronize]
# call from trivy_image_check.yaml, which in turn call from pull_request
# https://stackoverflow.com/a/71489231
workflow_call:
push:
branches:
- main
# make this job as dependency for trivy_image_check workflow
# https://stackoverflow.com/a/71489231
workflow_call:

defaults:
run:
shell: bash

jobs:
set_edition:
runs-on: [self-hosted, regular, selectel]
name: Set edition
outputs:
module_edition: ${{ steps.set-vars.outputs.MODULE_EDITION }}
steps:
- name: Get Pull Request Labels
id: get-labels
uses: actions/github-script@v7
with:
script: |
if (context.eventName === "pull_request" || context.eventName === "pull_request_target" ) {
const prNumber = context.payload.pull_request.number;
const { data: labels } = await github.rest.issues.listLabelsOnIssue({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
});
return labels.map(label => label.name);
} else {
return [];
}
result-encoding: string

- name: Set vars
id: set-vars
run: |
MODULE_EDITION=$DEFAULT_EDITION
EDITION=$(echo "${{ steps.get-labels.outputs.result }}" | grep -Po "edition/\K\w+" | head -n 1 || true)
[[ -n $EDITION ]] && MODULE_EDITION=$EDITION
echo "MODULE_EDITION=$MODULE_EDITION" >> "$GITHUB_OUTPUT"
dev_setup_build:
runs-on: [self-hosted, regular, selectel]
name: Build and Push images
needs: [set_edition]
env:
MODULE_EDITION: ${{needs.set_edition.outputs.module_edition}}
steps:
- name: Set vars for PR
if: ${{ github.ref_name != 'main' }}
run: |
MODULES_MODULE_TAG="$(echo pr${{ github.ref_name }} | sed 's/\/.*//g')"
echo "MODULES_MODULE_TAG=$MODULES_MODULE_TAG" >> "$GITHUB_ENV"
shell: bash
- name: Set vars for main
if: ${{ github.ref_name == 'main' }}
run: |
echo "MODULES_MODULE_TAG=${{ github.ref_name }}" >> "$GITHUB_ENV"
shell: bash
- name: Print vars
run: |
echo MODULES_REGISTRY=$MODULES_REGISTRY
echo CI_COMMIT_REF_NAME=$CI_COMMIT_REF_NAME
echo MODULES_MODULE_NAME=$MODULES_MODULE_NAME
echo MODULES_MODULE_SOURCE=$MODULES_MODULE_SOURCE
echo MODULES_MODULE_TAG=$MODULES_MODULE_TAG
shell: bash
echo MODULE_EDITION=$MODULE_EDITION

- uses: actions/checkout@v4
- uses: deckhouse/modules-actions/setup@v1
Expand Down
19 changes: 14 additions & 5 deletions .github/workflows/build_prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ on:
tags:
- '**'

defaults:
run:
shell: bash

jobs:
prod_ce_setup_build:
runs-on: [self-hosted, regular, selectel]
Expand All @@ -23,12 +27,13 @@ jobs:
- name: SET VAR
run: |
echo "MODULES_MODULE_SOURCE=$MODULES_REGISTRY/$MODULE_SOURCE_NAME/ce/modules" >> "$GITHUB_ENV"
echo "MODULE_EDITION=ce" >> "$GITHUB_ENV"
- run: |
echo $MODULES_REGISTRY
echo $MODULES_MODULE_NAME
echo $MODULES_MODULE_SOURCE
echo $MODULES_MODULE_TAG
shell: bash
echo $MODULE_EDITION
name: Show vars

- uses: actions/checkout@v4
Expand Down Expand Up @@ -56,12 +61,13 @@ jobs:
- name: SET VAR
run: |
echo "MODULES_MODULE_SOURCE=$MODULES_REGISTRY/$MODULE_SOURCE_NAME/ee/modules" >> "$GITHUB_ENV"
echo "MODULE_EDITION=ee" >> "$GITHUB_ENV"
- run: |
echo $MODULES_REGISTRY
echo $MODULES_MODULE_NAME
echo $MODULES_MODULE_SOURCE
echo $MODULES_MODULE_TAG
shell: bash
echo $MODULE_EDITION
name: Show vars

- uses: actions/checkout@v4
Expand Down Expand Up @@ -89,12 +95,13 @@ jobs:
- name: SET VAR
run: |
echo "MODULES_MODULE_SOURCE=$MODULES_REGISTRY/$MODULE_SOURCE_NAME/fe/modules" >> "$GITHUB_ENV"
echo "MODULE_EDITION=ee" >> "$GITHUB_ENV"
- run: |
echo $MODULES_REGISTRY
echo $MODULES_MODULE_NAME
echo $MODULES_MODULE_SOURCE
echo $MODULES_MODULE_TAG
shell: bash
echo $MODULE_EDITION
name: Show vars

- uses: actions/checkout@v4
Expand Down Expand Up @@ -122,12 +129,13 @@ jobs:
- name: SET VAR
run: |
echo "MODULES_MODULE_SOURCE=$MODULES_REGISTRY/$MODULE_SOURCE_NAME/se/modules" >> "$GITHUB_ENV"
echo "MODULE_EDITION=se" >> "$GITHUB_ENV"
- run: |
echo $MODULES_REGISTRY
echo $MODULES_MODULE_NAME
echo $MODULES_MODULE_SOURCE
echo $MODULES_MODULE_TAG
shell: bash
echo $MODULE_EDITION
name: Show vars

- uses: actions/checkout@v4
Expand Down Expand Up @@ -155,12 +163,13 @@ jobs:
- name: SET VAR
run: |
echo "MODULES_MODULE_SOURCE=$MODULES_REGISTRY/$MODULE_SOURCE_NAME/se-plus/modules" >> "$GITHUB_ENV"
echo "MODULE_EDITION=seplus" >> "$GITHUB_ENV"
- run: |
echo $MODULES_REGISTRY
echo $MODULES_MODULE_NAME
echo $MODULES_MODULE_SOURCE
echo $MODULES_MODULE_TAG
shell: bash
echo $MODULE_EDITION
name: Show vars

- uses: actions/checkout@v4
Expand Down
24 changes: 14 additions & 10 deletions .github/workflows/go_lint.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
name: Go linter for images

env:
GO_BUILD_TAGS: "ce se seplus ee csepro"

on:
pull_request:
push:
Expand All @@ -18,26 +21,27 @@ jobs:
- name: Setup Go environment
uses: actions/setup-go@v5
with:
go-version: '1.22'
go-version: '1.23.5'

- name: Install golangci-lint
run: go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.59.1
run: go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.61.0

- name: Run Go lint
run: |
basedir=$(pwd)
failed='false'
for dir in $(find images -type d); do
if ls $dir/go.mod &> /dev/null; then
echo "Running linter in $dir"
cd $dir
golangci-lint run
for i in $(find images -type f -name go.mod);do
dir=$(echo $i | sed 's/go.mod$//')
cd $basedir/$dir
# check all editions
for edition in $GO_BUILD_TAGS ;do
echo "Running linter in $dir (edition: $edition)"
golangci-lint run --build-tags $edition
if [ $? -ne 0 ]; then
echo "Linter failed in $dir"
echo "Linter failed in $dir (edition: $edition)"
failed='true'
fi
cd $basedir
fi
done
done
if [ $failed == 'true' ]; then
exit 1
Expand Down
12 changes: 9 additions & 3 deletions .github/workflows/go_modules_check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
- name: Setup Go environment
uses: actions/setup-go@v5
with:
go-version: '1.22'
go-version: '1.23.5'

- name: Run Go modules version check
run: |
Expand All @@ -44,8 +44,14 @@ jobs:
if [[ "$line" == *github.com/deckhouse/sds-* || "$line" == *github.com/deckhouse/csi-* || "$line" == *github.com/deckhouse/virtualization ]]; then
repository=$(echo "$line" | awk '{print $1}' | awk -F'/' '{ print "https://"$1"/"$2"/"$3".git" }')
pseudo_tag=$(echo "$line" | awk '{print $2}')

go_pkg=$(echo "$line" | awk '{print $1}')
if grep -q "^replace $go_pkg" $go_mod_file ;then
echo "Skipping $go_pkg check because it exists in replacement"
continue
fi

echo "Cloning repo $repository into $temp_dir"

git clone "$repository" "$temp_dir/$repository" >/dev/null 2>&1

if [ -d "$temp_dir/$repository/api" ]; then
Expand Down Expand Up @@ -79,4 +85,4 @@ jobs:
if [ $alert_lines_count != 0 ]; then
echo "We have non-actual pseudo-tags in repository's go.mod files"
exit 1
fi
fi
22 changes: 13 additions & 9 deletions .github/workflows/go_tests.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
name: Go tests for images

env:
GO_BUILD_TAGS: "ce se seplus ee csepro"

on:
pull_request:
push:
Expand All @@ -18,23 +21,24 @@ jobs:
- name: Setup Go environment
uses: actions/setup-go@v5
with:
go-version: '1.22'
go-version: '1.23.5'

- name: Run Go tests
run: |
basedir=$(pwd)
failed='false'
for dir in $(find images -type d); do
if ls $dir/*_test.go &> /dev/null; then
echo "Running tests in $dir"
cd $dir
go test -v
for i in $(find images -type f -name '*_test.go');do
dir=$(echo $i | sed 's/[a-z_A-Z0-9-]*_test.go$//')
cd $basedir/$dir
# check all editions
for edition in $GO_BUILD_TAGS ;do
echo "Running tests in $dir (edition: $edition)"
go test -v -tags $edition
if [ $? -ne 0 ]; then
echo "Tests failed in $dir"
echo "Tests failed in $dir (edition: $edition)"
failed='true'
fi
cd $basedir
fi
done
done
if [ $failed == 'true' ]; then
exit 1
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/trivy_check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,12 @@ jobs:

- name: Prepare sub repo
run: |
version=v`grep "UTIL_LINUX_VERSION :=" images/agent/werf.inc.yaml | awk -F'"' '{ print $2}'`
version=`grep '"UTIL_LINUX"' .werf/consts.yaml | awk -F'"' '{ print $4}'`
git clone --depth 1 --branch $version ${{ secrets.SOURCE_REPO }}/util-linux/util-linux.git ./util-linux
git clone ${{ secrets.SOURCE_REPO }}/lvmteam/lvm2.git ./lvm2
version=`grep "LVM2_VERSION :=" images/sds-utils-installer/werf.inc.yaml | awk -F'"' '{ print $2}'`
version=`grep '"LVM2"' .werf/consts.yaml | awk -F'"' '{ print $4}'`
cd ./lvm2
git checkout $version
git checkout $version
cd ..

- name: Run Trivy vulnerability scanner in fs mode
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,8 @@
.idea/
venv/
*.code-workspace
.vscode
*.code-workspace

# macOS Finder files
*.DS_Store
Expand Down
3 changes: 1 addition & 2 deletions .golangci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,7 @@ linters-settings:
- prefix(agent)
- prefix(sds-health-watcher-controller)
- prefix(sds-utils-installer)
errcheck:
ignore: fmt:.*,[rR]ead|[wW]rite|[cC]lose,io:Copy
- prefix(webhooks)

linters:
disable-all: true
Expand Down
11 changes: 10 additions & 1 deletion .werf/consts.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,20 @@
# base images
{{- $_ := set $ "BASE_ALT" "registry.deckhouse.io/base_images/alt:p10@sha256:f105773c682498700680d7cd61a702a4315c4235aee3622757591fd510fb8b4a" }}
{{- $_ := set $ "BASE_ALT_P11" "registry.deckhouse.io/base_images/alt:p11@sha256:e47d84424485d3674240cb2f67d3a1801b37d327e6d1eb8cc8d01be8ed3b34f3" }}
{{- $_ := set $ "BASE_GOLANG_1_23" "registry.deckhouse.io/base_images/golang:1.23.1-alpine3.20@sha256:716820a183116e643839611ff9eca9bd1c92d2bf8f7a5eda2f9fd16e8acbaa72" }}
{{- $_ := set $ "BASE_GOLANG_1_23" "registry.deckhouse.io/base_images/golang:1.23.5-alpine3.20@sha256:623ef3f63012bbd648021a2f097de3f411889332ba83bd98f0ac8d1288bdaa06" }}
{{- $_ := set $ "BASE_SCRATCH" "registry.deckhouse.io/base_images/scratch@sha256:653ae76965c98c8cd1c8c9ff7725316d2983986f896655b30e0f44d2f8b2dd7e" }}
{{- $_ := set $ "BASE_ALPINE" "registry.deckhouse.io/base_images/alpine:3.20.3@sha256:41628df7c9b935d248f64542634e7a843f9bc7f2252d7f878e77f7b79a947466" }}

# Edition module settings, default ce
{{- $_ := set . "MODULE_EDITION" (env "MODULE_EDITION" "ce") }}

# component versions
{{- $versions := dict }}
{{- $_ := set $versions "UTIL_LINUX" "v2.39.3" }}
{{- $_ := set $versions "LVM2" "d786a8f820d54ce87a919e6af5426c333c173b11" }}

{{- $_ := set $ "VERSIONS" $versions }}

# custom constants
{{- $_ := set $ "DECKHOUSE_UID_GID" "64535" }}
{{- $_ := set $ "ALT_CLEANUP_CMD" "rm -rf /var/lib/apt/lists/* /var/cache/apt/* && mkdir -p /var/lib/apt/lists/partial /var/cache/apt/archives/partial" }}
2 changes: 1 addition & 1 deletion api/go.mod
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
module github.com/deckhouse/sds-node-configurator/api

go 1.22.2
go 1.22.3

require k8s.io/apimachinery v0.31.3

Expand Down
29 changes: 29 additions & 0 deletions api/v1alpha1/const.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/*
Copyright 2025 Flant JSC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha1

const (
PhaseCreated = "Created"
PhasePending = "Pending"
PhaseResizing = "Resizing"
PhaseFailed = "Failed"
PhaseNotReady = "NotReady"
PhaseReady = "Ready"
PhaseTerminating = "Terminating"

LLVSNameTag = "storage.deckhouse.io/lvmLogicalVolumeSnapshotName"
)
Loading
Loading