Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JUnit annotate BK plugin #5473

Merged
merged 31 commits into from
Oct 8, 2024
Merged

Add JUnit annotate BK plugin #5473

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
fa8fa7f
Add JUnit annotate BK plugin
alexsapran Sep 9, 2024
7ad37e0
Split JUnit annotate into multiple steps
alexsapran Sep 10, 2024
300d9d2
Add BK annotation parameters
alexsapran Sep 10, 2024
2872ead
Rename the JUnit files when running in BK
alexsapran Sep 10, 2024
79cc5e5
Revert back to a single call to the plugin
alexsapran Sep 10, 2024
727942e
Fix fmt.sprintf wrong template
alexsapran Sep 10, 2024
7bd2cdb
Fix the coverage output filename
alexsapran Sep 10, 2024
eee4eda
Update the BK artifact path glob pattern
alexsapran Sep 10, 2024
8a8f73c
Guess how windows does env substitution
alexsapran Sep 10, 2024
2bcd61f
Move JUnit annotate to run on VM
alexsapran Sep 10, 2024
ed730a4
Another try on windows script
alexsapran Sep 11, 2024
98bfbfd
Update the go code for detecting if run in CI
alexsapran Oct 3, 2024
9ea2cb3
Fix the ps1 script
alexsapran Oct 4, 2024
c3631a9
Merge remote-tracking branch 'upstream/main' into bk/add-bk-plugin-ju…
alexsapran Oct 4, 2024
c4b22fc
Move the coverage file.
alexsapran Oct 4, 2024
982d626
Revert, test failing
alexsapran Oct 7, 2024
8825767
Move around the files again
alexsapran Oct 7, 2024
607c2c8
Fix prepare artifacts commands
alexsapran Oct 7, 2024
31f3820
Adjust bash flags
alexsapran Oct 7, 2024
c5812aa
Adjust coverage files
alexsapran Oct 7, 2024
1294512
Update the sonar step
alexsapran Oct 7, 2024
e922d16
Move and set properly the bash options
alexsapran Oct 7, 2024
b69f867
Update coverage upload
alexsapran Oct 7, 2024
2516493
Fix the bk artifact download
alexsapran Oct 7, 2024
af0eb07
Fix the file ext
alexsapran Oct 7, 2024
3317468
Try again with the cov.out ext
alexsapran Oct 7, 2024
02c6546
Allow glob expansion
alexsapran Oct 7, 2024
35e0ee5
Fix the find
alexsapran Oct 7, 2024
55e0200
gocoverage flag
alexsapran Oct 7, 2024
9727f8c
Fancy onliner
alexsapran Oct 7, 2024
71c20d8
Revert "Revert, test failing"
alexsapran Oct 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 54 additions & 35 deletions .buildkite/pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ steps:
key: "unit-tests-2204"
command: ".buildkite/scripts/steps/unit-tests.sh"
artifact_paths:
- "build/TEST-go-unit.html"
- "build/TEST-go-unit.xml"
- "build/TEST-*.html"
- "build/TEST-*.xml"
- "build/diagnostics/*"
- "coverage.out"
- "coverage-*.out"
agents:
provider: "gcp"
image: "family/core-ubuntu-2204"
Expand All @@ -35,10 +35,10 @@ steps:
key: "unit-tests-2204-arm64"
command: ".buildkite/scripts/steps/unit-tests.sh"
artifact_paths:
- "build/TEST-go-unit.html"
- "build/TEST-go-unit.xml"
- "build/TEST-*.html"
- "build/TEST-*.xml"
- "build/diagnostics/*"
- "coverage.out"
- "coverage-*.out"
agents:
provider: "aws"
imagePrefix: "core-ubuntu-2204-aarch64"
Expand All @@ -52,10 +52,10 @@ steps:
key: "unit-tests-win2022"
command: ".\\.buildkite\\scripts\\steps\\unit-tests.ps1"
artifact_paths:
- "build/TEST-go-unit.html"
- "build/TEST-go-unit.xml"
- "build/TEST-*.html"
- "build/TEST-*.xml"
- "build/diagnostics/*"
- "coverage.out"
- "coverage-*.out"
agents:
provider: "gcp"
image: "family/core-windows-2022"
Expand All @@ -70,10 +70,10 @@ steps:
key: "unit-tests-win2016"
command: ".\\.buildkite\\scripts\\steps\\unit-tests.ps1"
artifact_paths:
- "build/TEST-go-unit.html"
- "build/TEST-go-unit.xml"
- "build/TEST-*.html"
- "build/TEST-*.xml"
- "build/diagnostics/*"
- "coverage.out"
- "coverage-*.out"
agents:
provider: "gcp"
image: "family/core-windows-2016"
Expand All @@ -90,10 +90,10 @@ steps:
command: ".buildkite/scripts/steps/unit-tests.sh"
branches: main
artifact_paths:
- "build/TEST-go-unit.html"
- "build/TEST-go-unit.xml"
- "build/TEST-*.html"
- "build/TEST-*.xml"
- "build/diagnostics/*"
- "coverage.out"
- "coverage-*.out"
agents:
provider: orka
imagePrefix: generic-13-ventura-arm
Expand All @@ -105,10 +105,10 @@ steps:
key: "unit-tests-macos-13"
command: ".buildkite/scripts/steps/unit-tests.sh"
artifact_paths:
- "build/TEST-go-unit.html"
- "build/TEST-go-unit.xml"
- "build/TEST-*.html"
- "build/TEST-*.xml"
- "build/diagnostics/*"
- "coverage.out"
- "coverage-*.out"
agents:
provider: orka
imagePrefix: generic-13-ventura-x64
Expand All @@ -123,10 +123,10 @@ steps:
key: "unit-tests-win10"
command: ".\\.buildkite\\scripts\\steps\\unit-tests.ps1"
artifact_paths:
- "build/TEST-go-unit.html"
- "build/TEST-go-unit.xml"
- "build/TEST-*.html"
- "build/TEST-*.xml"
- "build/diagnostics/*"
- "coverage.out"
- "coverage-*.out"
agents:
provider: "gcp"
# TODO create own image
Expand All @@ -140,10 +140,10 @@ steps:
key: "unit-tests-win11"
command: ".\\.buildkite\\scripts\\steps\\unit-tests.ps1"
artifact_paths:
- "build/TEST-go-unit.html"
- "build/TEST-go-unit.xml"
- "build/TEST-*.html"
- "build/TEST-*.xml"
- "build/diagnostics/*"
- "coverage.out"
- "coverage-*.out"
agents:
provider: "gcp"
# TODO create own image
Expand All @@ -154,22 +154,41 @@ steps:
manual:
allowed: true


- label: ":junit: Junit annotate"
plugins:
- junit-annotate#v2.5.0:
artifacts: "build/*.xml"
always-annotate: true
agents:
provider: "gcp"
depends_on:
- step: "unit-tests-2204"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed something that might cause a problem.
Some steps produce the same artifact for reporting the test files, for example, unit-tests-2204 produce the same xml files as unit-tests-2204-arm64

allow_failure: true
- step: "unit-tests-2204-arm64"
allow_failure: true
- step: "unit-tests-win2022"
allow_failure: true
- step: "unit-tests-win2016"
allow_failure: true
- step: "unit-tests-macos-13-arm"
allow_failure: true
- step: "unit-tests-macos-13"
allow_failure: true
- step: "unit-tests-win10"
allow_failure: true
- step: "unit-tests-win11"
allow_failure: true


- label: "Merge coverage reports"
key: "merge-coverage"
env:
BUILDKITE_REPO: ""
command: "
command: |
.buildkite/scripts/steps/merge.sh
unit-tests-2204
unit-tests-2204-arm64
unit-tests-win2016
unit-tests-win2022
unit-tests-macos-13
unit-tests-win10
unit-tests-win11
"
artifact_paths:
- "build/TEST-go-unit.cov"
- "TEST-go-unit.cov"
agents:
image: "golang:1.22.6"
depends_on:
Expand Down Expand Up @@ -206,7 +225,7 @@ steps:
agents:
image: "docker.elastic.co/cloud-ci/sonarqube/buildkite-scanner:latest"
command:
- "buildkite-agent artifact download --step merge-coverage build/TEST-go-unit.cov ."
- "buildkite-agent artifact download --step merge-coverage TEST-go-unit.cov ."
- "/scan-source-code.sh"
depends_on:
- "merge-coverage"
Expand Down
21 changes: 6 additions & 15 deletions .buildkite/scripts/steps/merge.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,24 +1,15 @@
#!/bin/bash

# Downloads and merges coverage files from multiple steps into a single file (build/TEST-go-unit.cov).
# Usage: merge.sh <step1> <step2> ... Where <step> is the id of the step that contains the coverage artifact.#

# Usage: merge.sh <step1> <step2> ... Where <step> is the id of the step that contains the coverage artifact.#
set -euo pipefail
set -x # for debugging

COV_ARTIFACT="coverage.out"
MERGED_COV_FILE="build/TEST-go-unit.cov"
# Space separated list of paths to coverage files
COV_PATHS=""
MERGED_COV_FILE="TEST-go-unit.cov"

go install github.com/wadey/gocovmerge@latest
mkdir -p build

for STEP_ID in "$@"; do
mkdir -p $STEP_ID
buildkite-agent artifact download --step $STEP_ID $COV_ARTIFACT $STEP_ID
COV_PATHS="${COV_PATHS} $STEP_ID/$COV_ARTIFACT"
done

gocovmerge $COV_PATHS > $MERGED_COV_FILE
echo "Merged coverage file: $MERGED_COV_FILE. See artifacts"
buildkite-agent artifact download "coverage-*.out" .
# Space separated list of paths to coverage files
find coverage-*.out -exec printf '%s ' {} \; | xargs gocovmerge > "$MERGED_COV_FILE"
echo "Merged coverage file: $MERGED_COV_FILE. See artifacts"
10 changes: 6 additions & 4 deletions .buildkite/scripts/steps/unit-tests.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,20 @@ Write-Host "--- Build"
mage build

if ($LASTEXITCODE -ne 0) {
exit 1
exit 1
}

Write-Host "--- Unit tests"
$env:TEST_COVERAGE = $true
$env:RACE_DETECTOR = $true
mage unitTest
# Copy coverage file to build directory so it can be downloaded as an artifact
cp .\build\TEST-go-unit.cov coverage.out

Write-Host "--- Prepare artifacts"
$buildkiteJobId = $env:BUILDKITE_JOB_ID
Move-Item -Path "build/TEST-go-unit.cov" -Destination "coverage-$buildkiteJobId.out"
Move-Item -Path "build/TEST-go-unit.xml" -Destination "build/TEST-$buildkiteJobId.xml"
if ($LASTEXITCODE -ne 0) {
exit 1
exit 1
}


9 changes: 5 additions & 4 deletions .buildkite/scripts/steps/unit-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
#!/usr/bin/env bash
set -uo pipefail

source .buildkite/scripts/common.sh
set +euo pipefail
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We set pipefail in the common.sh so the script was never run and performed the proper coverage files.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What script was never run?

Copy link
Contributor Author

@alexsapran alexsapran Oct 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of the order in which we set the bash flags, the' pipe failwas overridden inside thecommon.sh`, so the exit code's wrapping was never run.

So if the mage command failed, the script would exit, not producing any coverage.out but only a Test-go-unit.cov

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coincidently, this failure is seen in this unrelated build https://buildkite.com/elastic/elastic-agent/builds/12911#_


echo "--- Unit tests"
RACE_DETECTOR=true TEST_COVERAGE=true mage unitTest
TESTS_EXIT_STATUS=$?
echo "--- Prepare artifacts"
# Copy coverage file to build directory so it can be downloaded as an artifact
cp build/TEST-go-unit.cov coverage.out
exit $TESTS_EXIT_STATUS
mv build/TEST-go-unit.cov "coverage-${BUILDKITE_JOB_ID:go-unit}.out"
mv build/TEST-go-unit.xml build/"TEST-${BUILDKITE_JOB_ID:go-unit}.xml"
exit $TESTS_EXIT_STATUS
6 changes: 4 additions & 2 deletions dev-tools/mage/gotest.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,10 @@ func makeGoTestArgs(name string) GoTestArgs {
}

func makeGoTestArgsForModule(name, module string) GoTestArgs {
fileName := fmt.Sprintf("build/TEST-go-%s-%s", strings.Replace(strings.ToLower(name), " ", "_", -1),
strings.Replace(strings.ToLower(module), " ", "_", -1))
fileName := fmt.Sprintf("build/TEST-go-%s-%s",
strings.Replace(strings.ToLower(name), " ", "_", -1),
strings.Replace(strings.ToLower(module), " ", "_", -1),
)
params := GoTestArgs{
LogName: fmt.Sprintf("%s-%s", name, module),
Race: RaceDetector,
Expand Down