Add JUnit annotate BK plugin #5473

alexsapran · 2024-09-09T11:15:53Z

What does this PR do?

This PR adds the JUnit annotate Buildkite plugin

Why is it important?

This is goint to help with easier triaging what failed on each build.

I pushed a temp commit to cause some errors. See the screenshot

Now, when a test fails, you see the failure at the top with an additional link to the step that caused that error.

Example build with a failure to show this in action https://buildkite.com/elastic/elastic-agent/builds/12866

How to test this PR locally

This is a Buildkite change, only testable on BK.

Related issues

Related: https://github.com/elastic/ingest-dev/issues/3773

mergify · 2024-09-09T11:16:34Z

This pull request does not have a backport label. Could you fix it @alexsapran? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

backport-v./d./d./d is the label to automatically backport to the 8./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

alexsapran · 2024-09-09T12:20:03Z

.buildkite/pipeline.yml

+    agents:
+      provider: "gcp"
+    depends_on:
+      - step: "unit-tests-2204"


I noticed something that might cause a problem.
Some steps produce the same artifact for reporting the test files, for example, unit-tests-2204 produce the same xml files as unit-tests-2204-arm64

alexsapran · 2024-09-10T13:05:01Z

dev-tools/mage/gotest.go

-	fileName := fmt.Sprintf("build/TEST-go-%s", strings.Replace(strings.ToLower(name), " ", "_", -1))
+	fileName := ""
+	_, isCI := os.LookupEnv("CI")
+	if isCI {


When running inside CI, the BK plugin will try to link the failure to the step if the generated file has this pattern.

see: https://github.com/buildkite-plugins/junit-annotate-buildkite-plugin#job-uuid-file-pattern-optional

mergify · 2024-09-10T21:09:17Z

backport-v8.x has been added to help with the transition to the new branch 8.x.

mergify · 2024-09-11T11:50:18Z

backport-v8.x has been added to help with the transition to the new branch 8.x.

This is goint to help with easier triaging what failed on each build. Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Because some steps are producing the same artifact names, when uploaded and re-downloaed they overwrite each other. This is why we are attempting to split the annotate after each step to make it easier to deferentiate between the artifacts from each step Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

When running inside BK, we are running the same commands but in different enviroments, this PR changes this and differentiate each resulting artifact of JUnit to have a unique name for the step that was generated. This in combination with the JUnit annotate plugin provide better understanding on what failed on which step in the build. Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

…nit-annotate

Since we are renaming the cov file for each step to use the same name, lets move it instead and spare the traffic of one more file Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

alexsapran · 2024-10-07T10:18:25Z

.buildkite/scripts/steps/unit-tests.sh

 source .buildkite/scripts/common.sh
+set +euo pipefail


We set pipefail in the common.sh so the script was never run and performed the proper coverage files.

What script was never run?

Because of the order in which we set the bash flags, the' pipe failwas overridden inside thecommon.sh`, so the exit code's wrapping was never run.

So if the mage command failed, the script would exit, not producing any coverage.out but only a Test-go-unit.cov

coincidently, this failure is seen in this unrelated build https://buildkite.com/elastic/elastic-agent/builds/12911#_

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

This reverts commit 982d626.

elastic-sonarqube · 2024-10-08T07:05:27Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

* Add JUnit annotate BK plugin Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co> (cherry picked from commit 0d5b335)

* Add JUnit annotate BK plugin Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co> (cherry picked from commit 0d5b335) Co-authored-by: Alexandros Sapranidis <alexandros@elastic.co>

alexsapran added ci Team:Ingest-EngProd labels Sep 9, 2024

alexsapran self-assigned this Sep 9, 2024

mergify bot added the backport-skip label Sep 9, 2024

alexsapran added the skip-changelog label Sep 9, 2024

alexsapran commented Sep 9, 2024

View reviewed changes

alexsapran commented Sep 10, 2024

View reviewed changes

mergify bot added the backport-v8.x label Sep 10, 2024

mergify bot added the backport-8.x Automated backport to the 8.x branch with mergify label Sep 11, 2024

v1v removed the backport-v8.x label Sep 11, 2024

ycombinator force-pushed the bk/add-bk-plugin-junit-annotate branch from 967b239 to c77020b Compare September 20, 2024 00:42

alexsapran added 12 commits October 3, 2024 13:33

Add JUnit annotate BK plugin

fa8fa7f

This is goint to help with easier triaging what failed on each build. Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Add BK annotation parameters

300d9d2

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Revert back to a single call to the plugin

79cc5e5

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Fix fmt.sprintf wrong template

727942e

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Fix the coverage output filename

7bd2cdb

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Update the BK artifact path glob pattern

eee4eda

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Guess how windows does env substitution

8a8f73c

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Move JUnit annotate to run on VM

2bcd61f

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Another try on windows script

ed730a4

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Update the go code for detecting if run in CI

98bfbfd

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

alexsapran force-pushed the bk/add-bk-plugin-junit-annotate branch from c77020b to 98bfbfd Compare October 3, 2024 10:33

alexsapran added 3 commits October 4, 2024 19:36

Fix the ps1 script

9ea2cb3

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Merge remote-tracking branch 'upstream/main' into bk/add-bk-plugin-ju…

c3631a9

…nit-annotate

Move the coverage file.

c4b22fc

Since we are renaming the cov file for each step to use the same name, lets move it instead and spare the traffic of one more file Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

alexsapran added 5 commits October 7, 2024 12:14

Fix prepare artifacts commands

607c2c8

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Adjust bash flags

31f3820

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Adjust coverage files

c5812aa

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Update the sonar step

1294512

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Move and set properly the bash options

e922d16

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

alexsapran commented Oct 7, 2024

View reviewed changes

alexsapran added 9 commits October 7, 2024 13:21

Update coverage upload

b69f867

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Fix the bk artifact download

2516493

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Fix the file ext

af0eb07

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Try again with the cov.out ext

3317468

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Allow glob expansion

02c6546

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Fix the find

35e0ee5

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

gocoverage flag

55e0200

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Fancy onliner

9727f8c

Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co>

Revert "Revert, test failing"

71c20d8

This reverts commit 982d626.

alexsapran removed the backport-skip label Oct 8, 2024

alexsapran requested a review from pazone October 8, 2024 06:53

alexsapran marked this pull request as ready for review October 8, 2024 09:36

alexsapran requested a review from a team as a code owner October 8, 2024 09:36

alexsapran requested review from blakerouse and swiatekm October 8, 2024 09:36

pazone approved these changes Oct 8, 2024

View reviewed changes

jlind23 approved these changes Oct 8, 2024

View reviewed changes

swiatekm removed their request for review October 8, 2024 13:55

alexsapran merged commit 0d5b335 into elastic:main Oct 8, 2024
15 checks passed

alexsapran deleted the bk/add-bk-plugin-junit-annotate branch October 8, 2024 14:09

mergify bot pushed a commit that referenced this pull request Oct 8, 2024

Add JUnit annotate BK plugin (#5473)

7a45921

* Add JUnit annotate BK plugin Signed-off-by: Alexandros Sapranidis <alexandros@elastic.co> (cherry picked from commit 0d5b335)

mergify bot mentioned this pull request Oct 8, 2024

[8.x](backport #5473) Add JUnit annotate BK plugin #5743

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add JUnit annotate BK plugin #5473

Add JUnit annotate BK plugin #5473

alexsapran commented Sep 9, 2024 •

edited

Loading

mergify bot commented Sep 9, 2024

alexsapran Sep 9, 2024

alexsapran Sep 10, 2024

mergify bot commented Sep 10, 2024

mergify bot commented Sep 11, 2024

alexsapran Oct 7, 2024

pazone Oct 8, 2024

alexsapran Oct 8, 2024 •

edited

Loading

alexsapran Oct 8, 2024

elastic-sonarqube bot commented Oct 8, 2024

Add JUnit annotate BK plugin #5473

Add JUnit annotate BK plugin #5473

Conversation

alexsapran commented Sep 9, 2024 • edited Loading

What does this PR do?

Why is it important?

How to test this PR locally

Related issues

mergify bot commented Sep 9, 2024

alexsapran Sep 9, 2024

Choose a reason for hiding this comment

alexsapran Sep 10, 2024

Choose a reason for hiding this comment

mergify bot commented Sep 10, 2024

mergify bot commented Sep 11, 2024

alexsapran Oct 7, 2024

Choose a reason for hiding this comment

pazone Oct 8, 2024

Choose a reason for hiding this comment

alexsapran Oct 8, 2024 • edited Loading

Choose a reason for hiding this comment

alexsapran Oct 8, 2024

Choose a reason for hiding this comment

elastic-sonarqube bot commented Oct 8, 2024

Quality Gate passed

alexsapran commented Sep 9, 2024 •

edited

Loading

alexsapran Oct 8, 2024 •

edited

Loading