Add upgrade test on an air-gapped environment #3724

AndersonQ · 2023-11-07T15:16:35Z

What does this PR do?

Add an upgrade test for air-gaped environments. It creates an artifacts server and blocks artifacts.elastic.co using iptables.

Why is it important?

To ensure the upgrade works correctly on air-gaped environments.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~[ ] I have made corresponding changes to the documentation~~
~~[ ] I have made corresponding change to the default configuration files~~
~~[ ] I have added tests that prove my fix is effective or that my feature works~~
~~[ ] I have added an entry in ./changelog/fragments using the [changelog tool]~~(https://github.com/elastic/elastic-agent#changelog)
I have added an integration test or an E2E test

How to test this PR locally

Run the test with:

AGENT_VERSION=8.12.0-SNAPSHOT AGENT_KEEP_INSTALLED=true INSTANCE_PROVISIONER="multipass" SNAPSHOT=true TEST_PLATFORMS="linux/amd64" mage integration:single TestFleetManagedAirGapedUpgrade

Related issues

Closes Add an integration test to emulate an air gapped agent upgrade #3403

Questions to ask yourself

How are we going to support this in production?
How are we going to measure its adoption?
How are we going to debug this?
What are the metrics I should take care of?
...

mergify · 2023-11-07T15:17:13Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 3403-airgapped-test upstream/3403-airgapped-test
git merge upstream/main
git push upstream 3403-airgapped-test

elasticmachine · 2023-11-08T22:17:55Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-11-15T15:49:06.228+0000
Duration: 67 min 41 sec

❕ Flaky test report

No test was executed to be analysed.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages.
run integration tests : Run the Elastic Agent Integration tests.
run end-to-end tests : Generate the packages and run the E2E Tests.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

AndersonQ · 2023-11-08T22:50:16Z

go.mod

@@ -182,3 +182,5 @@ replace (

 // Exclude this version because the version has an invalid checksum.
 exclude github.com/docker/distribution v2.8.0+incompatible
+
+replace github.com/elastic/elastic-agent-libs => github.com/AndersonQ/elastic-agent-libs v0.0.0-20231106171105-5bccf4058073


[Blocker]
Merge elastic/elastic-agent-libs#159 and release elastic-agent-libs then update the elastic-agent-libs version here

elasticmachine · 2023-11-08T22:50:40Z

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

AndersonQ · 2023-11-08T23:20:23Z

buildkite test this

AndersonQ · 2023-11-13T15:02:50Z

buildkite test this

blakerouse

Looks good. Really like seeing this type of test!

blakerouse · 2023-11-13T21:08:53Z

testing/integration/upgrade_fleet_test.go

+	require.NoErrorf(t, err, "could not get IPs for host %q", host)
+
+	// iptables -A OUTPUT -j DROP -d IP
+	t.Logf("found %v IPs for %q, blockingn them...", ips, host)


Suggested change

t.Logf("found %v IPs for %q, blockingn them...", ips, host)

t.Logf("found %v IPs for %q, blocking them...", ips, host)

mergify · 2023-11-14T10:08:06Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 3403-airgapped-test upstream/3403-airgapped-test
git merge upstream/main
git push upstream 3403-airgapped-test

AndersonQ · 2023-11-15T11:31:48Z

buildkite test this

michalpristas

LGTM

pkg/testing/multipass/provisioner.go

michalpristas · 2023-11-16T08:45:07Z

testing/integration/upgrade_fleet_test.go

+		Stack: &define.Stack{},
+		// The test uses iptables to simulate the air-gaped environment.
+		OS:      []define.OS{{Type: define.Linux}},
+		Isolate: true,  // It blocks IPs using iptables.


this test not the flag

mergify · 2023-11-16T10:49:59Z

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b 3403-airgapped-test upstream/3403-airgapped-test
git merge upstream/main
git push upstream 3403-airgapped-test

cmacknz · 2023-11-23T15:58:33Z

testing/integration/upgrade_fleet_test.go

+	testUpgradeFleetManagedElasticAgent(ctx, t, info, startFixture, endFixture, defaultPolicy())
+}
+
+func TestFleetAirGapedUpgrade(t *testing.T) {


Suggested change

func TestFleetAirGapedUpgrade(t *testing.T) {

func TestFleetAirGappedUpgrade(t *testing.T) {

cmacknz · 2023-11-23T16:12:20Z

testing/integration/upgrade_fleet_test.go

+
+	downloadSource := kibana.DownloadSource{
+		Name:      "local-air-gaped-" + uuid.NewString(),
+		Host:      s.URL + "/downloads/beats/elastic-agent/",


The route here should probably be a constant since it is also configured in the server.

cmacknz · 2023-11-23T16:16:57Z

I can see in the logs that the download is going to the file path and the remote GPG key download is failing as expected 👍

{"log.level":"info","@timestamp":"2023-11-23T14:05:26.049Z","log.origin":{"file.name":"upgrade/upgrade.go","file.line":131},"message":"Upgrading agent","log":{"source":"elastic-agent"},"version":"8.12.0-SNAPSHOT","source_uri":"","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:26.049Z","log.origin":{"file.name":"upgrade/cleanup.go","file.line":22},"message":"Cleaning up non-matching downloaded versions","log":{"source":"elastic-agent"},"version":"8.12.0","downloads.path":"/opt/Elastic/Agent/data/elastic-agent-11887c/downloads","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:26.049Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":500},"message":"updated upgrade details","log":{"source":"elastic-agent"},"upgrade_details":{"target_version":"8.12.0-SNAPSHOT","state":"UPG_DOWNLOADING","action_id":"4c8fb430-f991-4da8-ab22-feb74d65e786","metadata":{}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:26.049Z","log.origin":{"file.name":"upgrade/step_download.go","file.line":96},"message":"Downloading upgrade artifact","log":{"source":"elastic-agent"},"version":"8.12.0-SNAPSHOT","source_uri":"http://127.0.0.1:41181/downloads/beats/elastic-agent/","drop_path":"","target_path":"/opt/Elastic/Agent/data/elastic-agent-11887c/downloads","install_path":"/opt/Elastic/Agent/data/elastic-agent-11887c/install","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:26.049Z","log.origin":{"file.name":"upgrade/step_download.go","file.line":250},"message":"download attempt 1","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:26.060Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":500},"message":"updated upgrade details","log":{"source":"elastic-agent"},"upgrade_details":{"target_version":"8.12.0-SNAPSHOT","state":"UPG_DOWNLOADING","action_id":"4c8fb430-f991-4da8-ab22-feb74d65e786","metadata":{}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:27.444Z","log.origin":{"file.name":"http/progress_observer.go","file.line":71},"message":"download from http://127.0.0.1:41181/downloads/beats/elastic-agent/beats/elastic-agent/elastic-agent-8.12.0-SNAPSHOT-linux-x86_64.tar.gz completed in 1 second @ 698.9MBps","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:27.453Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":500},"message":"updated upgrade details","log":{"source":"elastic-agent"},"upgrade_details":{"target_version":"8.12.0-SNAPSHOT","state":"UPG_DOWNLOADING","action_id":"4c8fb430-f991-4da8-ab22-feb74d65e786","metadata":{"download_percent":1,"download_rate":"698.896443MBps"}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:27.455Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":500},"message":"updated upgrade details","log":{"source":"elastic-agent"},"upgrade_details":{"target_version":"8.12.0-SNAPSHOT","state":"UPG_DOWNLOADING","action_id":"4c8fb430-f991-4da8-ab22-feb74d65e786","metadata":{"download_percent":1,"download_rate":"698.896443MBps"}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:27.455Z","log.origin":{"file.name":"http/progress_observer.go","file.line":71},"message":"download from http://127.0.0.1:41181/downloads/beats/elastic-agent/beats/elastic-agent/elastic-agent-8.12.0-SNAPSHOT-linux-x86_64.tar.gz.sha512 completed in Less than a second @ +InfYBps","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:27.455Z","log.origin":{"file.name":"coordinator/coordinator.go","file.line":500},"message":"updated upgrade details","log":{"source":"elastic-agent"},"upgrade_details":{"target_version":"8.12.0-SNAPSHOT","state":"UPG_DOWNLOADING","action_id":"4c8fb430-f991-4da8-ab22-feb74d65e786","metadata":{"download_percent":1,"download_rate":"+Inf bps"}},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:28.859Z","log.origin":{"file.name":"download/verifier.go","file.line":231},"message":"Default PGP appended","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:38.752Z","message":"Non-zero metrics in the last 30s","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-monitoring","type":"filestream"},"log":{"source":"filestream-monitoring"},"service.name":"filebeat","monitoring":{"ecs.version":"1.6.0","metrics":{"beat":{"cgroup":{"cpu":{"id":"elastic-agent.service"},"memory":{"id":"elastic-agent.service","mem":{"usage":{"bytes":886800384}}}},"cpu":{"system":{"ticks":60,"time":{"ms":60}},"total":{"ticks":430,"time":{"ms":430},"value":430},"user":{"ticks":370,"time":{"ms":370}}},"handles":{"limit":{"hard":524288,"soft":524288},"open":19},"info":{"ephemeral_id":"9b7405df-87cf-4425-a74d-6d9fa07ca328","name":"filebeat","uptime":{"ms":30243},"version":"8.12.0"},"memstats":{"gc_next":42197592,"memory_alloc":23796280,"memory_sys":57799944,"memory_total":75058464,"rss":135041024},"runtime":{"goroutines":56}},"filebeat":{"events":{"active":0,"added":259,"done":259},"harvester":{"open_files":4,"running":4,"started":4}},"libbeat":{"config":{"module":{"running":1,"starts":1}},"output":{"events":{"acked":91,"active":0,"batches":2,"total":91},"read":{"bytes":2762,"errors":1},"type":"elasticsearch","write":{"bytes":21845}},"pipeline":{"clients":4,"events":{"active":0,"filtered":168,"published":91,"retry":75,"total":259},"queue":{"acked":91,"max_events":3200}}},"registrar":{"states":{"current":0}},"system":{"cpu":{"cores":2},"load":{"1":1.08,"15":1.01,"5":1.78,"norm":{"1":0.54,"15":0.505,"5":0.89}}}}},"log.logger":"monitoring","log.origin":{"file.line":187,"file.name":"log/log.go"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:39.279Z","message":"Non-zero metrics in the last 30s","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"beat/metrics-monitoring","type":"beat/metrics"},"log":{"source":"beat/metrics-monitoring"},"log.origin":{"file.line":187,"file.name":"log/log.go"},"service.name":"metricbeat","monitoring":{"ecs.version":"1.6.0","metrics":{"beat":{"cgroup":{"cpu":{"id":"elastic-agent.service"},"memory":{"id":"elastic-agent.service","mem":{"usage":{"bytes":886624256}}}},"cpu":{"system":{"ticks":60,"time":{"ms":60}},"total":{"ticks":330,"time":{"ms":330},"value":330},"user":{"ticks":270,"time":{"ms":270}}},"handles":{"limit":{"hard":524288,"soft":524288},"open":12},"info":{"ephemeral_id":"bb3e47f5-32dd-417a-9f05-580f021440bb","name":"metricbeat","uptime":{"ms":30280},"version":"8.12.0"},"memstats":{"gc_next":54164408,"memory_alloc":35782312,"memory_sys":53343496,"memory_total":51128432,"rss":138149888},"runtime":{"goroutines":29}},"libbeat":{"config":{"module":{"running":0}},"output":{"events":{"active":0},"type":"elasticsearch"},"pipeline":{"clients":0,"events":{"active":0},"queue":{"max_events":3200}}},"system":{"cpu":{"cores":2},"load":{"1":1.08,"15":1.01,"5":1.78,"norm":{"1":0.54,"15":0.505,"5":0.89}}}}},"log.logger":"monitoring","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:39.593Z","message":"Non-zero metrics in the last 30s","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"http/metrics-monitoring","type":"http/metrics"},"log":{"source":"http/metrics-monitoring"},"service.name":"metricbeat","monitoring":{"ecs.version":"1.6.0","metrics":{"beat":{"cgroup":{"cpu":{"id":"elastic-agent.service"},"memory":{"id":"elastic-agent.service","mem":{"usage":{"bytes":886886400}}}},"cpu":{"system":{"ticks":60,"time":{"ms":60}},"total":{"ticks":320,"time":{"ms":320},"value":320},"user":{"ticks":260,"time":{"ms":260}}},"handles":{"limit":{"hard":524288,"soft":524288},"open":13},"info":{"ephemeral_id":"7e5a6c0d-fc8f-4766-980e-b2e1f23abb41","name":"metricbeat","uptime":{"ms":30180},"version":"8.12.0"},"memstats":{"gc_next":51236408,"memory_alloc":40810832,"memory_sys":57799944,"memory_total":57194336,"rss":142475264},"runtime":{"goroutines":35}},"libbeat":{"config":{"module":{"running":1,"starts":1}},"output":{"events":{"acked":1,"active":0,"batches":1,"total":1},"read":{"bytes":2938,"errors":1},"type":"elasticsearch","write":{"bytes":3123}},"pipeline":{"clients":1,"events":{"active":0,"published":1,"retry":1,"total":1},"queue":{"acked":1,"max_events":3200}}},"metricbeat":{"http":{"json":{"events":1,"success":1}}},"system":{"cpu":{"cores":2},"load":{"1":1.08,"15":1.01,"5":1.78,"norm":{"1":0.54,"15":0.505,"5":0.89}}}}},"log.logger":"monitoring","log.origin":{"file.line":187,"file.name":"log/log.go"},"ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2023-11-23T14:05:58.860Z","log.origin":{"file.name":"download/verifier.go","file.line":268},"message":"Skipped remote PGP located at \"https://artifacts.elastic.co/GPG-KEY-elastic-agent\" because it's unavailable: 2 errors occurred:\n\t* Get \"https://artifacts.elastic.co/GPG-KEY-elastic-agent\": context deadline exceeded\n\t* Remote PGP download failed\n\n","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2023-11-23T14:05:59.026Z","log.origin":{"file.name":"download/verifier.go","file.line":270},"message":"Failed to fetch remote PGP key from \"https://70314541ec04430aaedc661b1ffa7313.fleet.us-west2.gcp.elastic-cloud.com:443/api/agents/upgrades/8.12.0/pgp-public-key\": call to 'https://70314541ec04430aaedc661b1ffa7313.fleet.us-west2.gcp.elastic-cloud.com:443/api/agents/upgrades/8.12.0/pgp-public-key' returned unsuccessful status code: 400","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:05:59.027Z","log.origin":{"file.name":"download/verifier.go","file.line":256},"message":"Using 1 PGP keys","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}
{"log.level":"warn","@timestamp":"2023-11-23T14:05:59.027Z","log.origin":{"file.name":"composed/verifier.go","file.line":53},"message":"Verifier failed!","log":{"source":"elastic-agent"},"verifier":"fs.verifier","error":{"message":"could not get .asc file: fetching asc file from '/opt/Elastic/Agent/data/elastic-agent-11887c/downloads/elastic-agent-8.12.0-SNAPSHOT-linux-x86_64.tar.gz.asc': open /opt/Elastic/Agent/data/elastic-agent-11887c/downloads/elastic-agent-8.12.0-SNAPSHOT-linux-x86_64.tar.gz.asc: no such file or directory"},"ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-11-23T14:06:00.254Z","log.origin":{"file.name":"download/verifier.go","file.line":231},"message":"Default PGP appended","log":{"source":"elastic-agent"},"ecs.version":"1.6.0"}

cmacknz · 2023-11-23T16:18:04Z

Now we just need the test to pass reliably.

As an aside there are a lot of formatting changes in here that made it harder to review the changes that add the test. Formatting changes are a good improvement, but its easier to review a lot of them if they are separated out. Lots of formatting changes also make it harder to spot bugs in the "real" part of the change.

AndersonQ · 2023-11-23T19:41:07Z

Now we just need the test to pass reliably.

I'm wondering if after the last rebase something broke because they were passing :/
I'm investigating it now

…irgapped-test

This reverts commit 4124a5b.

AndersonQ · 2023-11-25T01:30:36Z

buildkite test this

AndersonQ · 2023-11-25T03:03:15Z

buildkite test this

elastic-sonarqube · 2023-11-25T03:31:12Z

SonarQube Quality Gate

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

AndersonQ self-assigned this Nov 7, 2023

AndersonQ added Team:Elastic-Agent Label for the Agent team backport-skip skip-changelog labels Nov 7, 2023

AndersonQ force-pushed the 3403-airgapped-test branch from 99a0883 to f71df30 Compare November 7, 2023 15:20

AndersonQ mentioned this pull request Nov 8, 2023

Add CreateDownloadSource to kibana API elastic/elastic-agent-libs#159

Merged

1 task

AndersonQ commented Nov 8, 2023

View reviewed changes

AndersonQ marked this pull request as ready for review November 8, 2023 22:50

AndersonQ requested a review from a team as a code owner November 8, 2023 22:50

AndersonQ requested review from michalpristas and blakerouse November 8, 2023 22:50

AndersonQ marked this pull request as draft November 8, 2023 22:50

AndersonQ marked this pull request as ready for review November 8, 2023 23:20

AndersonQ changed the title ~~WIP: Add upgrade test on an air-gapped environment~~ Add upgrade test on an air-gapped environment Nov 9, 2023

michalpristas approved these changes Nov 9, 2023

View reviewed changes

AndersonQ force-pushed the 3403-airgapped-test branch from 7d5d309 to 6645eb7 Compare November 13, 2023 19:48

blakerouse approved these changes Nov 13, 2023

View reviewed changes

AndersonQ force-pushed the 3403-airgapped-test branch 2 times, most recently from 46e9c7f to 438480a Compare November 14, 2023 19:36

AndersonQ mentioned this pull request Nov 15, 2023

[Flaky Test] TestFleetManagedUpgrade fails waiting the watcher to start #3760

Closed

michalpristas approved these changes Nov 16, 2023

View reviewed changes

AndersonQ force-pushed the 3403-airgapped-test branch from 7402d58 to abf060b Compare November 21, 2023 12:09

AndersonQ added 12 commits November 23, 2023 10:34

fix tests

57f050b

add provider to Instance and use it for equality comparison

876c8c1

remove replace directive

ea79278

shorten test name to fit ogc resource.name

4988cd1

increase test timeout

57df016

add test error message

d6a03cb

liner: rename receivers

7ea791a

fix import

5be9f46

linter and spell fix

405b9ca

fix test

120423f

upgrade elastic-agent-libs to v0.7.1

5db5e1f

linter fixes and PR changes

11887c4

AndersonQ force-pushed the 3403-airgapped-test branch from 41af21d to 11887c4 Compare November 23, 2023 13:45

cmacknz reviewed Nov 23, 2023

View reviewed changes

AndersonQ added 2 commits November 23, 2023 16:14

Merge branch 'main' into 3403-airgapped-test

4ddab16

debug

4124a5b

AndersonQ added 5 commits November 23, 2023 16:41

Merge remote-tracking branch 'origin/3403-airgapped-test' into 3403-a…

1a9d562

…irgapped-test

.

9c1c9c1

Revert "debug"

46d41a9

This reverts commit 4124a5b.

fix test and log error on failure

7b74add

Merge branch 'main' into 3403-airgapped-test

56cc05e

AndersonQ merged commit 092d205 into elastic:main Nov 27, 2023
7 checks passed

AndersonQ deleted the 3403-airgapped-test branch November 27, 2023 12:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add upgrade test on an air-gapped environment #3724

Add upgrade test on an air-gapped environment #3724

AndersonQ commented Nov 7, 2023 •

edited

Loading

mergify bot commented Nov 7, 2023

elasticmachine commented Nov 8, 2023 •

edited

Loading

Build stats

AndersonQ Nov 8, 2023

elasticmachine commented Nov 8, 2023

AndersonQ commented Nov 8, 2023

AndersonQ commented Nov 13, 2023

blakerouse left a comment

blakerouse Nov 13, 2023

mergify bot commented Nov 14, 2023

AndersonQ commented Nov 15, 2023

michalpristas left a comment

michalpristas Nov 16, 2023

mergify bot commented Nov 16, 2023

cmacknz Nov 23, 2023

cmacknz Nov 23, 2023

cmacknz commented Nov 23, 2023

cmacknz commented Nov 23, 2023

AndersonQ commented Nov 23, 2023

AndersonQ commented Nov 25, 2023

AndersonQ commented Nov 25, 2023

elastic-sonarqube bot commented Nov 25, 2023

	t.Logf("found %v IPs for %q, blockingn them...", ips, host)
	t.Logf("found %v IPs for %q, blocking them...", ips, host)

	func TestFleetAirGapedUpgrade(t *testing.T) {
	func TestFleetAirGappedUpgrade(t *testing.T) {

Add upgrade test on an air-gapped environment #3724

Add upgrade test on an air-gapped environment #3724

Conversation

AndersonQ commented Nov 7, 2023 • edited Loading

What does this PR do?

Why is it important?

Checklist

How to test this PR locally

Related issues

Questions to ask yourself

mergify bot commented Nov 7, 2023

elasticmachine commented Nov 8, 2023 • edited Loading

💚 Build Succeeded

Build stats

❕ Flaky test report

🤖 GitHub comments

AndersonQ Nov 8, 2023

Choose a reason for hiding this comment

elasticmachine commented Nov 8, 2023

AndersonQ commented Nov 8, 2023

AndersonQ commented Nov 13, 2023

blakerouse left a comment

Choose a reason for hiding this comment

blakerouse Nov 13, 2023

Choose a reason for hiding this comment

mergify bot commented Nov 14, 2023

AndersonQ commented Nov 15, 2023

michalpristas left a comment

Choose a reason for hiding this comment

michalpristas Nov 16, 2023

Choose a reason for hiding this comment

mergify bot commented Nov 16, 2023

cmacknz Nov 23, 2023

Choose a reason for hiding this comment

cmacknz Nov 23, 2023

Choose a reason for hiding this comment

cmacknz commented Nov 23, 2023

cmacknz commented Nov 23, 2023

AndersonQ commented Nov 23, 2023

AndersonQ commented Nov 25, 2023

AndersonQ commented Nov 25, 2023

elastic-sonarqube bot commented Nov 25, 2023

AndersonQ commented Nov 7, 2023 •

edited

Loading

elasticmachine commented Nov 8, 2023 •

edited

Loading