[dev-utils/ship-ci-stats] fail when CI stats is down

[spalger]

CI stats experienced an outage caused by a load balancer misconfiguration in Elastic cloud https://status.elastic.co/incidents/cxggjn7drfbh

During this outage the CI Stats reporter was unable to ship metrics to the service which lead to the metrics not being reported for a build. This wasn't identified as an issue because initially the metrics weren't mission critical and were for informational purposes and we didn't want to trigger issues while getting started. CI Stats is now a part of our CI suite and used to validate things, so we rely on them and should fail builds if CI Stats is down.

This updates the script introduced in #90482 to fail the CLI when metrics are not properly shipped to the service.

[elasticmachine]

Pinging @elastic/kibana-operations (Team:Operations)

[kibanamachine]

Backport result

{"level":"info","message":"POST https://api.github.com/graphql (status: 200)"}
{"level":"info","message":"POST https://api.github.com/graphql (status: 200)"}
{"meta":{"labels":["Team:Operations","auto-backport","release_note:skip","v7.12.0","v8.0.0"],"branchLabelMapping":{"^v8.0.0$":"master","^v7.12.0$":"7.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"},"existingTargetPullRequests":[]},"level":"info","message":"Inputs when calculating target branches:"}
{"meta":["7.x"],"level":"info","message":"Target branches inferred from labels:"}
{"meta":{"killed":false,"code":2,"signal":null,"cmd":"git remote rm kibanamachine","stdout":"","stderr":"error: No such remote: 'kibanamachine'\n"},"level":"info","message":"exec error 'git remote rm kibanamachine':"}
{"meta":{"killed":false,"code":2,"signal":null,"cmd":"git remote rm elastic","stdout":"","stderr":"error: No such remote: 'elastic'\n"},"level":"info","message":"exec error 'git remote rm elastic':"}
{"level":"info","message":"Backporting [{\"sourceBranch\":\"master\",\"targetBranchesFromLabels\":[\"7.x\"],\"sha\":\"3ef3e9324fd9d77dd442b09e3355fb005c4b3caf\",\"formattedMessage\":\"[dev-utils/ship-ci-stats] fail when CI stats is down (#90678)\",\"originalMessage\":\"[dev-utils/ship-ci-stats] fail when CI stats is down (#90678)\\n\\nCo-authored-by: spalger <spalger@users.noreply.github.com>\",\"pullNumber\":90678,\"existingTargetPullRequests\":[]}] to 7.x"}

Backporting to 7.x:
{"level":"info","message":"Backporting via filesystem"}
{"meta":{"killed":false,"code":1,"signal":null,"cmd":"git cherry-pick 3ef3e9324fd9d77dd442b09e3355fb005c4b3caf","stdout":"CONFLICT (modify/delete): packages/kbn-dev-utils/src/ci_stats_reporter/ship_ci_stats_cli.ts deleted in HEAD and modified in 3ef3e9324fd ([dev-utils/ship-ci-stats] fail when CI stats is down (#90678)). Version 3ef3e9324fd ([dev-utils/ship-ci-stats] fail when CI stats is down (#90678)) of packages/kbn-dev-utils/src/ci_stats_reporter/ship_ci_stats_cli.ts left in tree.\n","stderr":"error: could not apply 3ef3e9324fd... [dev-utils/ship-ci-stats] fail when CI stats is down (#90678)\nhint: after resolving the conflicts, mark the corrected paths\nhint: with 'git add <paths>' or 'git rm <paths>'\nhint: and commit the result with 'git commit'\n"},"level":"info","message":"exec error 'git cherry-pick 3ef3e9324fd9d77dd442b09e3355fb005c4b3caf':"}
Commit could not be cherrypicked due to conflicts

[kibanamachine]

Backport result

{"level":"info","message":"POST https://api.github.com/graphql (status: 200)"}
{"level":"info","message":"POST https://api.github.com/graphql (status: 200)"}
{"meta":{"labels":["Team:Operations","auto-backport","release_note:skip","v7.12.0","v8.0.0"],"branchLabelMapping":{"^v8.0.0$":"master","^v7.12.0$":"7.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"},"existingTargetPullRequests":[]},"level":"info","message":"Inputs when calculating target branches:"}
{"meta":["7.x"],"level":"info","message":"Target branches inferred from labels:"}
{"meta":{"killed":false,"code":2,"signal":null,"cmd":"git remote rm kibanamachine","stdout":"","stderr":"error: No such remote: 'kibanamachine'\n"},"level":"info","message":"exec error 'git remote rm kibanamachine':"}
{"meta":{"killed":false,"code":2,"signal":null,"cmd":"git remote rm elastic","stdout":"","stderr":"error: No such remote: 'elastic'\n"},"level":"info","message":"exec error 'git remote rm elastic':"}
{"level":"info","message":"Backporting [{\"sourceBranch\":\"master\",\"targetBranchesFromLabels\":[\"7.x\"],\"sha\":\"3ef3e9324fd9d77dd442b09e3355fb005c4b3caf\",\"formattedMessage\":\"[dev-utils/ship-ci-stats] fail when CI stats is down (#90678)\",\"originalMessage\":\"[dev-utils/ship-ci-stats] fail when CI stats is down (#90678)\\n\\nCo-authored-by: spalger <spalger@users.noreply.github.com>\",\"pullNumber\":90678,\"existingTargetPullRequests\":[]}] to 7.x"}

Backporting to 7.x:
{"level":"info","message":"Backporting via filesystem"}
{"level":"info","message":"Creating PR with title: \"[7.x] [dev-utils/ship-ci-stats] fail when CI stats is down (#90678)\". kibanamachine:backport/7.x/pr-90678 -> 7.x"}
{"level":"info","message":"POST /repos/elastic/kibana/pulls - 201 in 1035ms"}
{"level":"info","message":"Adding assignees to #90706: spalger"}
{"level":"info","message":"POST /repos/elastic/kibana/issues/90706/assignees - 201 in 484ms"}
{"level":"info","message":"Adding labels: backport"}
{"level":"info","message":"POST /repos/elastic/kibana/issues/90706/labels - 200 in 366ms"}
View pull request: https://github.com/elastic/kibana/pull/90706

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 189da29
• Documentation Changes

Metrics [docs]

✅ unchanged

History

• 💛 Build #104890 was flaky 189da29

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream