Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(ci): add retry to the run test step for circleci #11368

Merged
merged 4 commits into from
Nov 14, 2024
Merged

chore(ci): add retry to the run test step for circleci #11368

merged 4 commits into from
Nov 14, 2024

Conversation

erikayasuda
Copy link
Contributor

@erikayasuda erikayasuda commented Nov 12, 2024 •
edited
Loading

Some CircleCI tests are flaky, which causes merge queue times to be long if the PR gets ejected for flakes. This will hopefully help remedy this symptom. Flakes should still be recorded since the same test will fail on the same commit, it just won't be a hard blocker if a retry will make the test pass.

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

@erikayasuda erikayasuda requested review from a team as code owners November 12, 2024 16:25
@erikayasuda erikayasuda added the changelog/no-changelog A changelog entry is not required for this PR. label Nov 12, 2024
@github-actions GitHub Actions
Copy link
Contributor

CODEOWNERS have been resolved as:

.circleci/config.templ.yml                                              @DataDog/python-guild @DataDog/apm-core-python

@datadog-dd-trace-py-rkomorn
Copy link

datadog-dd-trace-py-rkomorn bot commented Nov 12, 2024 •
edited
Loading

Datadog Report

Branch report: erikayasuda/add-circleci-retry
Commit report: 4b84538
Test service: dd-trace-py

✅ 0 Failed, 592 Passed, 694 Skipped, 18m 13.11s Total duration (18m 34.41s time saved)

@pr-commenter
Copy link

pr-commenter bot commented Nov 12, 2024 •
edited
Loading

Benchmarks

Benchmark execution time: 2024-11-14 19:43:58

Comparing candidate commit 4b84538 in PR branch erikayasuda/add-circleci-retry with baseline commit c60e52d in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 388 metrics, 2 unstable metrics.

emmettbutler
emmettbutler reviewed Nov 12, 2024
View reviewed changes
Copy link
Collaborator

@emmettbutler emmettbutler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't expect that this will alleviate the issue, instead it will make it happen a bit less frequently.

@erikayasuda
Copy link
Contributor Author

/merge

@dd-devflow
Copy link

dd-devflow bot commented Nov 13, 2024 •
edited
Loading

Devflow running: /merge

View all feedbacks in Devflow UI.

2024-11-13 22:38:19 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2024-11-13 22:38:21 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2024-11-14 02:38:23 UTC ⚠️ MergeQueue: This merge request was unqueued

This merge request was unqueued

@taegyunkim
Copy link
Contributor

/merge

@dd-devflow
Copy link

dd-devflow bot commented Nov 14, 2024 •
edited
Loading

Devflow running: /merge

View all feedbacks in Devflow UI.

2024-11-14 02:42:18 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2024-11-14 02:42:21 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2024-11-14 03:58:09 UTC ℹ️ MergeQueue: pull request added to the queue

The median merge time in main is 35m.

2024-11-14 04:33:25 UTC ❌ MergeQueue: The checks failed on this merge request

Tests failed on this commit 0d78e60:

What to do next?

  • Investigate the failures and when ready, re-add your pull request to the queue!
  • Any question, go check the FAQ.

@erikayasuda
Copy link
Contributor Author

/merge

@dd-devflow
Copy link

dd-devflow bot commented Nov 14, 2024 •
edited
Loading

Devflow running: /merge

View all feedbacks in Devflow UI.

2024-11-14 19:04:05 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2024-11-14 19:04:09 UTC ℹ️ MergeQueue: waiting for PR to be ready

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2024-11-14 19:45:42 UTC ℹ️ MergeQueue: merge request added to the queue

The median merge time in main is 35m.

@dd-mergequeue dd-mergequeue bot merged commit 6916049 into main Nov 14, 2024
536 checks passed
@dd-mergequeue dd-mergequeue bot deleted the erikayasuda/add-circleci-retry branch November 14, 2024 21:16
github-actions bot pushed a commit that referenced this pull request Nov 22, 2024
@erikayasuda @github-actions
chore(ci): add retry to the run test step for circleci (#11368)
923fdac 
Co-authored-by: taegyunkim <taegyun.kim@datadoghq.com>
(cherry picked from commit 6916049)
github-actions bot pushed a commit that referenced this pull request Nov 22, 2024
@erikayasuda @github-actions
chore(ci): add retry to the run test step for circleci (#11368)
9ae44f6 
Co-authored-by: taegyunkim <taegyun.kim@datadoghq.com>
(cherry picked from commit 6916049)
This was referenced Nov 22, 2024
Merged
Merged
github-actions bot pushed a commit that referenced this pull request Nov 22, 2024
@erikayasuda @github-actions
chore(ci): add retry to the run test step for circleci (#11368)
da04374 
Co-authored-by: taegyunkim <taegyun.kim@datadoghq.com>
(cherry picked from commit 6916049)
@github-actions github-actions bot mentioned this pull request Nov 22, 2024
Merged
2 tasks
erikayasuda added a commit that referenced this pull request Nov 22, 2024
@github-actions @erikayasuda
chore(ci): add retry to the run test step for circleci [backport 2.17] (
475d911 
#11515)

Backport 6916049 from #11368 to 2.17.

Some CircleCI tests are flaky, which causes merge queue times to be long
if the PR gets ejected for flakes. This will hopefully help remedy this
symptom. Flakes should still be recorded since the same test will fail
on the same commit, it just won't be a hard blocker if a retry will make
the test pass.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Co-authored-by: erikayasuda <153395705+erikayasuda@users.noreply.github.com>
erikayasuda added a commit that referenced this pull request Nov 27, 2024
@github-actions @erikayasuda
chore(ci): add retry to the run test step for circleci [backport 2.16] (
fd538fb 
#11513)

Backport 6916049 from #11368 to 2.16.

Some CircleCI tests are flaky, which causes merge queue times to be long
if the PR gets ejected for flakes. This will hopefully help remedy this
symptom. Flakes should still be recorded since the same test will fail
on the same commit, it just won't be a hard blocker if a retry will make
the test pass.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Co-authored-by: erikayasuda <153395705+erikayasuda@users.noreply.github.com>
erikayasuda added a commit that referenced this pull request Nov 27, 2024
@github-actions @erikayasuda
chore(ci): add retry to the run test step for circleci [backport 2.15] (
ca36727 
#11514)

Backport 6916049 from #11368 to 2.15.

Some CircleCI tests are flaky, which causes merge queue times to be long
if the PR gets ejected for flakes. This will hopefully help remedy this
symptom. Flakes should still be recorded since the same test will fail
on the same commit, it just won't be a hard blocker if a retry will make
the test pass.

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

Co-authored-by: erikayasuda <153395705+erikayasuda@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@emmettbutler emmettbutler emmettbutler left review comments

@brettlangdon brettlangdon brettlangdon approved these changes

@juanjux juanjux Awaiting requested review from juanjux juanjux is a code owner automatically assigned from DataDog/python-guild

Assignees
No one assigned
Labels
backport 2.15 backport 2.16 backport 2.17 changelog/no-changelog A changelog entry is not required for this PR. mergequeue-status: done
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@erikayasuda @taegyunkim @emmettbutler @brettlangdon