Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retry logic when rate limit #1896

Merged
merged 25 commits into from
Mar 9, 2023
Merged

Conversation

HantingZhang2
Copy link
Contributor

What does this PR do?

This PR implements retrying in the api client as well as a new test function and test cassette

Review checklist

Please check relevant items below:

  • This PR includes all newly recorded cassettes for any modified tests.

  • This PR does not rely on API client schema changes.

    • The CI should be fully passing.
  • Or, this PR relies on API schema changes and this is a Draft PR to include tests for that new functionality.

    • Note: CI shouldn't be run on this Draft PR, as its expected to fail without the corresponding schema changes.

.generator/src/generator/templates/client.j2 Outdated Show resolved Hide resolved
.generator/src/generator/templates/client.j2 Outdated Show resolved Hide resolved
.generator/src/generator/templates/client.j2 Outdated Show resolved Hide resolved
.generator/src/generator/templates/client.j2 Outdated Show resolved Hide resolved
.generator/src/generator/templates/client.j2 Outdated Show resolved Hide resolved
.generator/src/generator/templates/client.j2 Outdated Show resolved Hide resolved
tests/api/datadogV2/api_logs_test.go Outdated Show resolved Hide resolved
@HantingZhang2 HantingZhang2 marked this pull request as ready for review March 6, 2023 15:09
@HantingZhang2 HantingZhang2 requested a review from a team as a code owner March 6, 2023 15:09
@nkzou nkzou added the changelog/Added Added features results into a minor version bump label Mar 6, 2023
Copy link
Contributor

@therve therve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, made a few comments. Can you add a section in the README as well? Thanks.

.generator/src/generator/templates/client.j2 Outdated Show resolved Hide resolved
log.Printf("\n%s\n", string(dump))
fmt.Println("Max retries:", maxRetries, " Current retry:", retryCount)
if retryCount == maxRetries {
fmt.Println("Max retries reached")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should remove this, and log in !shouldRetry under.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By turning debug on we should dump out the response no matter retry or not

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I'm just talking about the last log line,

retryCount := 0
for {
if retryCount == maxRetries {
ccancel()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that the right behavior? I would expect the same as the !shouldRetry below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If max retry is reached then next request is cancelled and we can exit without returning a response. Which is not a condition covered by shouldRetryRequest.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAIU this will impact the next select, not the request itself? I think it would make sense to handle this in shouldRetryRequest.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep I moved the logic to shouldRetryRequest

api/datadog/client.go Show resolved Hide resolved
@github-actions github-actions bot added the documentation Documentation related changes label Mar 8, 2023
tests/scenarios/actions.go Outdated Show resolved Hide resolved
@HantingZhang2 HantingZhang2 merged commit da0646c into master Mar 9, 2023
@HantingZhang2 HantingZhang2 deleted the add-retry-logic-when-rate-limit branch March 9, 2023 12:09
github-actions bot pushed a commit that referenced this pull request Mar 9, 2023
Adding enable retry in the API client and tests

---------

Co-authored-by: ci.datadog-api-spec <packages@datadoghq.com>
Co-authored-by: Thomas Hervé <thomas.herve@datadoghq.com> da0646c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog/Added Added features results into a minor version bump documentation Documentation related changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants