Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address CPU usage metric corner cases #2366

Merged
merged 3 commits into from
Feb 19, 2020
Merged

Conversation

sparrc
Copy link
Contributor

@sparrc sparrc commented Feb 17, 2020

(this is a recreation of #2361)

Summary

This PR changes a few corner cases in the CPU percent metrics handling:

  1. Cap cpu usage percent at 1024*1024 to match telemetry codebase allow large cpu usage percents to get sent to telemetry backend but log a detailed error message.
  2. If two metrics arrive with identical timestamps, set cpu usage to previous value rather than leaving it at NaN
  3. Ensure that we don't emit cloudwatch metrics with sampleCount == 0

Testing

New unit tests were added to cover these changes

New tests cover the changes: yes

Description for the changelog

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

1. Cap cpu usage percent at 1024*1024 to match telemetry codebase
2. If two metrics arrive with identical timestamps, set cpu usage to
previous value rather than leaving it at NaN
3. Ensure that we don't emit cloudwatch metrics with sampleCount == 0
4. Make storage sets behavior consistent with cpu and memory
yumex93
yumex93 previously approved these changes Feb 17, 2020
yhlee-aws
yhlee-aws previously approved these changes Feb 17, 2020
@sparrc sparrc added this to the 1.37.0 milestone Feb 17, 2020
instead we want to be aware of when this happens, also added a detailed
log message for further debugging.
@sparrc sparrc force-pushed the cpu-usage-corner-cases-2 branch from 28337a3 to 2b1495d Compare February 17, 2020 22:48
@sparrc sparrc requested review from yhlee-aws and yumex93 February 17, 2020 22:49
@sparrc sparrc dismissed stale reviews from yhlee-aws and yumex93 February 17, 2020 22:50

significant changes have been added to PR

@sparrc sparrc modified the milestones: 1.37.0, 1.38.0 Feb 19, 2020
@sparrc sparrc force-pushed the cpu-usage-corner-cases-2 branch from 89d16a9 to 34ac24a Compare February 19, 2020 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants