Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add goroutine_by_scheduling_operation metric to kube_scheduler check #15697

Merged
merged 13 commits into from
Sep 8, 2023

Conversation

jennchenn
Copy link
Member

@jennchenn jennchenn commented Aug 25, 2023

What does this PR do?

Add a new metric kube_scheduler.goroutine_by_scheduling_operation to track the number of running scheduler goroutines split by scheduling operation.

Motivation

In Kubernetes v1.26.0, the metric to track the number of running goroutines in the scheduler was renamed to scheduler_goroutines; the old metric (scheduler_scheduler_goroutines) was deprecated, and removed in Kubernetes v1.28.0. This added metric follows the new naming convention so it can be captured by our users; it is named kube_scheduler.goroutine_by_scheduling_operation to avoid confusion with the kube_scheduler.goroutines metric we already provide.

Additional Notes

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Changelog entries must be created for modifications to shipped code
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.

@jennchenn jennchenn requested review from a team as code owners August 25, 2023 19:47
@jennchenn jennchenn requested a review from a team as a code owner August 25, 2023 19:48
@ghost ghost added documentation and removed changelog/Added labels Aug 25, 2023
@github-actions
Copy link

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

github-actions bot commented Aug 25, 2023

Test Results

  4 files    4 suites   10s ⏱️
  6 tests   6 ✔️ 0 💤 0
14 runs  12 ✔️ 2 💤 0

Results for commit 00ec865.

♻️ This comment has been updated with latest results.

@codecov
Copy link

codecov bot commented Aug 25, 2023

Codecov Report

Merging #15697 (00ec865) into master (3567074) will increase coverage by 4.17%.
Report is 4 commits behind head on master.
The diff coverage is 100.00%.

Flag Coverage Δ
confluent_platform ?
hudi ?
kube_scheduler 97.45% <100.00%> (+3.68%) ⬆️
weblogic ?

Flags with carried forward coverage won't be shown. Click here to find out more.

📢 Have feedback on the report? Share it here.

urseberry
urseberry previously approved these changes Aug 25, 2023
@jennchenn jennchenn changed the title Add scheduler_goroutines metric to kube_scheduler check Add goroutine_by_scheduling_operation metric to kube_scheduler check Aug 29, 2023
@@ -9,6 +9,7 @@ kube_scheduler.gc_duration_seconds.count,gauge,,,,"Number of the GC invocation",
kube_scheduler.gc_duration_seconds.quantile,gauge,,,second,"GC invocation durations quantiles",0,kube_scheduler,goroutines,
kube_scheduler.gc_duration_seconds.sum,gauge,,,second,"GC invocation durations sum",0,kube_scheduler,goroutines,
kube_scheduler.goroutines,gauge,,,,"Number of goroutines that currently exist",0,kube_scheduler,goroutines,
kube_scheduler.goroutine_by_scheduling_operation,gauge,,,,"Number of running goroutines split by the work they do such as binding (requires k8s v1.26+)",0,kube_scheduler,goroutine_by_scheduling_operation,
Copy link
Contributor

@clamoriniere clamoriniere Aug 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you also specify in the metric description that this metrics is in alpha stage so they can be removed upstream on the next kubernetes version

I think we have some metric on which we add this info

sblumenthal
sblumenthal previously approved these changes Sep 7, 2023
@@ -0,0 +1,132 @@
# (C) Datadog, Inc. 2023-present
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comments I would have here are the same as the ones I have for your other PR

Copy link
Contributor

@yzhan289 yzhan289 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for agent integ!

@jennchenn
Copy link
Member Author

/merge

@dd-devflow
Copy link

dd-devflow bot commented Sep 8, 2023

🚂 MergeQueue

This merge request is not mergeable yet, because of pending checks/missing approvals. It will be added to the queue as soon as checks pass and/or get approvals. You can remove it from the waiting list with /remove command.

you can cancel this operation by commenting your pull request with /merge -c!

@jennchenn
Copy link
Member Author

/remove

@dd-devflow
Copy link

dd-devflow bot commented Sep 8, 2023

🚂 Devflow: remove

@dd-devflow
Copy link

dd-devflow bot commented Sep 8, 2023

⚠️ MergeQueue

This merge request was unqueued

If you need support, contact us on slack #ci-interfaces!

@jennchenn jennchenn merged commit 8274fa9 into master Sep 8, 2023
@jennchenn jennchenn deleted the jenn/CONT-3655-add-kube-scheduler-goroutines-metric branch September 8, 2023 14:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants