-
Notifications
You must be signed in to change notification settings - Fork 906
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: reconcile pod ephemeral metadata in parallel #4130
feat: reconcile pod ephemeral metadata in parallel #4130
Conversation
Signed-off-by: Jordan Rodgers <jrodgers@figma.com>
fe13e00
to
4595bcb
Compare
appsv1 "k8s.io/api/apps/v1" | ||
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" | ||
|
||
"github.com/argoproj/argo-rollouts/pkg/apis/rollouts/v1alpha1" | ||
replicasetutil "github.com/argoproj/argo-rollouts/utils/replicaset" | ||
) | ||
|
||
// DefaultEphemeralMetadataThreads is the default number of worker threads to run when reconciling ephemeral metadata |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copied verbiage from
argo-rollouts/controller/controller.go
Lines 73 to 86 in a312e28
// DefaultRolloutThreads is the default number of rollout worker threads to start with the controller | |
DefaultRolloutThreads = 10 | |
// DefaultExperimentThreads is the default number of experiment worker threads to start with the controller | |
DefaultExperimentThreads = 10 | |
// DefaultAnalysisThreads is the default number of analysis worker threads to start with the controller | |
DefaultAnalysisThreads = 30 | |
// DefaultServiceThreads is the default number of service worker threads to start with the controller | |
DefaultServiceThreads = 10 | |
// DefaultIngressThreads is the default number of ingress worker threads to start with the controller | |
DefaultIngressThreads = 10 |
@@ -299,6 +302,7 @@ func newCommand() *cobra.Command { | |||
command.Flags().IntVar(&analysisThreads, "analysis-threads", controller.DefaultAnalysisThreads, "Set the number of worker threads for the Experiment controller") | |||
command.Flags().IntVar(&serviceThreads, "service-threads", controller.DefaultServiceThreads, "Set the number of worker threads for the Service controller") | |||
command.Flags().IntVar(&ingressThreads, "ingress-threads", controller.DefaultIngressThreads, "Set the number of worker threads for the Ingress controller") | |||
command.Flags().IntVar(&ephemeralMetadataThreads, "ephemeral-metadata-threads", rollout.DefaultEphemeralMetadataThreads, "Set the number of worker threads for the Ephemeral Metadata reconciler") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copied verbiage from
argo-rollouts/cmd/rollouts-controller/main.go
Lines 297 to 301 in a312e28
command.Flags().IntVar(&rolloutThreads, "rollout-threads", controller.DefaultRolloutThreads, "Set the number of worker threads for the Rollout controller") | |
command.Flags().IntVar(&experimentThreads, "experiment-threads", controller.DefaultExperimentThreads, "Set the number of worker threads for the Experiment controller") | |
command.Flags().IntVar(&analysisThreads, "analysis-threads", controller.DefaultAnalysisThreads, "Set the number of worker threads for the Experiment controller") | |
command.Flags().IntVar(&serviceThreads, "service-threads", controller.DefaultServiceThreads, "Set the number of worker threads for the Service controller") | |
command.Flags().IntVar(&ingressThreads, "ingress-threads", controller.DefaultIngressThreads, "Set the number of worker threads for the Ingress controller") |
Published E2E Test Results 4 files 4 suites 3h 11m 1s ⏱️ For more details on these failures, see this check. Results for commit da425f4. ♻️ This comment has been updated with latest results. |
Published Unit Test Results2 295 tests 2 295 ✅ 2m 59s ⏱️ Results for commit da425f4. ♻️ This comment has been updated with latest results. |
Signed-off-by: Jordan Rodgers <jrodgers@figma.com>
7023e70
to
da425f4
Compare
|
Right now, every pod has it's ephemeral updated serially. When dealing with a large number of pods, this can get to be quite slow, especially when you have things like validating/mutating webhooks that have to run for every request.
This updates it to simply use an errgroup with a default concurrency of 10 to update the pods in parallel, which should greatly speed things up 🎉
Checklist:
"fix(controller): Updates such and such. Fixes #1234"
.