📖 Add a design for a priority queue #3013
Conversation
|
/hold |
designs/priorityqueue.md
Outdated
| // Adding an item that is already there may update its wait | ||
| // period to the lowest of existing and new wait period or | ||
| // its priority to the highest of existing and new priority. | ||
| AddWithOpts(o AddOpts, items ...T) |
There was a problem hiding this comment.
The main alternate to this would be to have AddWithPriority, AddAfterWithPriority AddRatelimitedWithPriority etc - I found this easier but not a strong opinion either way.
There was a problem hiding this comment.
Definitely a lot easier to extend without breaking changes
ea46cac to
f988aa0
Compare
| reasons to prioritize some events") will always require implementation of a custom | ||
| handler or eventsource in order to inject the appropriate priority. | ||
|
|
||
| ## Implementation stages |
There was a problem hiding this comment.
Will the default controller be modified to make use of this new queue (if used), or will it rely on using a custom controller implementation?
There was a problem hiding this comment.
Yes it will be updated to make use of it - the only thing it needs is to re-use the priority though. Once we make it the default, I would also like to add a Priority parameter to the reconcile.Result - But really only once its enabled by default.
There was a problem hiding this comment.
Maybe also to the Request?
Ah, interesting thought. Is the idea because you want to be able to set the priority in a handler?
The way this currently works is that the workqueue doesn't have any understanding of the request object (and we should keep it that way IMHO). We could probably provide a thin wrapper for it that will use the priority from the request if the AddWithOptions call doesn't have one yet and inject that in the builder if its typed to reconcile.Request?
There was a problem hiding this comment.
I was thinking that the priority could also be useful information for the Reconcile func. But maybe it's a bad idea not sure 😀
There was a problem hiding this comment.
Ah. Because then it can use that as an input when returning a priority?
There was a problem hiding this comment.
That's one use case. Another one would be to reconcile Requests of different priorities differently :)
Maybe it makes sense in some cases to pass down the priority (if some other components are involved in reconciliation).
Maybe a controller would act differently if it can infer based on the priority if this is just a periodic resync (or something similar) vs an actual change. Or in general prioritize Requests with higher priority higher (if it has to schedule some "tasks" in other systems)
But I'm really not sure if this is a good idea or opening pandora's box :)
There was a problem hiding this comment.
Let's follow-up on this with more use cases later on, if needed
| reasons to prioritize some events") will always require implementation of a custom | ||
| handler or eventsource in order to inject the appropriate priority. | ||
|
|
||
| ## Implementation stages |
This change describes the motivation and implementation details for a priority queue in controller-runtime.
f988aa0 to
bd6eede
Compare
| reasons to prioritize some events") will always require implementation of a custom | ||
| handler or eventsource in order to inject the appropriate priority. | ||
|
|
||
| ## Implementation stages |
There was a problem hiding this comment.
Let's follow-up on this with more use cases later on, if needed
|
LGTM label has been added. Git tree hash: 8ebae6bb55c02eaf9f95880c35574df92895f6e0
|
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: alvaroaleman, vincepri The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/hold |
|
/hold cancel |
This change describes the motivation and implementation details for a priority queue in controller-runtime.
Ref #2374
POC of the changes is in #3014