feat(symbolicator): Check manual low priority queue kill switch when pre-processing events #28586
Conversation
src/sentry/tasks/store.py
Outdated
| symbolicate_event_from_reprocessing_low_priority | ||
| # if from_reprocessing | ||
| # else symbolicate_event_low_priority |
There was a problem hiding this comment.
i was thinking of ignoring reprocessing for the lowpriority and only use it for brand new events. but... maybe that's naive and you're right. It does mean we have to make sure to handle a bunch more cases.
There was a problem hiding this comment.
Yeah, my impression was that there's nothing that explicitly prevents events in the low priority queue from being reprocessed. It doesn't make sense to promote reprocessed LPQ events up to the regular queue for obvious reasons, and my understanding was that we deliberately have a separate code path for reprocessing for metrics or tracking purposes.
What are some of the cases you're thinking of as a result of this? I'll admit that I have a fairly surface-level understanding of reprocessing right now. I'm assuming this diff doesn't cover most of them.
e17f37f to
86f35ac
Compare
86f35ac to
c34dbdf
Compare
…pre-processing events
0127460 to
9c82029
Compare
b8cb387 to
980ed52
Compare
| Queue( | ||
| "events.reprocessing.symbolicate_event_low_priority", | ||
| routing_key="events.reprocessing.symbolicate_event_low_priority", | ||
| ), |
There was a problem hiding this comment.
we will need ops to make sure that this queue is handled exclusively by the same workers which handle the events.symbolicate_event_low_priority queue.
There was a problem hiding this comment.
good point, thanks for the heads up! when you phrase it this way it seems like maybe it's better to just throw reprocessing requests back into the LPQ... i'll start a conversation about this and change this later if it turns out that would be preferable. i assume by having a separate queue for reprocessing requests we could least easily drop everything in that queue without negatively impacting new requests.
Starter work to push symbolicator events to the low priority queue when the appropriate manual kill switch is specified.
Some code is commented out because it relies on changes made in #28561.
This follows up on the work done in #28576; This PR is the companion that finally uses the switches registered in #28576.