Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

engine: migrate status update manager to job queue #4164

Merged
merged 44 commits into from
Dec 10, 2024

Conversation

mastercactapus
Copy link
Member

Description:
Migrates status updates to use the job queuing system.

Previously, the status manager would run maintenance operations every tick, followed by each subscription in a tight loop (up to 100 per tick) keeping a slice of IDs to omit (which was passed to the DB for each call).

The new approach:

  • hourly periodic job to trim old subscriptions
  • every 5 seconds a look-for-work periodic job that schedules jobs for individual subscriptions
  • each subscription runs in it's own job in the queue, up to 5 concurrently (management tasks take priority)

Out of Scope:

  • Adding a trigger from the CreateOrUpdate method of alerts to immediately schedule things like status updates
  • With the above ^ we could update the "look for work" job to run once per minute instead

Describe any introduced user-facing changes:
N/A

Describe any introduced API changes:
N/A

AlaricWhitney
AlaricWhitney previously approved these changes Nov 29, 2024
AlaricWhitney
AlaricWhitney previously approved these changes Dec 3, 2024
KatieMSB
KatieMSB previously approved these changes Dec 4, 2024
@mastercactapus mastercactapus marked this pull request as draft December 6, 2024 18:08
@mastercactapus mastercactapus marked this pull request as ready for review December 9, 2024 20:05
@ethan-haynes ethan-haynes merged commit f5955f4 into master Dec 10, 2024
9 checks passed
@ethan-haynes ethan-haynes deleted the status-mgr-jobqueue branch December 10, 2024 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants