Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

engine/cleanupmgr: move schedule data cleanup to job queue #4203

Merged
merged 29 commits into from
Dec 17, 2024
Merged

engine/cleanupmgr: move schedule data cleanup to job queue #4203

merged 29 commits into from
Dec 17, 2024

Conversation

mastercactapus
Copy link
Member

@mastercactapus mastercactapus commented Dec 17, 2024 •
edited
Loading

Description:
This pull request moves the schedule data cleanup operations to the job queue. It is in charge of:

  • Removing shifts for invalid/deleted users
  • Removing temporary schedules that occur in the past

Out of Scope:

  • API key cleanup will be moved in a separate PR
  • Alert log cleanup will be moved in a separate PR

Additional Info:

  • Cleanup operation will run once per day, and each schedule is processed once per month
  • A catch was put in so that if a schedule has invalid JSON it will be logged and skipped (rather than perpetual failing)
  • Added missing periodic job for schedule shift cleanup

@mastercactapus
cleanup: implement context-aware sleep and cleanup manager functionality
0c3dbd6
@mastercactapus
cleanup: format code for consistency in alert store initialization
9cc35dd
@mastercactapus
cleanup: remove unused ConfigSource field from SetupArgs struct
e5c72d7
@mastercactapus
Merge branch 'master' into cleanup-mgr-alert-jobs
a208406
@mastercactapus
cleanup: refactor alert cleanup logic and add shift cleanup functiona…
7ba52bb 
…lity
@mastercactapus
cleanup: refactor alert cleanup logic to use whileWork for better con…
bb0b63e 
…trol flow
@mastercactapus
Merge branch 'cleanup-mgr-alert-jobs' into cleanup-mgr-shifts
a4cc12c
@mastercactapus
cleanup: add comment
859194a
@mastercactapus
Merge branch 'cleanup-mgr-alert-jobs' into cleanup-mgr-shifts
66b17bf
@mastercactapus
cleanup: remove unused cleanup statements from DB and update logic
be2a5d1
@mastercactapus
test: refactor alert auto-close and cleanup tests for improved reliab…
f3bf6dd 
…ility
@mastercactapus
Merge branch 'cleanup-mgr-alert-jobs' into cleanup-mgr-shifts
7f6a499
@mastercactapus
Merge branch 'master' into cleanup-mgr-shifts
145d1c7 
# Conflicts:
#	engine/cleanupmanager/db.go
#	engine/cleanupmanager/queries.sql
#	gadb/queries.sql.go
@mastercactapus
fix: update ShiftArgs Kind to reflect cleanup-manager-shifts
eeea937
@mastercactapus
Merge branch 'master' into cleanup-mgr-shifts
7976e6c
@mastercactapus
feat: add periodic jobs for schedule data cleanup and update queries
26523b7
@mastercactapus
Merge branch 'master' into cleanup-mgr-schedule-data
ff5d164 
# Conflicts:
#	engine/cleanupmanager/queries.sql
#	engine/cleanupmanager/update.go
@mastercactapus
feat: add workers for cleanup of shifts and schedule data
2a28887
@mastercactapus
fix: update periodic job interval for schedule data cleanup to 24 hours
f8c555d
@mastercactapus
feat: add logging support to cleanup manager and engine initialization
9ee4383
@mastercactapus
refactor: streamline schedule data cleanup by extracting user validat…
8494e91 
…ion and shift trimming logic
CameronJHall
CameronJHall requested changes Dec 17, 2024
View reviewed changes
Copy link
Collaborator

@CameronJHall CameronJHall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One question, the changes look like they accomplish what they're meant to though

CameronJHall
CameronJHall approved these changes Dec 17, 2024
View reviewed changes
Copy link
Collaborator

@CameronJHall CameronJHall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name refactoring helps distinguish the different inputs, thanks for making those changes

@mastercactapus mastercactapus merged commit 8884bbb into master Dec 17, 2024
7 checks passed
@mastercactapus mastercactapus deleted the cleanup-mgr-schedule-data branch December 17, 2024 22:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@CameronJHall CameronJHall CameronJHall approved these changes

@KatieMSB KatieMSB KatieMSB approved these changes

Assignees
No one assigned
Labels
size/l
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@mastercactapus @CameronJHall @KatieMSB