Celery worker stats

[galvana]

Description Of Changes

New endpoint that reports the Celery queue sizes and details of the tasks that are running or were reserved on each worker.

{
  "queues": {
    "fidesops.messaging": {
      "messages": 0
    },
    "fides.privacy_preferences": {
      "messages": 0
    },
    "fides": {
      "messages": 0
    }
  },
  "workers": {
    "celery@91efad939fb9": {
      "active_task": {
        "task_id": "99800e62-0e08-4315-83e8-97cb6552aee3",
        "task_name": "fides.api.task.execute_request_tasks.run_access_node",
        "args": [],
        "kwargs": {
          "privacy_request_id": "pri_d360a064-0488-4835-8c28-13d0582fd16f",
          "privacy_request_task_id": "req_8be1b9ab-e37f-4d13-a26b-f96403f8211d",
          "privacy_request_proceed": true
        },
        "started_at": "2025-02-03T16:43:20.188593"
      },
      "reserved_tasks": [
        {
          "task_id": "c74498dd-5388-4b54-9c3e-41fc4e448d4f",
          "task_name": "fides.api.task.execute_request_tasks.run_access_node",
          "args": [],
          "kwargs": {
            "privacy_request_id": "pri_d360a064-0488-4835-8c28-13d0582fd16f",
            "privacy_request_task_id": "req_ea0dc11c-6533-4fa5-86f1-c7227a460971",
            "privacy_request_proceed": true
          },
          "started_at": null
        },
        {
          "task_id": "f086b10d-b9d3-4c88-abf0-bf937f103596",
          "task_name": "fides.api.task.execute_request_tasks.run_access_node",
          "args": [],
          "kwargs": {
            "privacy_request_id": "pri_d360a064-0488-4835-8c28-13d0582fd16f",
            "privacy_request_task_id": "req_d5bbe078-8d03-4b13-bafc-1aa8e30b9e59",
            "privacy_request_proceed": true
          },
          "started_at": null
        }
      ],
      "registered_tasks": [
        "fides.api.service.messaging.message_dispatch_service.dispatch_message_task",
        "fides.api.service.privacy_request.request_runner_service.run_privacy_request",
        "fides.api.service.privacy_request.request_service.poll_for_exited_privacy_request_tasks",
        "fides.api.service.privacy_request.request_service.remove_saved_dsr_data",
        "fides.api.task.execute_request_tasks.run_access_node",
        "fides.api.task.execute_request_tasks.run_consent_node",
        "fides.api.task.execute_request_tasks.run_erasure_node"
      ]
    }
  }
}

Code Changes

• list your code changes here

Steps to Confirm

1. list any manual steps for reviewers to confirm the changes

Pre-Merge Checklist

• Issue requirements met
• All CI pipelines succeeded
• CHANGELOG.md updated
  • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
  • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
• Followup issues:
  • Followup issues created (include link)
  • No followup issues
• Database migrations:
  • Ensure that your downrev is up to date with the latest revision on main
  • Ensure that your downgrade() migration is correct and works
    • If a downgrade migration is not possible for this change, please call this out in the PR description!
  • No migrations
• Documentation:
  • Documentation complete, PR opened in fidesdocs
  • Documentation issue created in fidesdocs
  • If there are any new client scopes created as part of the pull request, remember to update public-facing documentation that references our scope registry
  • No documentation updates required

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
fides-plus-nightly	⬜️ Ignored (Inspect)	Visit Preview		Feb 4, 2025 11:33pm

[cypress]

fides    Run #12136

Run Properties:   Passed #12136  •  aa56aa8fff ℹ️: Merge 2cedd0c5122e3d45babc4f1628f22c226f604333 into ea3c9ec63f13bc7c33fc8182be56...

Project	fides
Branch Review	refs/pull/5725/merge
Run status	Passed #12136
Run duration	00m 46s
Commit	aa56aa8fff ℹ️: Merge 2cedd0c5122e3d45babc4f1628f22c226f604333 into ea3c9ec63f13bc7c33fc8182be56...
Committer	Adrian Galvan
View all properties for this run ↗︎

---

Test results
Failures	0
Flaky	0
Pending	0
Skipped	0
Passing	5
View all changes introduced in this branch ↗︎