-
-
Notifications
You must be signed in to change notification settings - Fork 267
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Split Celery queues into more functional categories #2907
Comments
I think this is a good idea! I also think we could simply not add imported reviews to the activity streams. |
WesleyAC
added a commit
to WesleyAC/bookwyrm
that referenced
this issue
Jul 20, 2023
WesleyAC
added a commit
to WesleyAC/bookwyrm
that referenced
this issue
Jul 20, 2023
WesleyAC
added a commit
to WesleyAC/bookwyrm
that referenced
this issue
Jul 20, 2023
WesleyAC
added a commit
to WesleyAC/bookwyrm
that referenced
this issue
Jul 20, 2023
WesleyAC
added a commit
to WesleyAC/bookwyrm
that referenced
this issue
Jul 20, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently, we have five celery queues: Low priority, Medium priority, High priority, Imports, and Broadcasts. I propose (as described in #2874 (comment)), splitting these up into more functionality-related categories (seeing as the different "priority" queues are not actually prioritized differently):
In particular, I propose starting with the following queues:
bookwyrm.suggested_users.bulk_add_instance_task
bookwyrm.suggested_users.bulk_remove_instance_task
bookwyrm.suggested_users.remove_suggestion_task
bookwyrm.suggested_users.remove_user_task
bookwyrm.suggested_users.rerank_suggestions_task
bookwyrm.suggested_users.rerank_user_task
bookwyrm.preview_images.generate_edition_preview_image_task
bookwyrm.preview_images.generate_site_preview_image_task
bookwyrm.preview_images.generate_user_preview_image_task
bookwyrm.preview_images.remove_user_preview_image_task
imagekit.cachefiles.backends._generate_file
bookwyrm.emailing.send_email
bookwyrm.models.activitypub_mixin.broadcast_task
bookwyrm.activitystreams.add_status_task
bookwyrm.activitystreams.add_book_statuses_task
bookwyrm.activitystreams.add_user_statuses_task
bookwyrm.activitystreams.handle_boost_task
bookwyrm.activitystreams.populate_stream_task
bookwyrm.activitystreams.remove_book_statuses_task
bookwyrm.activitystreams.remove_status_task
bookwyrm.activitystreams.remove_user_statuses_task
bookwyrm.models.import_job.import_item_task
bookwyrm.models.import_job.start_import_task
bookwyrm.connectors.connector_manager.create_edition_task
bookwyrm.connectors.connector_manager.load_more_data
bookwyrm.lists_stream.add_list_task
bookwyrm.lists_stream.add_user_lists_task
bookwyrm.lists_stream.populate_lists_task
bookwyrm.lists_stream.remove_list_task
bookwyrm.lists_stream.remove_user_lists_task
bookwyrm.views.inbox.activity_task
bookwyrm.activitypub.base_activity.set_related_field
bookwyrm.models.antispam.automod_task
bookwyrm.models.user.get_remote_reviews
bookwyrm.models.user.set_remote_server
There are also some cases where different tasks should be in different queues depending on other circumstances, such as
add_status_task
probably wanting to go in a import-related queue for backdated statuses (currently it goes in the low priority queue in that case)We will need to keep around the existing queues for a release in addition to the new queues, so that we can upgrade without dropping any tasks.
The text was updated successfully, but these errors were encountered: