Retrying failed jobs causes PHP memory exhaustion errors when dealing with thousands of failed jobs #49185
Labels
Comments
|
A similar issue was reported in #33874. |
crynobone
added a commit
that referenced
this issue
Nov 30, 2023
… when dealing with thousands of failed jobs fixes #49185 Signed-off-by: Mior Muhammad Zaki <crynobone@gmail.com>
taylorotwell
added a commit
that referenced
this issue
Nov 30, 2023
… when dealing with thousands of failed jobs (#49186) * [10.x] Fixes retrying failed jobs causes PHP memory exhaustion errors when dealing with thousands of failed jobs fixes #49185 Signed-off-by: Mior Muhammad Zaki <crynobone@gmail.com> * Apply fixes from StyleCI * formatting --------- Signed-off-by: Mior Muhammad Zaki <crynobone@gmail.com> Co-authored-by: StyleCI Bot <bot@styleci.io> Co-authored-by: Taylor Otwell <taylor@laravel.com>
taylorotwell
added a commit
to illuminate/queue
that referenced
this issue
Nov 30, 2023
… when dealing with thousands of failed jobs (#49186) * [10.x] Fixes retrying failed jobs causes PHP memory exhaustion errors when dealing with thousands of failed jobs fixes laravel/framework#49185 Signed-off-by: Mior Muhammad Zaki <crynobone@gmail.com> * Apply fixes from StyleCI * formatting --------- Signed-off-by: Mior Muhammad Zaki <crynobone@gmail.com> Co-authored-by: StyleCI Bot <bot@styleci.io> Co-authored-by: Taylor Otwell <taylor@laravel.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Laravel Version
10.34.2
PHP Version
8.1.26
Database Driver & Version
No response
Description
A project I'm working on has 300k failed jobs. When I try to run
artisan queue:retryit results in an error...PHP Fatal error: Allowed memory size of 536870912 bytes exhaustedAfter looking into how the
RetryCommandworks, I realized that rather than querying just the IDs from the failed jobs table, it loads the entire table and then plucks the ID column. No matter how high I set the memory limit, the server won't be able to handle loading all 300k failed jobs into memory.This is a very inefficient way to load the failed job IDs. Any thoughts on how this could be improved without breaking anything?
Steps To Reproduce
failed_jobstable with 300k failed jobsphp artisan queue:retryThe text was updated successfully, but these errors were encountered: