You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When replaying the jobs back into gearmand from redis it uses the KEYS command, which is a blocking operation in redis that the redis documentation recommend not be used in production:
Don't use KEYS in your regular application code.
If gearmand is hooked up to it's own redis instance this may not be an issue, but if other parts of your application share redis this is likely to be painful if gearmand restarts.
The quick 'fix' to this is to move from using KEYS to using SCAN, however this still has to scan over all keys in the redis instance which will be potentially slow if there are other keys being stored in redis from other applications.
A more complete fix is to track all of the keys added in a set, e.g. when calling SET key data, also do SADD gearman_jobs key, and the complementary DEL and SREM. This then allows for jobs to be reloaded using SMEMBERS which only looks through relevant keys contained within the set, instead of all keys.
The text was updated successfully, but these errors were encountered:
This could be further expanded to have a set for each function_name and then a set for all function_names. This would add an extra lookup, e.g. SMEMBERS gearman_functions then looping through the results to do SMEMBERS function_name, but would make inspecting jobs in redis more user friendly.
See my other comments on your other redis queue suggestions. This one is indeed something that would improve the redis plugin. If that's something you still want to do, patches will be welcomed.
When replaying the jobs back into gearmand from redis it uses the
KEYS
command, which is a blocking operation in redis that the redis documentation recommend not be used in production:If gearmand is hooked up to it's own redis instance this may not be an issue, but if other parts of your application share redis this is likely to be painful if gearmand restarts.
The quick 'fix' to this is to move from using
KEYS
to usingSCAN
, however this still has to scan over all keys in the redis instance which will be potentially slow if there are other keys being stored in redis from other applications.A more complete fix is to track all of the keys added in a set, e.g. when calling
SET key data
, also doSADD gearman_jobs key
, and the complementaryDEL
andSREM
. This then allows for jobs to be reloaded usingSMEMBERS
which only looks through relevant keys contained within the set, instead of all keys.The text was updated successfully, but these errors were encountered: