Documentation surrounding Queued unique jobs and logic

[joelharkes]

This is as far as i understand the jobs processing works in combination with locks. Can anyone validate what i wrote is correct?

• Worker: downloads messages from queue and starts processing
• CallQueuedHandler: actually extracting and processing jobs

Jobs can be made unique by:

• uniqueFor() => period it should be unique for, or when the job is handled. => default = 0
• uniqueId() => makes job unique within tis context.
• uniqueVia() => choose caching interface (replace default cache)

Locks are set when job is put on the queue via PendingDispatch, which is not done when you manually call (dispatcher::class)->dispatch()!. but is done calling the dispatch($job) helper function.

Lock releases are done in CallQueuedHandler.
Both if job fails (exception thrown, with still some retries to go) or is released manually within the job, the lock is not removed, but also not reset!

Catches

So the catches are the following:

• make sure you dispatch your job via the PendingDispatch class otherwise locks are not set.
• uniqueFor() => is the uniqueness period over all jobs retries! if job fails the lock timeout won't be recalculated
  • DatabaseLock/cache implementation has default uniqueFor of 1 day! => this can risk delayed jobs being dispatched multiple times if delay is more than 1 day.
• if you use: ShouldBeUniqueUntilProcessing, lock is removed before processing, no matter if the job fails afterwards.
• Redis default lock time doesnt expire, but database lock time default is 1 day: $this->seconds > 0 ? time() + $this->seconds : Carbon::now()->addDays(1)->getTimestamp();

Retry attemps

the attemps are increased when the job is popped from the queue. (always 1 for non queued jobs)

• When you manually release jobs again there is no check on max attemps
• When you throw an exception the Worker class will check the maximum amount of retries (set on job or default set when running queue:work job) and throw an MaxAttemptsExceededException when ran out of attempts
• when retrying a failed job via command: queue:retry the attempts are reset to 0.

[joelharkes]

The database lock expiry of 1 day is been there since the start, this i think makes it confusing to use locks in combination with database Caching: b78880a

[joelharkes]

added PR to at least make it configurable: #47854