AWX 9.2 stops processing jobs

[wbieniek]

ISSUE TYPE

• Bug Report

SUMMARY

AWX stops processing/accepting new callback jobs,pending jobs grown to 5K in few hours. There is no running jobs.
We have 6K servers in AWX with callbacks configured to check back every 6 hours.
Only way to fix it is to delete all pending jobs and processing will start again for few hours, then stops again.

ENVIRONMENT

• AWX version: 9.2
• AWX install method: openshift with 15 pods
  Ansible version: X.Y.Z
• Operating System:RH7
• Web Browser:

STEPS TO REPRODUCE

Upgrade to 9.2 from 9.1

EXPECTED RESULTS

ACTUAL RESULTS

Upgraded to 9.2 from 9.1 and within hours AWX stopped processing new requests

ADDITIONAL INFORMATION

Reverted to 9.1 and all is good again.

[wbieniek]

We also have stage AWX 9.2 deployment on OpenShift with 2 pods and 60 hosts under management and we never experience any issues there.

[ryanpetrello]

Duplicate of #5617

[ryanpetrello]

@wbieniek,

If you're interested, I'm looking for people to check if #6093 resolves the issue for them (a few people - including one in issue #5617 - have reported promising results).