-
-
Notifications
You must be signed in to change notification settings - Fork 827
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CiviUnitTestCase (etal) - Resolve spooky interaction #25855
Conversation
This fixes an issue observed in review of 25673. With that patch included, `JobProcessMailingTest` would still pass... but only in isolation. When combined with `ExceptionTest`, it would fail. I tried disabling the entire substance of `ExceptionTest`, and it still failed. Say what? The problem was _the order_ in which `ExceptionTest` does its initialization (`parent::setUp()` and `$this->useTransaction()`). I suppose that order determines the effectiveness of the transaction. Most tests call `setup()` then `useTransaction()`. I looked for outliers which flipped the order -- and found `EntityTagACLTest` has the same init and the same spooky interaction. After fixing the outliers, I'm getting the expected results on `JobProcessMailingTest`.
(Standard links)
|
Sounds like it might fail a few times before it passes but good to merge when it does |
After looking at the other suites, the notion of "common" and "uncommon" changed, but (so far) not enough to change the approach.
Going literally and by pure numbers, (Full transparency: I don't have a sense for either order being intrinsically better than the other. But standardizing should make things more predictable/consistent, and it should be easier to standardize on the behavior that's more common. 🤞 And it fixed the flakiness on the locking PR.) |
Yeah - I don't care either way - I generally find the |
I'm gonna downgrade this to "merge ready". We want to make sure that #25673 passes for the right reasons, and it might be best to merge this after that one is settled. |
Overview
In #25673, there was a spooky interaction between tests -- which originated in a subtle detail of the
setUp()
process.That PR includes the minimal change. This PR aims to be more proactive -- normalizing the
setUp()
and (hopefully) preventing similar spookiness in the future.Before
Four tests in
api_v3_*
use this uncommon formulation:After
The tests now match their 54 siblings in
api_v3_*
which use:Anything with a flipped sequence will raise an error.
Comments
I only looked at
tests/phpunit/api/v3
. The testbot should now complain about similar issues in other suites.