[tests-only] [full-ci] Add acceptance tests #1030
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The PHP unit tests were passing both before and after PR #817 - so the unit tests were not effective in finding the problem with code after the Guzzle7 update in core. See issue #1029
To cover the real Guzzle client code that lists available apps and downloads app tarballs from the market-place, we really have to do real requests to the market-place. I looked at the unit tests and it didn't look like it would be easy/possible to mock out stuff while still actually covering the things that might go wrong.
So I have added acceptance tests. The downside is that they require the live production market-place server to be up. So they might sometimes fail if the market-place server is down (for maintenance or broken or...) The tests only run in the CI of this app, so it is not going to be an issue for regular core... CI.
I locally tried reverting the code fix in PR #817 and running the acceptance tests. I get:
So the acceptance tests do cover that problem.