Add timeouts to TestCoordinatorReportsInvalidPolicy

[AndersonQ]

What does this PR do?

Adds a timeout to TestCoordinatorReportsInvalidPolicy when trying to receive from the state channel.

Why is it important?

Even thought the coordinator.runLoopIteration is synchronous, the actual state update, which is sent on stateChan, happens on a goroutine. Therefore there is the possibility when the test tries to read the channel, the stat isn't there yet. Whereas unlikely, it has been observed and made the test flaky.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• [ ] I have added tests that prove my fix is effective or that my feature works
• [ ] I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

How to test this PR locally

I tried running it 500000 times and still could not catch a failure, therefore I'd not try to catch the flakyness. Let's CI do its job

Related issues

• Closes [Flaky Test]: TestCoordinatorReportsInvalidPolicy – Vars change should cause state update #4117

Use cases

Screenshots

Logs

Questions to ask yourself

• How are we going to support this in production?
• How are we going to measure its adoption?
• How are we going to debug this?
• What are the metrics I should take care of?
• ...

[elastic-sonarqube]

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarQube

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[blakerouse]

Makes sense to add a little wait for the change, all depends on how golang schedules the goroutines.

I believe the issue you are reference is incorrect, I was unable to find that issue.