disable tests because react-netty aggregate random order of response … #7770
Conversation
…context occursionally
|
Not a fan of disabling failing tests due to a legitimate issue in the SDK/underlying infrastructure, this feels like a scenario where we say we'll get around to resolving the underlying issue soon but forget until we get a customer reported issue. If we end up merging this please submit an issue for the underlying problem and get it triaged appropriately so we don't let the problem fester. |
|
I agree with @alzimmermsft. I do not want to be in the business of disabling tests eagerly. I would like to understand more about what the issue is and why we can't fix the underlying issue, before we disable these tests. |
|
@anuchandy and I had investigated the failure of tests. I am also not a fan of disabling tests. But since it caused by the react-netty aggregate response content in random order, we can't really solve this problem immediately and we want to make sure other tests work property. Before we file an issue to react-netty, Anu is investigating it and double confirm. |
|
Have these tests ever successfully passed in successive runs, or have they always been flakey? |
|
@JonathanGiles These tests some times failed some times successfully passed. |
|
@mssfang To avoid any confusion, this PR is now pending you making the changes that Josh has requested. |
|
@JonathanGiles Yes. Anu wants to have a final verified before file an issue. Will file an issue and merged it after confirmed. |
|
So we are going to merge this PR. Ran a live test pipeline on this PR: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=246641&view=results. All other tests look good. We are keeping investigating it tomorrow. And leave the issue #7771 open. |
|
Related PR with fix: #7837 |
Disabled listing tests because of unstable react-netty aggregate random order of responses.
Opened a tracking issue for these disabled tests: #7771