Slow Running tests when window doesn't have focus

[jpetitcolas]

Not sure it it is a bug, or a misunderstanding from mine.

Current behavior:

When testing a MaterialUI component, I got some trouble with hot reloading. If I refresh my browser with F5, my code behaves correctly:

Yet, if test is relaunched using hot reload, select list takes a long time before closing:

Desired behavior:

I would expect the behavior to be the same between fresh refresh or hot reload.

Steps to reproduce:

I created a repository to reproduce the issue: https://github.com/jpetitcolas/cypress-bug

Clone it, then launch the React app and open Cypress:

git clone https://github.com/jpetitcolas/cypress-bug.git
npm install
npm start

# in another term
npm run cypress

Then, comment or uncomment the following line to let browser to hot reload.

Versions

• Cypress: 3.0.1
• OS: Linux Mint 18
• Chrome: Version 66.0.3359.139 (Official Build) (64-bit)

[kuceb]

@jpetitcolas I was able to reproduce this by removing focus of the mouse from the test runner during a run. Notice the run will be fast as long as the mouse is over the window. Can you confirm this behavior, and see if you can reproduce on 2.1.0 ?

[kuceb]

confirmed bug on all cypress versions

• slow in cypress run and cypress open, when window doesn't have focus

[brian-mann]

We detected that the root cause has to do with a combination of things.

1. The browser window must not be in focus
2. You perform an event which should fire focus, but the browser does not, so we polyfill that behavior and hold a reference to the element 'that should have focus but does not'
3. After we perform the action, you perform another action on a different element which should receive focus but does not.
4. We perform the routines to move focus to that element, by blurring the previously focused element which we have an internal reference to.
5. Before we do that, your application changes the activeElement to something other than the one we have a reference to by calling the native .focus() method.
6. Your browser window must still not be in focus.
7. We attempt to blur the previous element we had a reference to, but the blur event does not fire, so we polyfill its behavior.
8. We expect that the blur event is called, but it is not because the browser refuses to fire it due to the activeElement being changed
9. The cy.blur() command does not take this into account and reaches its timeout limit, thus failing
10. The command does not actually fail because its routine was a subroutine as part of another command cy.click().
11. The error does not bubble up and instead it just adds unnecessary time, and Cypress does not properly cleanup its handlers for this situation since it's an edge case

To fix this problem (and others) we need to make the sub focus/blur routes non-async (to the best of our ability) or add checks after async actions to take into account if the activeElement was manually changed by application code instead. In that case, the blur should belong to the activeElement not the internal one we have a reference to.

In the future, we should consider not actually polyfilling the focus and blur events that the browser would normally fire if the window is in focus. I tested how the browser performs with real native events, and it does fire them if the window is visible but not active - but it does not fire them if the window is not visible - such as if your IDE is covering it up, or another chrome window is as well.

[jbinto]

I don't have a full reproduction case or anything ready, so sorry if this is polluting the issue but I will chime in with what I know. We have an issue on one of our repos at work with Cypress and Material-UI which results in erratic behavior when the test runner is not focused. Here is an edited excerpt from our internal report:

---

tl;dr
Until we figure this one out: Don't focus away from cypress when you run it locally. Press play, don't touch the keyboard or mouse, let it do its thing.

Description
Note: This issue only seems to pop up locally in the Cypress interactive/GUI test runner. So far it hasn't happened in CI or on when using npm run cypress:run locally.

When I run cypress tests, it seems now to matter whether or not the Cypress window is focused/in the foreground. If I am on a single laptop screen and tab away from Cypress for too long, and the entire window is obscured/not visible, I start getting test failures.

Versions used
Cypress 2.1.0 using Electron 59 browser
macOS 10.13.4

Steps to reproduce
In a cypress test, navigate to XXXXX and immediately interact with an input (e.g. cy.type() in a field, or cy.click() a delete button).
Put this cypress test in a loop to repeat the test 100 times.
Run the test. Observe it pass e.g. 25 times. If you leave it alone, you will have 100% success rate.
Tab away from cypress and do other things. Make sure the cypress window is not visible.
At some point, you will see intermittent test failures that look like this:
This element '<input.MuiInput-input-278>' is not visible because it has CSS property: 'position: fixed' and its being covered by another element:

Comments
Very strange... curious if anyone else can reproduce. This was happening to me on XXXXXX I was able to get a small reproduction case here:

It passed 34 times in a row. I focused away from cypress, then it started failing repeatedly. I wondered if it was because I pressed CMD+TAB, and accidentally sent a TAB command to the focused browser? No -- that wouldn't make subsequent tests fail.

I feel like this has something to do with the behaviour of Chromium changing because the window isn't visible, i.e. a rendering optimization in Chromium (or macOS, or the display drivers) that interferes with Cypress' interaction, or maybe exposes a race condition that isn't normally there. Also, since I don't see it happen anywhere but on XXXXXX, maybe a hardware-accelerated animation in MaterialUI's CSS is somehow triggering this behaviour.

This is way too vague to report to cypress for now, and I'd like to see if anyone else can reproduce it before going that far.

[brian-mann]

@jbinto we're actually aware of the root issue and have already fixed it in a local branch. This will get released with the next patch update.

[brian-mann]

Released in 3.0.2.

[agualis]

Could this issue be related with this?