Clicks handled differently in headless vs. headed (React + Material UI use case)

[harvitronix]

Current behavior:

I have test cases that click on a styled select and then click on an item in the resulting menu. When I run my tests headed, using npx cypress open, with either Chrome or Electron, the click events happen almost instantly, as expected. However, when I run the same tests headlessly, with npx cypress run, each select item click takes "timeout" seconds to complete the click. This leads to a very slow test suite when timeout is a few seconds and I have hundreds of such clicks.

Here is a video showing the headless run and unexpected behavior:

https://youtu.be/MCbIhEoN7Is

You will see the down click seems to be initiated immediately (the ripple is triggered), but it takes "timeout" ms to complete the click. (I set the defaultCommandTimeout config option to 10,000 to exaggerate the problem.)

Desired behavior:

Here is a video showing the headed run and expected behavior:
https://youtu.be/vVzrdBvhnlg

Steps to reproduce:

I'm using React with Material UI. Here is the component under test in the videos:

import React, { Component } from 'react';
import TextField from '@material-ui/core/TextField';
import MenuItem from '@material-ui/core/MenuItem';

const items = ['Item 1', 'Item 2', 'Item 3'];

class App extends Component {
  state = { item: null };

  handleChange = e => this.setState({ item: e.target.value });

  render() {
    return (
      <div>
        <TextField
          select
          label="Item"
          value={this.state.item}
          onChange={this.handleChange}
          data-cy="select"
        >
          {items.map(item => (
            <MenuItem value={item} key={item} data-cy="menu-item">
              {item}
            </MenuItem>
          ))}
        </TextField>
        <p data-cy="selected-item-label">Selected: {this.state.item || 'None'}</p>
      </div>
    );
  }
}

export default App;

And here is the spec:

describe('Material UI Component Tests', () => {
  beforeEach(() => cy.visit('http://localhost:3001'));
  [0, 1, 2, 1, 0].forEach((i) => {
    it(`should select item ${i} quickly`, () => {
      cy.get('[data-cy="select"]').click()
      cy.get('[data-cy="menu-item"]').eq(i).click()
      cy.get('[data-cy="selected-item-label"]')
        .contains(`Selected: Item ${i + 1}`)
    })
  })
})

Versions

Cypress 3.0.1
Electron 59

[brian-mann]

Does it work correctly with cypress run --browser chrome ?

[harvitronix]

@brian-mann Yes, when I run headed with either Chrome or Electron, it works correctly. It seems the only time it runs incorrectly is headless Electron.

Edit: I updated my CI to use your handy Docker image with Chrome, and when running it in CI, the clicks happen quickly, which is nice. Unfortunately I can no longer get video output, though, so there's a pretty high cost.

[kuceb]

@harvitronix can you see if this is an active window issue, by running cypress run --browser chrome and then quickly clicking on a different window to take focus away from cypress?
We've seen some specs fail in this case

[harvitronix]

@bkucera wow, interesting. If I run with chrome as you suggested and click away from the window, the clicks become very slow again. Then if I put the Chrome browser window back in focus, it starts going fast again.

Good find! @brian-mann does this help at all?

[kuceb]

@harvitronix we're working on this issue, related here: #1909

[harvitronix]

This was fixed in 3.0.2. 💯

[piotralefeducation]

@bkucera I believe this issue is partially fixed.
I have a similar scenario, however, I am clicking on two Material-UI select components one by one. After updating to 3.0.2 first select component works (in headless and in out of focus chrome) but it hangs on the second one which is being tested just after the first one. The problem does not occur when I keep my focus on the cypress Chrome browser.

[brian-mann]

@piotralefeducation that is fixed in 3.0.3

We've normalized all of the focus/blur events to work identically whether or not the browser is in focus

[piotralefeducation]

@brian-mann Thank you and I'm waiting for the release 3.0.3

[medric]

Hi @brian-mann I'm experiencing the same issue with cypress 3.3.2. I have the following link element :

<a class="link browse-tasks-link" data-ui-test="browse-tasks-link">Browse tasks</a>

Looks like it doesn't get clicked when running tests in headless mode. Chrome + Electron --headed work perfectly though. I ran Electron and created a BrowserView of the page where this element lives - was able to manipulate the DOM and trigger the event without any problem.

Do you have any idea of what can go wrong ?

Thank you

[jennifer-shehane]

Hey @medric, please create a new issue detailing exactly how to reproduce the issue and we can look at it. Thanks!