Prove that MetaMask can be properly e2e tested with cypress

[danjm]

The task is to prove that we can properly test MetaMask with https://github.com/cypress-io/cypress

Cypress has an immense amount of benefits compared to our current e2e testing tools. It also solve a number of issue inherent in those tools. For a quick overview of advantages: https://www.cypress.io/features/

Cypress does not currently support testing of chrome extensions out of the box. There is an ongoing issue regarding this cypress-io/cypress#1965. Some users seem to have had some success with work arounds and some amount of stubbing extension apis.

It is unclear if we can stub all the extension behaviour necessary for a thorough testing of MetaMask. This task could be successfully resolved by showing this is possible via a single test of MetaMask interacting with a dapp (e.g. confirming a "Send" from the test dapp currently in our e2e/beta directory).

An alternative would be to hack on the Cypress codebase and prove that it can be updated to support chrome extensions like MetaMask. The above link issue is currently assigned to a Cypress team member, but if we were to make a PR, we might be able to help progress the issue. Even if our PR is rejected, if we put together a working set of changes, we can develop our tests while waiting for Cypress to resolve the issue themselves.

It is unclear to me which of these alternatives will require the most/least time and effort.

cc @whymarrh @tmashuang

[rstormsf]

Have you looked at
https://www.npmjs.com/package/metamask-puppeteer based on https://github.com/GoogleChrome/puppeteer

There is some repo with examples:
https://github.com/OasisDEX/metamask-e2e

@danjm

[danjm]

thanks @rstormsf, this looks promising

[danjm]

Doing this will be a lot of work, and too much risk of delay due to unknown unknowns. Also a risk that we will come up with a solution that will require changes after the new version of the chrome manifest comes into effect.

I will provide more detail later, but going to put this on ice for now.

[mitar]

There is also this plugin which claims to provide testing browser extensions in Cypress: https://github.com/ejoubaud/cypress-browser-extension-plugin

[Gudahtt]

We're no longer planning to use Cypress, as it still doesn't support testing extensions.

[jtakalai]

Jakub seems to have succeeded in Metamask testing with Cypress in headed mode by injecting Metamask into the browser with puppeteer, or something. See:

• https://github.com/Synthetixio/synpress
• https://www.reddit.com/r/QualityAssurance/comments/kdm6qw/synpress_cypress_wrapper_with_metamask/
• https://stackoverflow.com/questions/61863739/cypress-test-approach-for-testing-with-metamask