Storybook: Add StoryShots integration to generate unit tests

[gziolo]

Description

Part of #17973.
Depends on #18030.

StoryShots adds automatic Jest Snapshot Testing for Storybook.

I followed the docs at this page:
https://github.com/storybookjs/storybook/tree/master/addons/storyshots/storyshots-core

Testing

Run npm run test-unit packages/components/storybook/

Known issue

There is a known issue with using React hooks with components in stories:
storybookjs/storybook#8177

I'm seeing the following:

  ● @wordpress/components › Checkbox Control › Default

    Invariant Violation: Invalid hook call. Hooks can only be called inside of the body of a function component. This could happen for one of the following reasons:
    1. You might have mismatching versions of React and the renderer (such as React DOM)
    2. You might be breaking the Rules of Hooks
    3. You might have more than one copy of React in the same app
    See https://fb.me/react-invalid-hook-call for tips about how to debug and fix this problem.

      17 | 
      18 | export const _default = () => {
    > 19 |      const [ isChecked, setChecked ] = useState( true );
         |                                        ^
      20 |      return (
      21 |              <CheckboxControl
      22 |                      label="Is author"

This issue was resolved by refactoring #18030. This PR was rebased with update/storybook-enhancements to solve it here as well.

[gziolo]

I figured out that some tests can't work in this setup. ClipboardButton is one of them:

We can skip such stories using the convention. In this case, it would be DontTest keyword in the title. It shows up in the UI at the moment, but it should be easy to introduce a helper function which appends this pattern only in the test env.

[gziolo]

cc @diegohaz and @ItsJonQ who recently spent some time on writing unit tests for UI components. I'd love to hear your feedback on the presented approach.

[epiqueras]

I figured out that some tests can't work in this setup. ClipboardButton is one of them:

What makes it break? Is there anyway we can mock it?

In this case, it would be DontTest keyword in the title.

We can also just add an explicit list of (?!) to the storyKindRegex since the list will probably be somehow limited.

[gziolo]

I figured out that some tests can't work in this setup. ClipboardButton is one of them:

What makes it break? Is there anyway we can mock it?

In this case, it would be DontTest keyword in the title.

We can also just add an explicit list of (?!) to the storyKindRegex since the list will probably be somehow limited.

I figured out that some tests can't work in this setup. ClipboardButton is one of them:

This is the full error:

  ● @wordpress/components › ClipboardButton › Default

    TypeError: Cannot read property 'firstChild' of null

      27 |      componentDidMount() {
      28 |              const { container, getText, onCopy } = this;
    > 29 |              const button = container.firstChild;
         |                                       ^
      30 | 
      31 |              this.clipboard = new Clipboard( button, {
      32 |                      text: getText,

      at ClipboardButton.firstChild [as componentDidMount] (packages/components/src/clipboard-button/index.js:29:28)
      at commitLifeCycles (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:10930:22)
      at commitLayoutEffects (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:13583:7)
      at HTMLUnknownElement.callCallback (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:10487:14)
      at invokeEventListeners (node_modules/jsdom/lib/jsdom/living/events/EventTarget-impl.js:193:27)
      at HTMLUnknownElementImpl._dispatch (node_modules/jsdom/lib/jsdom/living/events/EventTarget-impl.js:119:9)
      at HTMLUnknownElementImpl.dispatchEvent (node_modules/jsdom/lib/jsdom/living/events/EventTarget-impl.js:82:17)
      at HTMLUnknownElementImpl.dispatchEvent (node_modules/jsdom/lib/jsdom/living/nodes/HTMLElement-impl.js:30:27)
      at HTMLUnknownElement.dispatchEvent (node_modules/jsdom/lib/jsdom/living/generated/EventTarget.js:157:21)
      at Object.invokeGuardedCallbackDev (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:10537:16)
      at invokeGuardedCallback (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:10590:31)
      at commitRootImpl (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:13355:9)
      at unstable_runWithPriority (node_modules/scheduler/cjs/scheduler.development.js:643:12)
      at runWithPriority (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:1887:10)
      at commitRoot (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:13184:3)
      at runRootCallback (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:12420:20)
      at node_modules/react-test-renderer/cjs/react-test-renderer.development.js:1935:24
      at unstable_runWithPriority (node_modules/scheduler/cjs/scheduler.development.js:643:12)
      at runWithPriority (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:1887:10)
      at flushSyncCallbackQueueImpl (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:1931:7)
      at flushSyncCallbackQueue (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:1920:3)
      at scheduleUpdateOnFiber (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:12297:9)
      at scheduleRootUpdate (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:15087:3)
      at updateContainerAtExpirationTime (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:15115:10)
      at updateContainer (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:15134:10)
      at create (node_modules/react-test-renderer/cjs/react-test-renderer.development.js:15824:5)
      at getRenderedTree (node_modules/@storybook/addon-storyshots/dist/frameworks/react/renderTree.js:24:16)
      at node_modules/@storybook/addon-storyshots/dist/test-bodies.js:21:18
      at Object.<anonymous> (node_modules/@storybook/addon-storyshots/dist/api/snapshotsTestsTemplate.js:44:33)

What makes it break? Is there anyway we can mock it?

I guess there is something going on with how it gets mounted. I don't think it's so much important and we can skip it but feel free to investigate further.

In this case, it would be DontTest keyword in the title.

We can also just add an explicit list of (?!) to the storyKindRegex since the list will probably be somehow limited.

We can start with the blacklist at first. Good idea 👍

[epiqueras]

Sounds good, let's do it.

[gziolo]

With 7c6e7b3 we have a list of skipped components in the Storyshots configuration. It should be ready to go. This branch is based on the #18030 so we can merge it there once approved.

[ItsJonQ]

I'd love to hear your feedback on the presented approach.
@gziolo Hallo! Apologies for the delay!

From my experience, snapshot testing (in general) is definitely better than nothing, but I prefer having and writing unit/integration tests.

I also think writing unit tests should also be easier, which can be achieved by having testing utils (if we're staying with Enzyme) or leveraging another testing solution.

Personally, I haven't used this particular addon for Storybook.

I'm okay with keeping this in to see where it goes/how it feels :)

Maybe it can automate a portion of manually writing basic render tests, freeing up time/focus to write unit/integration tests for more complex scenarios.

[jorgefilipecosta]

I was able to run the tests with success by using "npm run test-unit storybook/test/index.js".

I'm not sure if these tests will be useful in catching regressions or not but I think the best way is to merge them and check if they helped us or they added developer overhead to update the snapshots.
We also have some interesting potential follow-ups pending like the a11y tests, so I guess we should merge this PR and see where these type of tests leaves us.

[gziolo]

I invested more time into this issue and I managed to mock those nodes which use refs with the following commits:
1a59025
ccc705b

This means, we no longer skip any stories from snapshot testing. However, the side effect is that if there is some special use case for refs, it might require custom logic in the mocking helper. The following seem to work for now:

		createNodeMock: ( element ) => {
			if ( story.kind === 'Components|ClipboardButton' ) {
				return {
					firstChild: document.createElement( 'button' ),
				};
			}
			return element.type && document.createElement( element.type );
		},

I followed links shared by @epiqueras:

• https://reactjs.org/blog/2016/11/16/react-v15.4.0.html#mocking-refs-for-snapshot-testing
• https://github.com/storybookjs/storybook/tree/master/addons/storyshots/storyshots-core#using-createnodemock-to-mock-refs

It looks like it's a general issue with snapshot testing in React apps.

[diegohaz]

One thing I noticed is that storyshot tests will still fail if a console warning is emitted. On #17875, I created a story for a deprecated thing so I could see the UI worked and the warning was shown.

On normal tests I can check if the warning was shown. Can I do that with Storyshots?

[gziolo]

One thing I noticed is that storyshot tests will still fail if a console warning is emitted. On #17875, I created a story for a deprecated thing so I could see the UI worked and the warning was shown.

This raises the question of whether we should keep the legacy version of Toolbar in Storybook 🤔

I could make the point that it should be removed from there as it should be discouraged from using in new code. Having it included in the docs might create the temptation for folks to copy the example and use it in their projects when they find that it fits their needs.

I'm aware that it isn't probably the type of answers you expected but this is something we should discuss more broadly before we jump into finding a technical solution on how to mitigate it. Maybe we don't have to do it. The only downside is that this story is very useful in the process of developing the migration path for the Toolbar component.

[gziolo]

I included documentation in 92f46dd.