multiple node_modules colision

[lgandecki]

Issue description or question

When I have a structure:

test-package/
package.json
node_modules/
   server/
       package.json
       node_modules/
   mobileclient/
       package.json
       node_modules/

And I add the node_modules to NODE_PATH like this:

process.env.NODE_PATH +=
    path.delimiter + path.join(wallaby.localProjectDir, 'node_modules') +
    path.delimiter + path.join(wallaby.localProjectDir, 'server', 'node_modules') +
    path.delimiter + path.join(wallaby.localProjectDir, 'mobileClient', 'node_modules') 

I get errors.. while jest works.

(some background, feel free to skip @ArtemGovorov but I hope this will get you as excited about the possibility and wow factor here as we are :-) )
We've been building an example application that's a monorepo of decoupled apps, but where

1. you can test them in a way that breaking your server brakes tests for both mobile and web clients.
2. you run the same tests with enzyme, and cypress (the same exact file describes action, and behind the scenes with help of babel rewrites, they either use enzyme to perform them, or cypress). It works beautifully with jest.
   With a single package, when I was not reusing the server files to mock the graphql server on the web/mobile clients I was able to have fantastic, mind-blowing tests of type 2 (meaning, I have cypress e2e and also jest/enzyme tests with wallaby passing, I change something in one test file in wrong way - they both fail, also, when I change something in my script, they both fail and turn green in the same times.) It's easier to show than to explain and we are preparing to do just that in early Febraury, but we do want to give a huge shotout to wallabyjs there, if we can get this piece to work :)

In general - what this means, almost-end to end tests (with graphql server and in memory database, with no state shared between tests!) with wallaby, and immediate feedback. :) The only thing we are not testing here is whether the server can connect to the express server properly, and the way passport does the authorization, for that we will have a full-run e2e test that works on a unmocked client, running server, and real db running in a background. This will just ensure that everything is glued properly together, most of the other stuff is either pure logic, or frontend orchestration (like changing state/routes etc)

(/background)

Repro:

git clone https://github.com/TheBrainFamily/TheBrain2.0/
cd TheBrain2.0
npm run buildAll
npm run enzymeTest
npm run enzymeTestMobile

(they both should be green, as for circle build: https://circleci.com/gh/TheBrainFamily/TheBrain2.0/1009 )

and then use either wallaby-mobile.js or wallaby-web.js to see the problems

My guess is that all node_modules are treated as being on "the same level", so when I do require("graphql") in server/ and I included client/node_modules first (that also include graphql), it resolves the graphql from client/node_modules instead of server/node_modules causing some issues. Would there be any way to resolve node_modules in a way that honors the path of the file that does the require/import? How difficult is this?

Please let us know if there is any chance of having this supported in the next few weeks! I'm happy to help with any way I can. I actually made this prematurely open source just so you can play around with it.

Wallaby.js configuration file

https://github.com/TheBrainFamily/TheBrain2.0/blob/wallabyIssue/wallaby-mobile.js
https://github.com/TheBrainFamily/TheBrain2.0/blob/wallabyIssue/wallaby-web.js

Code editor or IDE name and version

WebStorm

OS name and version

OSX

[ArtemGovorov]

This is indeed an exciting setup! Thanks for sharing.

And I add the node_modules to NODE_PATH like this:

This was causing the issues that you are observing, because it is confusing Jest (when ran by Wallaby). You also don't need to include any node_modules in your files.

Would there be any way to resolve node_modules in a way that honors the path of the file that does the require/import? How difficult is this?

No changes from our side are required.

You may just tell Jest (when running from wallaby) how to resolve node_modules by using the moduleDirectories setting. When running without wallaby, Jest is able to run without the setting, but Wallaby run Jest in its own cache dir, so the hit is required. In case of your mobile testing setup, you also need to clear the process.env.NODE_PATH = '';, because wallaby sets it to local node_modules by default (and it's not desirable in your case because you have react-apollo and a few other modules installed in both ./node_modules and ./mobileClient/node_modules).

This is the only/most important change(s) that you really need. Other than that, you may clean up your files list to get rid of node_modules where required, and only include what you need for your tests to run.

I have sent you 2 PRs with config changes:
Mobile: TheBrainFamily/TheBrain2.0#94
Web: TheBrainFamily/TheBrain2.0#93

So your wallaby-mobile.js now looks very simple:

process.env.BABEL_ENV = "test"
process.env.ENZYME = true
process.env.NODE_ENV = "TESTING"

module.exports = (wallaby) => {
  return {
    debug: true,
    testFramework: 'jest',
    files: [
      'mobileClient/**/*.js',
      '!mobileClient/node_modules/**',
      'server/src/**/*.js',
      '!server/src/**/*.spec.js',
      'testing/testHelpers/**/*.js',
      'testingMobile/**/*.snap'
    ],
    tests: [
      'testingMobile/App.test.js',
    ],
    compilers: {'**/*.js': wallaby.compilers.babel()},
    env: {type: 'node'},
    setup: function (wallaby) {
      process.env.NODE_PATH = '';
      const jestConfig = require('./package.json').jest;
      jestConfig.moduleDirectories = [
        'node_modules', '<rootDir>/server/node_modules', '<rootDir>/mobileClient/node_modules'
      ];
      wallaby.testFramework.configure(jestConfig);
    }
  }
};

and your wallaby-web.js is even simpler:

process.env.BABEL_ENV="test"
process.env.ENZYME=true
process.env.NODE_ENV="TESTING"

module.exports = (wallaby) => {
  return {
    debug: true,
    testFramework: 'jest',
    files: [
      'client/src/**/*.js',
      'client/tests/testHelpers/**/*.js',
      '!client/src/**/*.test.js',
      'server/src/**/*.js',
      '!server/src/**/*.spec.js',
      'cypress/integration/pageObjects/*.js',
      'testing/testHelpers/**/*.js',
      'clientEnzymeTest.json'
    ],
    tests: [
      'cypress/integration/landingPage.test.js',
    ],
    compilers: {'**/*.js': wallaby.compilers.babel()},
    env: {type: 'node'},
    setup: function (wallaby) {
      const jestConfig = require('./clientEnzymeTest');
      jestConfig.moduleDirectories = [
        'node_modules', '<rootDir>/server/node_modules', '<rootDir>/client/node_modules'
      ];
      wallaby.testFramework.configure(jestConfig);
    }// --config ./clientEnzymeTest.json
  }
};

[ArtemGovorov]

It's easier to show than to explain and we are preparing to do just that in early Febraury, but we do want to give a huge shotout to wallabyjs there, if we can get this piece to work :)

Awesome, try the config changes above, it should work. If you mention Wallaby in some article/blog/video describing this setup, please let us know, would love to check it out!

[lgandecki]

Great! They all start up separately now. So looks like this will be possible to have set up properly :-)

As always - thanks for a best-class support!

I still have problems though:
Looks like I can't run them at the same time, and that's really what I need (and what I currently do with jest).
If i run mobile first, and then web I get this error:

 Cannot find module 'AsyncStorage' from 'setup.js'

      1 | /* eslint-env jest */$_$wp(93);
    > 2 | $_$w(93, 0), jest.mock('AsyncStorage', function () {
      3 |     $_$wf(93);
      4 |     return $_$w(93, 1), mockImpl;
      5 | });
      
      at Resolver.resolveModule (node_modules/jest-resolve/build/index.js:194:17)
      at Object.<anonymous> (../../Library/Caches/WebStorm2017.3/wallaby/projects/ad5382c13ebbdc55/instrumented/mobileClient/tests/helpers/setup.js:2:19)
      at Runtime.u._wallabyReplaced.l.requireModule (../../Library/Caches/WebStorm2017.3/wallaby/wallaby/runners/node/jest@0.4.3/initializer.js:14:17457)
      at Object.<anonymous> (../../Library/Caches/WebStorm2017.3/wallaby/projects/ad5382c13ebbdc55/instrumented/testingMobile/App.test.js:52:30)
      at Runtime.u._wallabyReplaced.l.requireModule (../../Library/Caches/WebStorm2017.3/wallaby/wallaby/runners/node/jest@0.4.3/initializer.js:14:17457)

If I run web first and then mobile it complaines about missing jquery

mobile first, and then server, the server complains about

Failed to run compilers on testing/testHelpers/startAppEnzyme.js, SyntaxError: testing/testHelpers/startAppEnzyme.js: Unexpected token, expected , (59:20)
  57 | 
  58 |   const wrapper = mount(
> 59 |     <ApolloProvider client={client} store={createTheBrainStore(history)}><MainContainer history={history}/></ApolloProvider>);
     |                     ^
  60 | 
  61 |   return new EnzymeDriver(wrapper)
  62 | }

Which it shouldn't even load.

With server first and then web:

I initially get no errors in web (but server side tests stop to reload when I change the resolvers (server/src/api/resolvers.js , test for them are in resolvers-graphql.spec.js in the same folder), and I see errors in the log:

 wallaby:project Error: ENOENT: no such file or directory, open '/Users/lukaszgandecki/Library/Caches/WebStorm2017.3/wallaby/projects/ad5382c13ebbdc55/original/src/api/resolvers.js'

client test is in: cypress/integration/landingPage.test.js

mobile: testingMobile/App.test.js

They all work properly separately. Which suggests to me that there is some kind of a collision between wallabyjs caches?

Also, do I even get this right, even if they started up, would I be able to see separate columns of dots there? (on the server, for example, I would expect to see three - one from the server unit tests, one from the mobile integration tests and one from the web integration tests. I would also expect to reload all three kind of tests when I change my graphql resolvers)

[ArtemGovorov]

Wallaby doesn't support (never had) running multiple configurations within one IDE instance. Apart from a lot of complexity that such behaviour implementation would bring, there's also a few things to consider regarding merging/isolating feedback/coverage from different configurations.

Having said that, if you're able to open multiple IDE windows with the same project, you should be able to start each wallaby config in each window (however I'm not sure if it's possible to open the same project in different WebStorm instances).

[lgandecki]

shoot :-) this would be mind blowing if it worked..
Is there any chance we could help prioritizing this feature by "sponsoring" it? I see there were a few good suggestions in the post you linked.
For now we could get away with running wallaby for what we are directly working with, but long term the usability would be much better and more convincing..

Our goal here is to have a starter of a react-native (expo)/react(create-react-app)/graphql-express server with everything set up and connected some example tests and wallaby configured, with bare minimum, and also a real-life example of a medium-sized app ( https://thebrain.pro ) to give people an understanding of how to structure and do more advanced things (like how to sue fb login, notifications on mobile, or optimistic ui with graphql, etc) .
I hope this would also convince more people to get your product :)

[lgandecki]

The other way of having this to work would be to use jest to run tests with multiple configs..
I see this: jestjs/jest#2970 where they state "This is happening in the next version of Jest. " but I don't think that actually happened? I will dig.

Or, maybe this is something wallabyjs could do itself? To run all those tests using one wallaby config?

[ArtemGovorov]

Thanks for sharing your plans and for your offer. It would be really good to have the feature working for your project, as well as for other multi-config projects, such as full stack JS apps with front end plus node server. The team is fully occupied right now and for the next few months, but we are planning to expand the team later this year, so while I can't make any promises, we'll definitely consider implementing the feature and offer to help.

…

On 14 Jan 2018, at 5:01 pm, Łukasz Gandecki ***@***.***> wrote: shoot :-) this would be mind blowing if it worked.. Is there any chance we could help prioritizing this feature by "sponsoring" it? I see there were a few good suggestions in the post you linked. For now we could get away with running wallaby for what we are directly working with, but long term the usability would be much better and more convincing.. Our goal here is to have a starter of a react-native (expo)/react(create-react-app)/graphql-express server with everything set up and connected some example tests and wallaby configured, with bare minimum, and also a real-life example of a medium-sized app ( https://thebrain.pro ) to give people an understanding of how to structure and do more advanced things (like how to sue fb login, notifications on mobile, or optimistic ui with graphql, etc) . I hope this would also convince more people to get your product :) — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub, or mute the thread.

[ArtemGovorov]

The other way of having this to work would be to use jest to run tests with multiple configs

I think Jest supports with the projects setting, however Wallaby only runs global Jest config, not project specific ones.

[lgandecki]

looks like projects would work indeed - is this something that would be easier to implement with wallabyjs then multi-wallabyjs functionality?

[ArtemGovorov]

Will need to investigate, it may be easier than implementing multiple running configurations feature.

[lgandecki]

I just had a moment and made our repo work with jest projects, pretty cool stuff, here is what I had to change:

TheBrainFamily/TheBrain2.0#97

basically I run the tests with
ENZYME=true BABEL_ENV=test NODE_ENV=TESTING jest --projects testingMobile cypress server

the projects are paths to the directories that have jest.config.js inside them, and setup the test execution for tests found in that particular folder.

I thought I'd share in case you wanted a playground to investigate this. There is no rush, but if you had a chance to put a few hours into this during the next week or so that would be fantastic!