Importing Meteor packages (VulcanJS integration)

[SachaG]

A lot of VulcanJS would love to be able to use Storybook, but the problem is that Vulcan relies a lot on Meteor packages, which if I'm not mistaken cannot currently be imported inside Storybook.

And even if we could get rid of most exports in Vulcan component files and end up with "pure" React files, there is still the issue that Vulcan uses a global component system, where instead of doing import MyComponent, you can just write <Components.MyComponent/> (this is in order to enable theming among other things).

I know there's probably no easy workaround currently, but I would love to start a discussion about how to get Storybook working with Vulcan, even if it requires us changing some thing internally?

[SachaG]

See related discussion here: VulcanJS/Vulcan#1939

[Hypnosphi]

FWIW, working with globals shouldn't be a problem

[Hypnosphi]

cannot currently be imported inside Storybook

What's exactly the problem with that? I have found no hints on that in the issue you've linked

[tmeasday]

The problem is that Meteor projects are full of things like import X from 'meteor/X' where meteor/X resolves (via Meteor's build system, not webpack) to an export from a Meteor (not npm) package.

There are some projects that allow you to use Meteor in webpack. I'm not sure how well they work. That would probably be the place to start @SachaG.

[tmeasday]

On the globals thing, how does Components.MyComponent get registered? Is it auto imported by Meteor (i.e. from outside the imports/ folder)? It might not be possible to write a webpack resolve.context that simulates what Meteor does here.. I'm not sure.

[Hypnosphi]

So is it possible to build some assets with Meteor and import them in .storybook/config.js maybe?

[SachaG]

It's not auto imported, all components just get stored in a global Components object using a registerComponent function. See an example here:

https://github.com/VulcanJS/Vulcan-Starter/blob/master/packages/example-simple/lib/components/movies/MoviesList.jsx

[tmeasday]

Well as long as webpack can interpret the imports, the global variables won't be a problem @SachaG

[eric-burel]

Hi, I still think that removing Meteor features in React components to avoid those issue is a no-go. This is the current best solution yet it is only a palliative and not a fix. Vulcan has some nice innovative features, they should not be abandonned because a 3rd party tool gave up Meteor support.

Storybook is historically meant for visual development, however in practice it definitely allows to test the integration of unit components based on state.
E.g. when you use Redux, you can easily test components that communicates with the store or even distant API in storybook.
You should even be able to load your whole app in Storybook, this is great if you want to repro some bugs that happens with a given combination of state and props. Using some mocks, I can test some components that uses Electron in Storybook.

If we can support all the ES7/ES8/ES29 features with babel and webpack and all, and we can wrap components with stores and API and whatever side effect we want, I see no reason why suddenly we should make an exception with Meteor. Even if technically this issue is hard or very hard in my opinion it is definitely worth it.

Now, the question is how we do that :)

[eric-burel]

My idea was to replace Node by Meteor when running tools as Storybook (but also jest and whatever 3rd party tool we use), I don't know if it is doable? That's how we work usually with Electron, this way we can support native modules compiled for Electron for example while using tools meant for Node in the first place.
Edit: I checked it and it does not seem to be doable, the meteor node my-file.js command expects the code to be already bundled, it does not tolerate import as I would have expected.

[jefflau]

So it sounds like there are two realistic options from what I can see:

1. Replace Meteor to another Node Framework within Vulcan so we are totally npm based and webpack compatible.
2. Extend the webpack build system for storybook so it can resolve Meteor imports.

I'm guessing Meteor will never drop their build system and it might be a very long time til Meteor can be totally npm based as there are tons of atmosphere packages too that would need to be converted

[tmeasday]

@eric-burel you know your idea reminded me of something I've been wanting to try that I think could be used to solve this problem. The idea is to render Storybook's preview iframe inside the user's app.

Basically this could ease setup of Storybook because it means you don't need re-write your webpack config for Storybook (in the typical case) or try and run your app inside webpack in Meteor's case.

Consider this proof-of-concept: #3360

[SachaG]

Could https://github.com/ardatan/meteor-webpack be helpful at all?

[Hypnosphi]

@SachaG looks like it solves an opposite problem: use webpack inside Meteor

[eric-burel]

Maybe it would help to run storybook in Meteor as @tmeasday proposed

[DylanLyon]

Has there been any updates on making Meteor work with Storybook? I'm not using VulcanJS but I'm having the same issue when using 'import from meteor/X'. Assuming that changing all my code to not use Meteor import statements is not an option (it would be more trouble than it's worth) what are some alternative options?

Did anyone ever get it working by any of the methods mentioned above?

[tmeasday]

I think with the refactoring that's been going on moving everything into @storybook/core it is probably easier than ever to run storybook on a route from within your Meteor app as I prototyped in #3360.

On that PR I discuss some of the downsides of the approach, but it feels like maybe worth making some small tweaks to core to enable a Meteor package that embeds a route like the above. I'd be happy to make the changes to SB core and guide someone who wants to create that Meteor package.

---

@Hypnosphi what do you think about the idea of making SB compatible with the above? In short, what we need is a way to:

1. Provide an option to storybook's server webpack config so it doesn't build a preview and points the Provider at an arbitrary URL. Something like yarn storybook --previewUrl=http://localhost:3000/__storybook_preview__.

2. run (say) @storybook/react in a mode where it doesn't auto-render to #root, but instead exports a renderTo(domElement) <- to be called by Meteor's router when the user hits __storybook_preview__. This part is mostly trivial after your last PR.

[Hypnosphi]

@tmeasday sounds like a good idea to me

[tmeasday]

Is anyone who is keen on using Meteor with Storybook interested in maintaining a Meteor package and working with me to get this going? Probably not a lot in the Meteor package beyond setting up a route and calling into Storybook..

[eric-burel]

@tmeasday I have not much free time but I am interested, I use storybook intensively whenever I develop UIs. If you create a repo you can add me to it and I'll do my best to investigate the issues you encounter at least.

[DylanLyon]

@tmeasday I'm extremely interested in making SB work with Meteor but I'm afraid I'm out of my depth here so I'm not sure how much help I can be. That being said I'm available to help with anything I can.

[SachaG]

We'd probably use this inside the core Vulcan repo and/or the Vulcan starter repo so I think you'd get a fair share of potential users/maintainers at least.

[tmeasday]

Ok, I'll report back on this issue when I've had a chance to do the two things above.

[DylanLyon]

Any luck on this issue? I've been trying to get meteor-imports-webpack-plugin working because I think it will solve the 'import Meteor from meteor/meteor' webpack issue but the documentation is sorely lacking (not to mention the very first thing it tell you to do is impossible unless I'm just understanding it incorrectly).

[tmeasday]

@DylanLyon that seems pretty cool, definitely worth trying. What do you mean by "very first thing it tell you to do is impossible"? Do you mean having your Meteor project in a subdirectory of your webpack project? That's sort of going to be weird for Storybook but I guess it'll work?

[DylanLyon]

@tmeasday I had misunderstood the documentation was was trying to make a meteor instance inside another meteor instance, which is not allowed. The creator clarified so now I'm going to take another crack at it!

[tmeasday]

Yeah, please let us know!

[stale]

Hi everyone! Seems like there hasn't been much going on in this issue lately. If there are still questions, comments, or bugs, please feel free to continue the discussion. Unfortunately, we don't have time to get to every issue. We are always open to contributions so please send us a pull request if you would like to help. Inactive issues will be closed after 30 days. Thanks!

[tmeasday]

Any luck @DylanLyon?

[DylanLyon]

@tmeasday unfortunately, no. Tried several different things and figured it was more trouble than it was worth and ended up writing my own front-end UI for our documentation using JSDoc data scrapped with react-docgen directly.

[tmeasday]

Cool, thanks for letting us know. I'll circle back around to the other approach sometime soon I hope!

[Discordius]

I would be happy to help maintain any package in this space.

[Batistleman]

This has been quite a roadblock for us to adopt storybooks. It's a bit tricky, but I think this might be a solution:

1. use webpack-strip-block to strip away code when using webpack

     npm install --save-dev webpack-strip-block

2. add this code to your storybooks webpack.config.js

    config.module.rules.push({
        test: /\.js$/,
        enforce: 'pre',
        exclude: /(node_modules|\.spec\.js)/,
        use: [{loader: 'webpack-strip-block'}]
    });

3. Make an /imports/libs/meteor.js file that looks like this:

/* develblock:start */
import { Meteor as importedMeteor }     from 'meteor/meteor';
/* develblock:end */

let Meteor;
if(typeof importedMeteor === 'undefined') {
    Meteor = {};
}
else {
    Meteor = importedMeteor;
}

export {
    Meteor,
};

• this skips the import block (not possible in webpack) the meteor builder will just import as normal.
• the 'typeof' is a trick to check if the variable is defined
  • if it is, we overwrite it with our mocks
  • if it's not, we overwrite it with the imported package

4. change the import for the files you want to test:

from this:
import { Meteor } from 'meteor/meteor';

to this:
import { Meteor } from '/imports/libs/meteor.js';

[tmeasday]

@Batistleman -- interesting approach, although I guess it will fall down if you actually want to us the Meteor import inside your component (as referenced in this comment)

Your approach could probably be extended to a babel plugin that simply does something like:

// replace this
import Meteor from 'meteor/meteor';

// with this
const Meteor = {};

[Discordius]

@tmeasday: Was curious whether you made any progress on this. Still happy to help if there is a way for me to do so.

[tmeasday]

Hi @Discordius -- haven't had a chance to play with this further. I think the most fruitful way forwards at this stage are:

1. Writing the babel plugin mentioned just above -- although it isn't a perfect solution, I think it wouldn't be too hard to write and would make it significantly easier to use Meteor with Storybook. I would be happy to help in any way although I haven't written a babel plugin before.

2. Investigating "running Meteor in webpack" solutions like meteor-imports-webpack-plugin. I'd be interested to hear if that works at all.

I am still interested in the idea of running storybook inside the Meteor app, but it is not something I anticipate working on in the near future. It may end up making sense with some things we've been thinking about re: documentation, and it's definitely something I'll keep in mind when thinking about such things, but I really don't know if it is a direction that makes sense or not in the future yet.

[jbaileyashe]

@tmeasday Not related to meteor - but I would love to host Storybook inside my React app. My motiviation is to be able to re-use my existing webpack/babel/ts-loader configuration (created via create-react-typescript-app). After fighting bugs with awesome-typescript-loader using storybook as a standalone app to no avail, it seems a lot simpler to just host Storybook using my existing configuration.

I looked at your PR #3360, which made it seem this is more than a trivial request. However, it would be quite useful.

Just my $0.02, thanks for all your work on this.

[tmeasday]

Well I suppose we are one step closer to supporting it with the manager/preview split in 4.1 @jbaileyashe. I'll try to revisit the conversation with @ndelangen post SB5 (so in a few weeks time with luck!)

[BudgieInWA]

#6323 introduces a "manager only" mode, with storybook --preview-url=http://localhost:3000, that loads everything except the stories. It seems like a Meteor Test Driver Package that loads the stories (like mocha loads tests) and uses the storybook API to serve the story previews on localhost:3000 would be a good solution. We can let meteor build our source but still configure and run the real storybook UI.

[tmeasday]

That's a cool idea @BudgieInWA!

Another option is building on #6382 to allow the stories to be served off a subpath, and then doing something like:

if (Meteor.isDevelopment) {
  import './.storybook/config';
}

[eric-burel]

For reference we have very experimental but working solution in Vulcan (VulcanJS/Vulcan-Starter#122).

It is similar to the solution proposed by @Batistleman (though less advanced concerning the scraping), it simply mocks Meteor as much as possible and scraps out unmocked Meteor packages imports.
As expected calls to Meteor packages/globals will fail at runtime (or be replaced by mocks), however for frontend development that's good enough since we don't call Meteor systematically client side, at least not in React components. It does not require any change our existing code.

[tmeasday]

@eric-burel yes there are kind of two sides we can approach the "meteor/vulcan in storybook" problem and getting your Vulcan components running in a Meteor-agnostic way (via mocking or better yet never importing from the meteor/ namespace at all) is sort of the "better" way as it would mean you can also use them in a variety of other tools without Meteor.

However, I realise that's a challenge; if you can get it going, awesome! If not, a solution where the component part of SB runs inside Meteor may be a better solution at least for now.

[Type1J]

I just started using StoryBook for a Webpack project, and I saw that it could detect Meteor projects when I set it up, but this thread sounds like there may still be some problems. Most of my other projects are Meteor based, so I'm hoping for this issue to be resolved before I get back to them.

[eric-burel]

Related issue: I tried to use Jest, which is basically the same fundamental problem but 10 times harder: VulcanJS/Vulcan#2270

For Jest, you would prefer Babel based approach, which makes more sense than using Webpack (Babel feels a bit harder but is really magic when you master plugins, + a lot of them are available already). However I failed to handle Meteor package transpilation, so I dropped the issue for now.

(also closing the ticket because we've had Storybook in Vulcan for a while)