Fixes Storybook/Webpack Out of Memory Error by using relative path to config stories location #1509
Conversation
| @@ -6,7 +6,7 @@ const { getSharedPlugins } = require('../webpack.common') | |||
|
|
|||
| module.exports = { | |||
| stories: [ | |||
| `${getPaths().web.src}/**/*.stories.{tsx,jsx,js}`.replace(/\\/g, '/'), | |||
| '../../../../web/src/**/*.stories.{tsx,jsx,js}', | |||
There was a problem hiding this comment.
Is there no way to solve this via improvement to getPaths()?
I'm not sure if this is correct, but I feel like we keep running into various path issues, patching them, and then running into them once again down the road.
Seems soon we need improved util functions that handle all cases and have corresponding tests. @dthyresson Maybe step one is to get this in. Then in a separate PR come back to getPaths() and harden appropriately.
There was a problem hiding this comment.
Yeah, maybe we should use: https://www.npmjs.com/package/escape-path-with-spaces
There was a problem hiding this comment.
@thedavidprice > Is there no way to solve this via improvement to getPaths()?
Perhaps. But from what I have seen, paths are often used in different contexts and depending on that context how they are "formed" or escaped (or not) matters.
I initially thought the same -- fix the origin "path" and all downhill will work fine.
But some cases still failed.
One such case is the DATABASE_URL used as an env-ar in the Prisma schema. If that has extra " or was escaped, it failed.
Are you suggesting that in the internal paths here:
web: {
routes,
base: path.join(BASE_DIR, 'web'),
pages: path.join(BASE_DIR, PATH_WEB_DIR_PAGES),
components: path.join(BASE_DIR, PATH_WEB_DIR_COMPONENTS),
layouts: path.join(BASE_DIR, PATH_WEB_DIR_LAYOUTS),
src: path.join(BASE_DIR, PATH_WEB_DIR_SRC),
...
Instead of path.join can a function that escape-path-with-spaces and does a path.join()?
Not sure that will help with processPagesDir since it also does its own path.joins or ensurePosixPath that does its own.
Should ensurePosixPath really be escape-path-with-spaces?
For this PR I will try to use escape-path-with-spaces on the Storybook stories config using the existing get paths and see for that fairs.
There was a problem hiding this comment.
I tried
module.exports = {
stories: [
`${escape(getPaths().web.src)}/**/*.stories.{tsx,jsx,js}`
],
and unfortunately, this did not help.
I still got memory errors because of the path:
╰───────────────────────────────────────────────────╯
webpack building...
10% building 2/2 modules 0 active
<--- Last few GCs --->
[97886:0x110008000] 241721 ms: Scavenge (reduce) 4027.7 (4100.7) -> 4027.1 (4101.7) MB, 4.2 / 0.0 ms (average mu = 0.089, current mu = 0.032) allocation failure
[97886:0x110008000] 241731 ms: Scavenge (reduce) 4027.7 (4100.7) -> 4027.2 (4102.0) MB, 5.7 / 0.0 ms (average mu = 0.089, current mu = 0.032) allocation failure
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
1: 0x1012b8fd5 node::Abort() (.cold.1) [/Users/dthyresson/.nvm/versions/node/v14.7.0/bin/node]
Perhaps because not only do I have a space, but I also have parentheses.
So far the only approach that works is to:
- enclose these paths in double quotes as needed
- relative paths
|
I'd be happy to give this a run through, but I don't currently have a good project to run it on. @dthyresson Do you have project up on GitHub I could clone and run this on? I tried running storybook on just the tutorial project (only first part of the tutorial completed), but didn't get any Out of Memory errors. I get some other errors, but I don't think they're related, and storybooks seems to start and run fine
|
What I do is:
|
That's basically what I did too, except I only have spaces in my path, not parentheses. So will try that as well |
|
So, with parentheses I still don't get any OOM errors. But it also doesn't work 😜
As you can see in the screen shot above, it did some funny quoting with the path, starting at the left parenthesis. I will try with your fix now. |
|
Updated to RW 0.21 and for some reason the quoting is different
Still doesn't find any stories though. @dthyresson Here's the output after applying your fix
And with that fix, StoryBook works
|
|
TL;DR I don't seem to get Out of Memory errors, but I do get other errors. @dthyresson's changes fixes my errors, and StoryBook works. |
|
@Tobbe confirms that the relative path approach works on Windows as well -- with a path that has spaces and parentheses. |
…h-tokens * 'main' of github.com:redwoodjs/redwood: Move whatwg-fetch from devDep to dep Adds mockCurrentUser() to api-side jest v0.22.1 v0.22.0 Revert "Configure ApolloProvider to always use fresh tokens (redwoodjs#1609)" (redwoodjs#1611) Configure ApolloProvider to always use fresh tokens (redwoodjs#1609) Ignore *.scenarios.* files. (redwoodjs#1607) upgrade prisma v2.12.1 (redwoodjs#1604) Test Scenarios (redwoodjs#1465) Use relative path to config stories location (redwoodjs#1509)
Fixes #1496
An out of memory error occurs in my project:
I determined that the root cause -- and as with several other recent issues -- had to do with spaces and other problem characters in my path.
I believe that Storybook was trying to look at some other very "root"-like or user paths for stories which could contain a huge number of files and the splat simply caused an out of memory error while traversing it.
The following change from:
to a relative path
as in:
Worked.
This PR now uses a relative path to configure where Storybook should find the stories.
Questions for @peterp and others like @Tobbe (who is on Windows):