Failure to resolve modules from yarn workspaces when module has rootDir/outDir #13197

jeantil · 2020-05-21T16:42:12Z

Bug report

Describe the bug

In a yarn workspace based monorepo (derived from examples/with-yarn-workspace), with a typescript module which uses rootDir and outDir settings in tsconfig.json and path aliases in the shared tsconfig.json config.
Next fails to resolve the module even though the tsc build works fine.

To Reproduce

clone the reproduction repository
checkout 2c7f4bd
check that typescript builds correctly yarn workspace web-app tsc -b -v
run yarn dev
the build fails to resolve the baz module

error - ./pages/index.tsx
Module not found: Can't resolve 'baz' in '/home/jean/dev/jsdev/src/demo3/packages/web-app/pages'
ModuleNotFoundError: Module not found: Error: Can't resolve 'baz' in '/home/jean/dev/jsdev/src/demo3/packages/web-app/pages'
    at /home/jean/dev/jsdev/src/demo3/node_modules/webpack/lib/Compilation.js:925:10
    at /home/jean/dev/jsdev/src/demo3/node_modules/webpack/lib/NormalModuleFactory.js:401:22
    at /home/jean/dev/jsdev/src/demo3/node_modules/webpack/lib/NormalModuleFactory.js:130:21
    at /home/jean/dev/jsdev/src/demo3/node_modules/webpack/lib/NormalModuleFactory.js:224:22
    at /home/jean/dev/jsdev/src/demo3/node_modules/neo-async/async.js:2830:7
    at /home/jean/dev/jsdev/src/demo3/node_modules/neo-async/async.js:6877:13
    at /home/jean/dev/jsdev/src/demo3/node_modules/webpack/lib/NormalModuleFactory.js:214:25
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/Resolver.js:213:14
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/home/jean/dev/jsdev/src/demo3/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:13:1)
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/UnsafeCachePlugin.js:44:7
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/home/jean/dev/jsdev/src/demo3/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:13:1)
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/Resolver.js:285:5
    at eval (eval at create (/home/jean/dev/jsdev/src/demo3/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:25:1)
    at /home/jean/dev/jsdev/src/demo3/node_modules/enhanced-resolve/lib/DescriptionFilePlugin.js:67:43

Expected behavior

the build completes and the index page displays all three strings foo bar baz

System information

OS: Linux
Browser (if applies) not relevant but the tests were done in firefox
Version of Next.js: next@latest (resolved to 9.4.2 see yarn.lock)
Version of Node.js: v12.16.3

Additional context

Trying to apply next-transpile-modules to baz didn't help either.
This does not use project references, only path aliases.
Removing the main entry in the baz module's package.json doesn't change the issue

The text was updated successfully, but these errors were encountered:

jeantil · 2020-05-26T16:19:01Z

TLDR;

Use tsconfig-paths-webpack-plugin (see repro repository for the full config file )

    config.resolve.plugins = [
      ...config.resolve.plugins,
      new TsconfigPathsPlugin(),
    ];

The full rambling story

Since this was considered a good first issue I figured I might be able to figure it out

The only way I found to dump the actual effective webpack config and stat output was to plug console.log calls directly in packages/next/build/compiler.ts not sure if there was something more appropriate.

I was able to make the project compile by changing the webpack config but it opens almost more questions than it solves.
Assuming <rootDir> is the root of the monorepo (where the .git folder lives).
Webpack reports that it is running in <rootDir>/packages/web-app which makes sense.

Resolving the baz alias using a path relative to the current directory doesn't work so ../baz/src which would translate to <rootDir>/packages/web-app/../baz/src doesn't resolve the module.

However baz/src does resolve to <rootDir>/node_modules/baz/src probably thanks to
webpack.resolve.modules which contains [node_modules]. (nevermind the fact that <rootDir>/packages/web-app/node_modules is empty. Webpack somehow manages to find the <rootDir>/node_module, I haven't been able to figure out how it does that.

So the imports of baz now resolve to <rootDir>/node_modules/baz/src/index.ts which gets transpiled to javascript without any additional configuration. just like <rootDir>/node_modules/foo/index.ts.
Since duplicating all the mappings with all the right relative paths is fairly tedious, there is a webpack plugin for this which worked right away.

The reproduction repository master has been updated accordingly at jeantil/next-9-ts-aliases-workspaces@ad0b342

Oh my tsx

Interestingly, the bar directory which contains index.tsx won't compile without enabling the next-transpile-modules even thought the .tsx files in <rootDir>/packages/web-app/pages compile just fine and the .ts files in the workspace packages also compile just fine without it.

Adding"include": ["src/**/*.ts", "src/**/*.tsx"] to a tsconfig file in bar didn't change anything

I thought for a moment the tsx file was excluded by return /node_modules/.test(path) at https://github.com/zeit/next.js/blob/canary/packages/next/build/webpack-config.ts#L766 since with the plugin webpack reported a resolution in node_modules so I reverted the tsconfig-path-webpack-plugin for a minute and added the following and the

 config.resolve.modules = [
      path.join(__dirname, ".."),
      ...config.resolve.modules,
    ];
config.resolve.alias = {
      ...config.resolve.alias,
      baz: "baz/src",
    };

to the webpack config.

The foo, bar and baz modules are now resolved to ../foo,../bar and ../baz instead of <rootDir>/node_module/{foo,bar,baz} but bar still fails to compile.

../bar/index.tsx 1:18
Module parse failed: Unexpected token (1:18)
You may need an appropriate loader to handle this file type, currently no loaders are configured to process this file. See https://webpack.js.org/concepts#loaders
> const Bar = () => <strong>bar</strong>
| 
| export default Bar

Dumping all the paths handled by the exclude rule on next-babel-loader is also confusing since none of foo, bar or baz files seem to be handled by it, or at least not directly (I'll pass on how unhelpful the webpack error message seems to be :) )

My next best guess was that there is some kind of config in babel loader that knows how to handle .ts files but not .tsx

I tried a custom babel.config.json[1] with the following content

  "presets": [
    [
      "next/babel",
      {
        "preset-typescript": { 
            "isTsx": true , 
            "allExtensions": true,
        }
      }
    ]
  ]
}

Since I wasn't sure the config was being applied I also checked the next babel preset and added both options right in there but that still failed.

That's as far as I was able to go, I guess i'll let the next-transpile-modules be for a while longer.
Hopefully people who come across a similar issue later will find this issue, my explorations and be able to understand and propose a fix for that last wart.

[1] I'm not sure why the documentation recommends .babelrc over babel.config.json when the babel documentation recommends the latter (but I admit that the use case section at the top doesn't make much sense to me)

jeantil · 2020-05-26T21:53:15Z

`next-transpile-modules` counter-attacks

In the previous episode, resolving the typescript module baz proved difficult but ~~the resistance~~ the developper was able to ~~successfully blow up the death star~~ successfully build the project by fixing relative paths or using ~~the force~~ a webpack typescript plugin.

In this new episode the developer has turned a bit more to the dreaded ~~dark side of the force~~ typescript and used a forbidden feature: enum this leads to a new compile error which seems to demonstrate that the initial fix was not enough and that the ~~corrupt republic~~ framework was lying when it said the project compiled fine, that was only because the typescript code was processable as js code and not using actual typescript features.

TLRD;

Using next-transpile-modules works and the build completes

The longer read

In an interesting plot twist:

putting the enum in a ts file in packages/web-app/components or in packages/web-app/pages and using a relative import to that file in pages/index.tsxdoes compile fine
putting the enum in a ts file in packages/web-app/components or in packages/web-app/pages and using a relative import to that file in pages/index.tsxdoes compile fine
putting the enum in a ts file in packages/baz/src/index.ts and using a relative file import in pages/index.tsx to ../../baz/src/index does not compile
putting the enum in packages/baz/src/index.ts and using an import alias to module baz does not compile
So it looks like something, somewhere is configured to only transpile files in or below the directory where next.config.js resides.
Any import even absolute outside of that directory will not be transpiled.

digging further into the default build config, it looks like the baseUrl resolution mecanism, doesn't honor typescript's extends directive. Therefore if the baseUrl and paths have been defined in an extended base configuration file it will not be detected by next.

a workaround is to manually recreate a next-babel-loader which includes the baseUrl and duplicates the existing next-babel-loader, this works but is quite brittle. updates to next base configuration would have to be actively replicated. identifying the existing rule to inject the baseUrl is not trivial either as rules seem to be highly polymorphic which makes locating the loader name pretty hard.

In any case it looks like not including resolvedBaseUrl in https://github.com/zeit/next.js/blob/canary/packages/next/build/webpack-config.ts#L761 is a mistake but it wouldn't be enough to fix the specific issue demonstrated in the repo.

Typescript configuration can inherit from files above cwd in the filesystem. If a baseUrl was declared in such a file, it would not be picked up by the webpack config. This would force users to use the next-transpile-module and duplicate configuration with unintuitive path relations (see vercel#13197 for a detailed analysis) If baseUrl is resolved it should be used instead of dir as the root include for babel-resolve-loader. An even nicer DX would auto detect the presence of a `paths` section in the typescript config and automatically include `tsconfig-paths-webpack-plugin`

Typescript configuration can inherit from files above cwd in the filesystem. If a baseUrl was declared in such a file, it would not be picked up by the webpack config. This would force users to use the next-transpile-module and duplicate configuration with unintuitive path relations (see vercel#13197 for a detailed analysis) If baseUrl is resolved it should be used instead of dir as the root include for babel-resolve-loader.

balazsorban44 · 2022-01-29T20:07:17Z

This issue has been automatically locked due to no recent activity. If you are running into a similar issue, please create a new issue with the steps to reproduce. Thank you.

timneutkens added good first issue Easy to fix issues, good for newcomers help wanted labels May 25, 2020

jeantil mentioned this issue May 29, 2020

improves baseUrl resolution in typescript monorepos #13542

Merged

Timer removed the help wanted label Jun 3, 2020

maraisr mentioned this issue Jul 17, 2020

jsx file doesnt like jsx? #15251

Closed

timneutkens closed this as completed in #13542 Jul 28, 2020

vercel locked as resolved and limited conversation to collaborators Jan 29, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failure to resolve modules from yarn workspaces when module has rootDir/outDir #13197

Failure to resolve modules from yarn workspaces when module has rootDir/outDir #13197

jeantil commented May 21, 2020 •

edited

Loading

jeantil commented May 26, 2020

jeantil commented May 26, 2020

balazsorban44 commented Jan 29, 2022

Failure to resolve modules from yarn workspaces when module has rootDir/outDir #13197

Failure to resolve modules from yarn workspaces when module has rootDir/outDir #13197

Comments

jeantil commented May 21, 2020 • edited Loading

Bug report

Describe the bug

To Reproduce

Expected behavior

System information

Additional context

jeantil commented May 26, 2020

TLDR;

The full rambling story

Oh my tsx

jeantil commented May 26, 2020

next-transpile-modules counter-attacks

TLRD;

The longer read

balazsorban44 commented Jan 29, 2022

jeantil commented May 21, 2020 •

edited

Loading

`next-transpile-modules` counter-attacks