-
Notifications
You must be signed in to change notification settings - Fork 27.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refactor: merge turbopack crates into next.js #68127
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
### Description Update missing fixes for the features. Closes PACK-2722
### Description <!-- ✍️ Write a short summary of your work. If necessary, include relevant screenshots. --> ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2724
…borepo#7719) ### Description This can actually happen e. g. with next/font, which creates virtual css files that are not inside of the project directory ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2732
### Description * Slow start for the concurrency of parallel booting node.js processes (1, 2, 4, ...) * Depending on historic statistics avoid booting up a new process, but wait for one to finish computing Loading the `/home` route on 16-core machine Before: ``` 16 x Node.js process startup 9.09s avg 568.07ms 16 x Node.js initialization 10.58s avg 660.96ms 276 x Node.js evaluation 30.17s avg 109.30ms ``` After: ``` 7 x Node.js process startup 1261.79ms avg 180.26ms 7 x Node.js initialization 3.15s avg 450.21ms 276 x Node.js evaluation 29.41s avg 106.55ms ``` ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2680
…ercel/turborepo#7720) ### Description Async module handling was missing for side effects optimized modules ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2733
…cel/turborepo#7718) ### Description `lightningcss` enforces the class name to end with `[local]`. See: https://lightningcss.dev/css-modules.html#css-grid ### Testing Instructions I'll add a test soon. Closes PACK-2731
…urbopack (vercel/turborepo#7711) ### Description Currently the development Turbopack runtime is always used as production builds are not fully implemented yet. This is the first step to having a production-specific runtime by adding a distinction between Development and Production for the runtime. This PR does not implement the production runtime, it only adds the plumbing for adding the production specific runtime. The runtime implementation will follow in a later PR.
…ercel/turborepo#7715) This: - Implements a hook that enables callers like Next.js to perform their own logic (to be used in an upcoming linked PR) when HMR updates fail, such as anonymous function components - Logs an actionable message similar to Next.js+Webpack when updates fail Closes PACK-2726
Whether due to a runtime error or an apply error, it should be reloaded. Closes PACK-2751
### Description When accessing many exports of a barrel file with many `export *` it was a bit inefficient before since it was walking all star exports for every accessed export. After this change it only creates a map of the star exports once and accessing every export only looks up the result in the map. This makes it 4 times more efficient for a file with 140 `export *`. Probably even more for bigger barrel files. ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2747
### Description For the `failsafe_analyze`, make it not failable by not bubbling up the error - the error will be propagated by chunk generation anyway. Closes PACK-2764
### Description <!-- ✍️ Write a short summary of your work. If necessary, include relevant screenshots. --> ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2784
### Description The default for asset prefix is `"/"` and not `""` ### Testing Instructions #62134
### Description * Handle source maps returned webpack loaders * add workaround for Node.js bug to allow it using sectioned source maps * import source map for Source file ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2783
### Description We already return the source map and don't want a source map comment on the generated code. (It's added by the chunk) ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2794
…urbo-tasks (vercel/turborepo#7774) ### Description ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2795
…ack to raw (vercel/turborepo#7785) This causes Turbopack to emit an error-level issue when a module doesn’t have a type set on it. Previously, it fell back to assuming a raw type. Closes PACK-2802
### Description * implement all logger methods * pass logging to rust * show error when using error logging ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2811 --------- Co-authored-by: Will Binns-Smith <wbinnssmith@gmail.com>
### Description * fix `alias` in `getResolve` * Include error message in resolving issue description * avoid emitting unsupported errors when request is aliased ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2820
### Description * add import context `layer`, `supports`, `media` to single css chunks ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2797
### Description The port number is too high and in the automatically chosen range. This could lead to port being already used. ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2798
### Description * add support for recording attributes for spans after creation * In the trace viewer when searching only expand spans that contain the searched value * fix reseting the trace store to avoid accessing invalid indicies * remove old convert-trace tool ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2786
…urborepo#7764) ### Description * disable GC for tasks which use the streaming hack * invalidate dependent tasks when assigning to a recomputing cell ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2787
### Description While debugging https://vercel.slack.com/archives/C03EWR7LGEN/p1711146825831219?thread_ts=1711143217.892349&cid=C03EWR7LGEN, found out sourcemap lookup fails on certain source and always returns synthetictoken only. Weirdly, the guards in this PR https://github.com/vercel/turbo/pull/7823/files#diff-2ce67e28c5b3144ec6f7a89167f3c96da9f9e268abf3fd685ce881d75a4cd8a5L319 is preventing those lookup - removing it makes correct sourcemap lookup occurs. I updated pkg to the version what swc uses and removed + ran next.js tests, seems like most of tests are running just fine (#63624) Still not 100% sure if this'll be ok or not, if not open to change for the correct fixes. Some related fixes getsentry/rust-sourcemap#60 might be the reason we don't see any regressions in the test.
### Description fixes a problem where an async module wrapper is generated for external modules in a "locals" module part when there is usage of the external module in the locals part fixes #63485 ### Testing Instructions <!-- Give a quick description of steps to test your changes. --> Closes PACK-2810 fixes PACK-2805
### Description Fix server action and rename the injected variable. The previous code fails if the name of the top-level variable is `v`. This was not a problem for almost all modules, but some packages like [`@firebase/webchannel-wrapper`](https://unpkg.com/@firebase/webchannel-wrapper@0.2.41/dist/index.esm.js) is published after mangling, so it can be a problem ### Testing Instructions See #66689
### Description Avoids recompiling too often ### Testing Instructions <!-- Give a quick description of steps to test your changes. -->
### Description skips import to modules that are side effect free and have no exports referenced This allows to e. g. skip the `package` import completely in the following code: ``` js import { something } from "package"; // package is marked as side effect free if (falsy) { something(); } ``` This could happen e. g. with code like `if(process.env.NODE_ENV === "production")` ### Testing Instructions <!-- Give a quick description of steps to test your changes. -->
### Description Remove `ModuleType::Mdx` and instead handle mdx files using ```rust ModuleRuleEffect::ModuleType(ModuleType::Typescript { transforms: ts_app_transforms, tsx: true, analyze_types: enable_types, options: ecmascript_options_vc, }), ModuleRuleEffect::SourceTransforms(Vc::cell(vec![Vc::upcast( MdxTransform::new(mdx_transform_options), )])), ``` ### Testing Instructions I ran these Next.js tests, which appear to be all mdx related tests there are ``` mdx with-mdx-rs app directory ✓ should work in initial html (1262 ms) ✓ should work using browser (1460 ms) ✓ should work in initial html with mdx import (145 ms) ✓ should work using browser with mdx import (1179 ms) ✓ should allow overriding components (1163 ms) ✓ should allow importing client components (26 ms) ✓ should work with next/image (424 ms) pages directory ✓ should work in initial html (1091 ms) ✓ should work using browser (1216 ms) ✓ should work in initial html with mdx import (147 ms) ✓ should work using browser with mdx import (1202 ms) ✓ should allow overriding components (1236 ms) ```
dd4ed78
to
c4e331d
Compare
This hides them from Next.js for now I'll add a test case for Next.js as well Closes PACK-2877
Trying vercel/turborepo#8031 again ~~Depends on vercel/turborepo#8766 <img width="979" alt="Bildschirmfoto 2024-07-30 um 15 35 48" src="https://github.com/user-attachments/assets/32d88ffc-c816-41fb-928b-79d27e89b56d"> --------- Co-authored-by: 강동윤 (Donny) <kdy1997.dev@gmail.com>
c4e331d
to
93986d9
Compare
…ercel/turborepo#8873) ### Description Allows to handle `if (unknown && falsy)` `if (unknown || truthy)` `if (unknown ?? nullish)` in static analysis. Hint: review each commit on it's own, since the first commit is only changing to a named struct. ### Testing Instructions <!-- Give a quick description of steps to test your changes. -->
93986d9
to
8a2347b
Compare
…ercel/turborepo#8890) ### Description * tracks early returns and their conditions * replaces unreachable code with `"TURBOPACK unreachable"` * finds hoisted declarations in unreachable code and keeps them ### Testing Instructions <!-- Give a quick description of steps to test your changes. -->
8a2347b
to
ee90487
Compare
ee90487
to
becc655
Compare
### What? Follow up for the previous PR in the stack updating references, lockfiles and snapshots. ### Why? https://vercel.com/blog/turbopack-moving-homes ### How? https://github.com/vercel/next-turbo-merge Closes PACK-3163
sokra
approved these changes
Aug 1, 2024
ForsakenHarmony
added a commit
that referenced
this pull request
Aug 14, 2024
Moves all turbopack crates (and a few js packages) into the next.js repo. https://vercel.com/blog/turbopack-moving-homes `git-filter-repo` https://github.com/newren/git-filter-repo (better than filter-branch) and `git subtree` See: https://github.com/vercel/next-turbo-merge Short version: ```bash git clone git@github.com:vercel/next.js.git --filter=blob:none --no-tags --single-branch --branch canary "$NEXT_DIR" git --git-dir="$NEXT_DIR/.git" remote add --fetch --track main --no-tags turbo git@github.com:vercel/turbo git switch -C hrmny/turbopack-staging turbo/main --no-track git filter-repo --force --refs hrmny/turbopack-staging \ --path crates/ \ --path xtask/ \ --path packages/devlow-bench/ \ --path packages/node-file-trace/ \ --path packages/node-module-trace/ \ --path packages/turbo-pack/ \ --path packages/turbo-tasks/ \ --path packages/turbo-tasks-examples/ \ --path packages/turbo-tasks-fs/ \ --path packages/turbo-tasks-macros/ \ --path packages/turbo-tracing-next-plugin/ \ --path packages/webpack-nmt/ git filter-repo --force --refs hrmny/turbopack-staging \ --path-glob 'crates/turborepo*' --invert-paths git filter-repo --refs hrmny/turbopack-staging --commit-callback ' commit.message = re.sub(b"(^|\\(| )(#\\d\\d\\d\\d)(\\)| |\\r|$)", b"\\1vercel/turbo\\2\\3", commit.message, flags=re.MULTILINE) ' git gc git switch -C hrmny/merge-turbopack origin/canary --no-track git subtree add -P turbopack hrmny/turbopack-staging ``` Closes PACK-3163
ForsakenHarmony
added a commit
that referenced
this pull request
Aug 15, 2024
# Warning: DO NOT squash merge ### What? Moves all turbopack crates (and a few js packages) into the next.js repo. ### Why? https://vercel.com/blog/turbopack-moving-homes ### How? `git-filter-repo` https://github.com/newren/git-filter-repo (better than filter-branch) and `git subtree` See: https://github.com/vercel/next-turbo-merge Short version: ```bash git clone git@github.com:vercel/next.js.git --filter=blob:none --no-tags --single-branch --branch canary "$NEXT_DIR" git --git-dir="$NEXT_DIR/.git" remote add --fetch --track main --no-tags turbo git@github.com:vercel/turbo git switch -C hrmny/turbopack-staging turbo/main --no-track git filter-repo --force --refs hrmny/turbopack-staging \ --path crates/ \ --path xtask/ \ --path packages/devlow-bench/ \ --path packages/node-file-trace/ \ --path packages/node-module-trace/ \ --path packages/turbo-pack/ \ --path packages/turbo-tasks/ \ --path packages/turbo-tasks-examples/ \ --path packages/turbo-tasks-fs/ \ --path packages/turbo-tasks-macros/ \ --path packages/turbo-tracing-next-plugin/ \ --path packages/webpack-nmt/ git filter-repo --force --refs hrmny/turbopack-staging \ --path-glob 'crates/turborepo*' --invert-paths git filter-repo --refs hrmny/turbopack-staging --commit-callback ' commit.message = re.sub(b"(^|\\(| )(#\\d\\d\\d\\d)(\\)| |\\r|$)", b"\\1vercel/turbo\\2\\3", commit.message, flags=re.MULTILINE) ' git gc git switch -C hrmny/merge-turbopack origin/canary --no-track git subtree add -P turbopack hrmny/turbopack-staging ``` Closes PACK-3163 # Conflicts: # .eslintignore # .eslintrc.cli.json # Cargo.lock # Cargo.toml # package.json # packages/next/package.json # pnpm-lock.yaml
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What?
Moves all turbopack crates (and a few js packages) into the next.js repo.
Why?
https://vercel.com/blog/turbopack-moving-homes
How?
git-filter-repo
https://github.com/newren/git-filter-repo (better than filter-branch)and
git subtree
See: https://github.com/vercel/next-turbo-merge
Short version:
Closes PACK-3163