Skip to content

feat(core): add native node bindings for hashing #14476

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 54 commits into from
Feb 1, 2023

Conversation

Cammisuli
Copy link
Member

@Cammisuli Cammisuli commented Jan 18, 2023

Current Behavior

Current implementation of our file hashers uses a node based one and a git one. We fallback to the node implementation when a workspace isn't a git workspace (like using other version control systems), and when we do, hashing becomes very slow.

Moreover, using git to do our hashing is very quick, but requires a lot of "hand waving" to get things working well. Also, when there are a lot of changes in the workspace (i.e merging branches, etc) the git hasher becomes slow as well.

Expected Behavior

This PR introduces a native hasher, that is built with Rust. Right now it's feature flagged behind the NX_NATIVE_HASHER env variable.

This allows us to have consistent performance, and have more control over the hashing.

This PR also changes how we publish Nx as we will need to build a native binary for all the platforms we support, which are the following:

"aarch64-apple-darwin",
"aarch64-unknown-linux-gnu",
"aarch64-unknown-linux-musl",
"aarch64-pc-windows-msvc",
"armv7-unknown-linux-gnueabihf",
"x86_64-unknown-linux-musl",
"x86_64-unknown-linux-gnu",
"x86_64-apple-darwin",
"x86_64-pc-windows-msvc"

Related Issue(s)

Fixes #

@vercel
Copy link

vercel bot commented Jan 18, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated
nx-dev ⬜️ Ignored (Inspect) Jan 31, 2023 at 10:06PM (UTC)

@nx-cloud
Copy link

nx-cloud bot commented Jan 18, 2023

☁️ Nx Cloud Report

CI is running/has finished running commands for commit 715293d. As they complete they will appear below. Click to see the status, the terminal output, and the build insights.

📂 See all runs for this branch


✅ Successfully ran 1 target

Sent with 💌 from NxCloud.

@Cammisuli Cammisuli changed the title feat(core): add native-extensions feat(core): add nx-native Jan 19, 2023
@Cammisuli Cammisuli changed the title feat(core): add nx-native feat(core): add native support for nx Jan 20, 2023
@FrozenPandaz FrozenPandaz force-pushed the native_extensions branch 2 times, most recently from ff023e5 to 7997118 Compare January 31, 2023 22:03

let git_folder = workspace_root.clone() + ".git";
// We should make sure to always ignore node_modules
let node_folder = workspace_root.clone() + "node_modules";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to handle nested node_modules directories too.

@FrozenPandaz
Copy link
Collaborator

Don't merge this yet.

@github-actions
Copy link

github-actions bot commented Mar 4, 2023

This pull request has already been merged/closed. If you experience issues related to these changes, please open a new issue referencing this pull request.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 4, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants