From 1f37abc435a9c7aa690a64c1e08a53f701032025 Mon Sep 17 00:00:00 2001 From: Rebecca Turner Date: Thu, 26 Apr 2018 14:23:36 -0700 Subject: [PATCH 1/5] rfc: new npm link command --- accepted/0003-npm-link-changes.md | 192 ++++++++++++++++++++++++++++++ 1 file changed, 192 insertions(+) create mode 100644 accepted/0003-npm-link-changes.md diff --git a/accepted/0003-npm-link-changes.md b/accepted/0003-npm-link-changes.md new file mode 100644 index 000000000..027b42a1a --- /dev/null +++ b/accepted/0003-npm-link-changes.md @@ -0,0 +1,192 @@ +# Refreshing the `npm link` command + +## Summary + +Remove `npm install --link`. Change `npm link` behavior to directly create symlinks for dependencies that already exist and record this fact to the lock-file. Make `npm unlink` stop being an alias for `npm rm` and instead undo what the new `npm link` does, restoring a non-linked copy of the dependency. + +Note: This is a breaking change due to the all new `npm link`, the splitting of `npm unlink` from `npm rm` and the removal of `npm install --link`. + +## Motivation + +The introduction of lock files and a synchronization workflow in `npm@5` broke certain assumptions that the `npm link` and `npm install --link` commands took advantage of. They expected that they could replace a real module with a symlink to one of the same version and `npm install` would leave their symlink alone. + +Starting with `npm@5`, any symlink is expected to be recorded in the `package.json` or lock-file as a `file:` type dependency. + +The result is that any run of `npm install` after creating a symlink with `npm link` or `npm install --link` when you have a lock-file results in the symlink being removed and a copy of the module installed in the usual way. + +## Detailed Explanation + +### USE CASES FOR DEVELOPMENT ONLY LINKING + +When fixing a bug in a dependency or transitive dependency, it is often desirable to be able to test it directly in the project that consumes it. With `npm link` you can create a symlink to the under-development dependency while continuing to otherwise work with your module normally. By persisting this to your lock-file it allows this decision to be branch specific. + +When working with a project with a number of tightly coupled dependencies it is often desirable to work on all of them simultaneously. `npm link` provides a facility for symlinking them and keeping them symlinked and the lock-file persistence means that new member of the project can get this configuration with little ceremony. + +### PACKAGE-LOCK + +lock-file files have a new lockfile property on dependency objects: `link` + +The value of `link` is a path relative to the lock-file. + +### CHANGES TO COMMANDS + +#### `npm link` + +With no subcommand, it is an alias for `npm link create`. With an invalid subcommand the error message should note the change and behavior and point the user at `npm help link`. + +#### `npm link create` + +Create all development links referenced in the lock-file. Failures due to missing link sources or missing package data (or invalid package data) are WARNINGs that result in no link being created. Version mismatches in linked modules are also warnings but they still create a link. + +With no development links in lock-file warn with usage, but exit without error code. + +#### `npm link create module-name[@version-spec]` + +Search the lock-file for modules matching `module-name` and optionally `version-spec`, create links for those. If no modules match, exit with an error. + +#### `npm link clear` + +Remove all development links from your `node_modules`. This does not modify your lock-file, it just undoes what `npm link create` does. + +#### `npm link clear module-name[@version-spec]` + +Remove development links for modules matching the `module-name` and if provided the `version-spec`. + +#### `npm link add /path/to/module-name` +#### `npm link add /path/to/module-name version-spec` + +Create a new development link in your project for the module contained in `/path/to/module-name`. If a version specifier is included (eg `^1.0.0`) then only modules that match it are replaced with a link. If no version specifier is included then ALL modules with the same name are replaced. + +Links are saved in new `link` field on the dependency record they replace in your lock-file. The dependencies in the lock-file do not take into account any development links. Linked dependencies have their own separate dependency tree installed inside their own `node_modules` and their own lock-file. + +#### `npm link remove module-name` +#### `npm link remove /path/to/module-name` + +(Aliases: `npm link rm`, `npm unlink`) + +Remove an existing development link from your project. This does not remove `module-name`—it removes the `link` field from the dependency record and replaces any symlinks to `module-name` with a normally installed copy of it. + +#### `npm install` + +`npm install --link` is no longer be a valid option. + +`npm install` leaves any development links created by `npm link create` in place. + +`npm install module-name@latest` where `module-name` is currently a development link, will update your `package.json` and lock-file but will not do anything on disk. It warns that the link was left in place. + +#### `npm outdated` +#### `npm update` + +Outdated and update run against your lock-file and thus use the unlinked versions when determining what needs doing. Because `npm update` is built on top of `npm install` it gets the "don't clobber links" behavior automatically. + +#### `npm ls` + +Listing your install tree reports development linked dependencies as fulfilling their requirements. In the case where the version of the development linked dependency doesn't match the version in your `package.json` they are flagged visually. + +#### `npm pack` and `npm publish` + +`npm pack` and `npm publish` fail if any bundled dependencies are currently installed as a development link. + +#### `npm unlink` + +Is an alias for `npm link clear`. (Previously this was an alias for `npm rm`.) + +### CAVEATS + +This specification of the new `npm link` command provides no facility for replacing a specific transitive instance of a dependency. Concretely: + +If you depend on `module-a` and `module-b` and both of them depend on the same version of `module-shared`, there is NO facility for linking `module-a`'s `module-shared` without ALSO linking `module-b`'s `module-shared`. It SHOULD be possible to build a stand alone tool to support this, but it's out of scope for the initial `npm link` implementation. + +## Rationale and Alternatives + +Alternative options: + +1. Not do anything. This means `npm link` and `npm install --link` with a lock-file are not useful as links will immediately be erased. +2. Leave all links alone. This would restore previous behavior around `npm link` and `npm install --link` but would make it impossible for npm to do the right thing if a `package.json` dependency changed from a `file:` specifier to a registry one as that would be indistinguishable from a link created with `npm link`. + +The second option is appealing for existing users, but introduces an inconsistency with how npm models it's use. npm views its job as synchronizing your `node_modules` with your `package.json` and lock-file. + +## Implementation + +Broken out by command: + +### `npm link` +### `npm unlink` + +All new commands that inherit from the `Installer` class. They do the normal load of the existing tree and lock-file and then mutate that to include their new kinds of links. They rely on changes to `npm install` to propagate their changes to the user's `node_modules` and lock-file. + +### `npm outdated` +### `npm update` + +Likely these will not require changes, as the changes to the resolver will make the implementation transparent. + +### `npm ls` + +Currently `file:` type specifiers are displayed as: + +``` +name@version -> /path/to/thing +``` + +They will be changed to display as: + +``` +name@file:/path/to/thing +``` + +Dev linked items will be displayed as: + +``` +name@version -> /path/to/thing +``` + +Where `version` is the version from the lock-file that would have been used without the link. + +### `npm pack` & `npm publish` + +Will scan the tree after reading it to look for any dev linked bundled items. If any are found they will abort with an error. + +### The resolver: `install/deps.js` + +Has three new modes of operation when selecting new dependencies to resolve (via `loadDeps` or `loadDevDeps`) and matching existing real deps to requested deps (`doesChildVersionMatch`). + +1. Ignore the `link` lock-file property and only match and install the `package.json` and `lock-file` specifiers. This is the current behavior. +2. Match existing dependencies based on the `link` lock-file property OR the specifiers, but when installing missing dependencies use the specifier. This is the new `npm install` behavior. +3. Match existing dependencies based on the `link` lock-file property, and when installing missing dependencies use the `link` lock-file property. This is the `npm link create` behavior. + + +### `npm link create` + +A subclass of the `Installer` class that uses the third mode for the resolver as discussed above. + +### `npm link create module-name[@version-spec]` + +Not quite the same flow as `npm i module-name[@version-spec]`. The `install` variant removes any existing copy of `module-name` then installs a new one. By contrast `create link` needs to request that just that dependency be made available via `link` metadata, without mutating the lock-file. + +### `npm link clear` +### `npm link clear module-name[@version-spec]` + +A subclass of the `Installer` class that uses the first mode for the resolver as discussed above. + +### `npm link add /path/to/module-name` +### `npm link add /path/to/module-name version-spec` + +As with `npm link create` but edits the lock-file prior to resolving the ideal tree. + +### `npm link remove module-name` + +As with `npm link clear` but edits the lock-file prior to resolving the ideal tree. + +## Prior Art + +This provides development time only linking similarly to _workspaces_. The primary difference is that _workspaces_ are built for use with monorepos and `npm link` is not. We intend to implement _workspaces_ separately in the future. + +## Out of scope + +Previous discussion of this RFC discussed the needs of some users to not prune extraneous dependencies of any kind. Supporting that use case is out of scope for this RFC and should be addressed separately. + +## Open Discussion + +How to integrate this with the use case raised by @giltayar: + +> Ephemeral style, where you want to `npm link` elsewhere, just for trying things out. This can be to a package that resides in a totally different git repo. This is where you must _not_ persist this link in `package-lock.json` as it is relevant only for that certain developer on that certain computer, and therefore should not be commited to source control. This is also relevant for people not using `package-lock.json`. From 6cd6640a5c2dda38d8924aee5c18736de3d54fb4 Mon Sep 17 00:00:00 2001 From: Rebecca Turner Date: Thu, 31 Jan 2019 14:21:20 -0800 Subject: [PATCH 2/5] npm-link: Revise for non-lock-file storage --- accepted/0003-npm-link-changes.md | 73 +++++++++---------------------- 1 file changed, 21 insertions(+), 52 deletions(-) diff --git a/accepted/0003-npm-link-changes.md b/accepted/0003-npm-link-changes.md index 027b42a1a..0fef32a02 100644 --- a/accepted/0003-npm-link-changes.md +++ b/accepted/0003-npm-link-changes.md @@ -2,7 +2,7 @@ ## Summary -Remove `npm install --link`. Change `npm link` behavior to directly create symlinks for dependencies that already exist and record this fact to the lock-file. Make `npm unlink` stop being an alias for `npm rm` and instead undo what the new `npm link` does, restoring a non-linked copy of the dependency. +Remove `npm install --link`. Change `npm link` behavior to directly create symlinks for dependencies that already exist and record this fact into a local project file. Make `npm unlink` stop being an alias for `npm rm` and instead undo what the new `npm link` does, restoring a non-linked copy of the dependency. Note: This is a breaking change due to the all new `npm link`, the splitting of `npm unlink` from `npm rm` and the removal of `npm install --link`. @@ -18,46 +18,28 @@ The result is that any run of `npm install` after creating a symlink with `npm l ### USE CASES FOR DEVELOPMENT ONLY LINKING -When fixing a bug in a dependency or transitive dependency, it is often desirable to be able to test it directly in the project that consumes it. With `npm link` you can create a symlink to the under-development dependency while continuing to otherwise work with your module normally. By persisting this to your lock-file it allows this decision to be branch specific. +When fixing a bug in a dependency or transitive dependency, it is often desirable to be able to test it directly in the project that consumes it. With `npm link` you can create a symlink to the under-development dependency while continuing to otherwise work with your module normally. -When working with a project with a number of tightly coupled dependencies it is often desirable to work on all of them simultaneously. `npm link` provides a facility for symlinking them and keeping them symlinked and the lock-file persistence means that new member of the project can get this configuration with little ceremony. +When working with a project with a number of tightly coupled dependencies it is often desirable to work on all of them simultaneously. `npm link` provides a facility for symlinking them and keeping them symlinked. -### PACKAGE-LOCK +### NEW FILE & LINK PERSISTENCE -lock-file files have a new lockfile property on dependency objects: `link` +We will introduce a new file for tracking local install specific metadata with `node_modules/.dependency-info.json`. This file is not intended to be committed. -The value of `link` is a path relative to the lock-file. +Links will be recorded in a `links` property that is an object mapping package names to link paths. ### CHANGES TO COMMANDS #### `npm link` -With no subcommand, it is an alias for `npm link create`. With an invalid subcommand the error message should note the change and behavior and point the user at `npm help link`. - -#### `npm link create` - -Create all development links referenced in the lock-file. Failures due to missing link sources or missing package data (or invalid package data) are WARNINGs that result in no link being created. Version mismatches in linked modules are also warnings but they still create a link. - -With no development links in lock-file warn with usage, but exit without error code. - -#### `npm link create module-name[@version-spec]` - -Search the lock-file for modules matching `module-name` and optionally `version-spec`, create links for those. If no modules match, exit with an error. - -#### `npm link clear` - -Remove all development links from your `node_modules`. This does not modify your lock-file, it just undoes what `npm link create` does. - -#### `npm link clear module-name[@version-spec]` - -Remove development links for modules matching the `module-name` and if provided the `version-spec`. +With no subcommand or an invalid subcommand, display help for the new subcommands. #### `npm link add /path/to/module-name` #### `npm link add /path/to/module-name version-spec` Create a new development link in your project for the module contained in `/path/to/module-name`. If a version specifier is included (eg `^1.0.0`) then only modules that match it are replaced with a link. If no version specifier is included then ALL modules with the same name are replaced. -Links are saved in new `link` field on the dependency record they replace in your lock-file. The dependencies in the lock-file do not take into account any development links. Linked dependencies have their own separate dependency tree installed inside their own `node_modules` and their own lock-file. +Links are saved in the `links` field in the dependency-info file. Linked dependencies have their own separate dependency tree installed inside their own `node_modules` and their own lock-file. #### `npm link remove module-name` #### `npm link remove /path/to/module-name` @@ -66,11 +48,13 @@ Links are saved in new `link` field on the dependency record they replace in you Remove an existing development link from your project. This does not remove `module-name`—it removes the `link` field from the dependency record and replaces any symlinks to `module-name` with a normally installed copy of it. +If no further links remain in the dependency-info file and no other information is contained in it, it should be unlinked. + #### `npm install` `npm install --link` is no longer be a valid option. -`npm install` leaves any development links created by `npm link create` in place. +`npm install` creates an links specified in the dependency-info file, along side the usual install results. `npm install module-name@latest` where `module-name` is currently a development link, will update your `package.json` and lock-file but will not do anything on disk. It warns that the link was left in place. @@ -103,6 +87,7 @@ Alternative options: 1. Not do anything. This means `npm link` and `npm install --link` with a lock-file are not useful as links will immediately be erased. 2. Leave all links alone. This would restore previous behavior around `npm link` and `npm install --link` but would make it impossible for npm to do the right thing if a `package.json` dependency changed from a `file:` specifier to a registry one as that would be indistinguishable from a link created with `npm link`. +3. Save links to the package-lock to persist them between users. The second option is appealing for existing users, but introduces an inconsistency with how npm models it's use. npm views its job as synchronizing your `node_modules` with your `package.json` and lock-file. @@ -148,34 +133,24 @@ Will scan the tree after reading it to look for any dev linked bundled items. If ### The resolver: `install/deps.js` -Has three new modes of operation when selecting new dependencies to resolve (via `loadDeps` or `loadDevDeps`) and matching existing real deps to requested deps (`doesChildVersionMatch`). +Changes the selection of new dependencies to resolve (via `loadDeps` or `loadDevDeps`) and matching existing real deps to requested deps (`doesChildVersionMatch`): -1. Ignore the `link` lock-file property and only match and install the `package.json` and `lock-file` specifiers. This is the current behavior. -2. Match existing dependencies based on the `link` lock-file property OR the specifiers, but when installing missing dependencies use the specifier. This is the new `npm install` behavior. -3. Match existing dependencies based on the `link` lock-file property, and when installing missing dependencies use the `link` lock-file property. This is the `npm link create` behavior. +Match existing dependencies based on the `links` dependency-info property, and when installing missing dependencies use the `links` dependency-info property. +### `npm link add /path/to/module-name` +### `npm link add /path/to/module-name version-spec` -### `npm link create` +A subclass of the `Installer` class that edits the dependency-info links property before doing ideal tree resolution. -A subclass of the `Installer` class that uses the third mode for the resolver as discussed above. +### `npm link remove module-name` -### `npm link create module-name[@version-spec]` +Alias: `npm link rm` -Not quite the same flow as `npm i module-name[@version-spec]`. The `install` variant removes any existing copy of `module-name` then installs a new one. By contrast `create link` needs to request that just that dependency be made available via `link` metadata, without mutating the lock-file. +As with `npm link add`. If no links remain and no other data is in the dependency-info file then it should be removed. ### `npm link clear` -### `npm link clear module-name[@version-spec]` - -A subclass of the `Installer` class that uses the first mode for the resolver as discussed above. - -### `npm link add /path/to/module-name` -### `npm link add /path/to/module-name version-spec` -As with `npm link create` but edits the lock-file prior to resolving the ideal tree. - -### `npm link remove module-name` - -As with `npm link clear` but edits the lock-file prior to resolving the ideal tree. +As with `npm link remove` but removes ALL links. ## Prior Art @@ -184,9 +159,3 @@ This provides development time only linking similarly to _workspaces_. The prima ## Out of scope Previous discussion of this RFC discussed the needs of some users to not prune extraneous dependencies of any kind. Supporting that use case is out of scope for this RFC and should be addressed separately. - -## Open Discussion - -How to integrate this with the use case raised by @giltayar: - -> Ephemeral style, where you want to `npm link` elsewhere, just for trying things out. This can be to a package that resides in a totally different git repo. This is where you must _not_ persist this link in `package-lock.json` as it is relevant only for that certain developer on that certain computer, and therefore should not be commited to source control. This is also relevant for people not using `package-lock.json`. From 7faab6e0770cbde154454c5663739a3cb448193b Mon Sep 17 00:00:00 2001 From: Rebecca Turner Date: Mon, 4 Feb 2019 12:00:51 -0800 Subject: [PATCH 3/5] Get rid of link subcommands --- accepted/0003-npm-link-changes.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/accepted/0003-npm-link-changes.md b/accepted/0003-npm-link-changes.md index 0fef32a02..8be1ee891 100644 --- a/accepted/0003-npm-link-changes.md +++ b/accepted/0003-npm-link-changes.md @@ -30,26 +30,29 @@ Links will be recorded in a `links` property that is an object mapping package n ### CHANGES TO COMMANDS -#### `npm link` +#### `npm link /path/to/module-name` +#### `npm link /path/to/module-name [module-name@]version-spec` -With no subcommand or an invalid subcommand, display help for the new subcommands. +(Aliases: `npm ln`) -#### `npm link add /path/to/module-name` -#### `npm link add /path/to/module-name version-spec` +With no arguments or `.`, display usage. Create a new development link in your project for the module contained in `/path/to/module-name`. If a version specifier is included (eg `^1.0.0`) then only modules that match it are replaced with a link. If no version specifier is included then ALL modules with the same name are replaced. Links are saved in the `links` field in the dependency-info file. Linked dependencies have their own separate dependency tree installed inside their own `node_modules` and their own lock-file. -#### `npm link remove module-name` -#### `npm link remove /path/to/module-name` +#### `npm unlink` +#### `npm unlink module-name[@verson-spec]` +#### `npm unlink /path/to/module-name` -(Aliases: `npm link rm`, `npm unlink`) +Without arguments, removes all links. Remove an existing development link from your project. This does not remove `module-name`—it removes the `link` field from the dependency record and replaces any symlinks to `module-name` with a normally installed copy of it. If no further links remain in the dependency-info file and no other information is contained in it, it should be unlinked. +Removal should indicate how to recreate the link should the user wish to do so. + #### `npm install` `npm install --link` is no longer be a valid option. From 06ec091a1653d3cee7f9d2180d0cbcb96a70da1c Mon Sep 17 00:00:00 2001 From: Rebecca Turner Date: Thu, 7 Feb 2019 13:52:12 -0800 Subject: [PATCH 4/5] Update last remaining subcommands --- accepted/0003-npm-link-changes.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/accepted/0003-npm-link-changes.md b/accepted/0003-npm-link-changes.md index 8be1ee891..e8294706c 100644 --- a/accepted/0003-npm-link-changes.md +++ b/accepted/0003-npm-link-changes.md @@ -140,20 +140,17 @@ Changes the selection of new dependencies to resolve (via `loadDeps` or `loadDe Match existing dependencies based on the `links` dependency-info property, and when installing missing dependencies use the `links` dependency-info property. -### `npm link add /path/to/module-name` -### `npm link add /path/to/module-name version-spec` +### `npm link /path/to/module-name` +### `npm link /path/to/module-name [name@]version-spec` A subclass of the `Installer` class that edits the dependency-info links property before doing ideal tree resolution. -### `npm link remove module-name` +### `npm unlink [module-name[@version-spec]]` -Alias: `npm link rm` +As with `npm link`. If no links remain and no other data is in the dependency-info file then it should be removed. -As with `npm link add`. If no links remain and no other data is in the dependency-info file then it should be removed. +With no arguments removes ALL the links. -### `npm link clear` - -As with `npm link remove` but removes ALL links. ## Prior Art From 457047a2adfd0513082705c131708b66a4568b62 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kat=20March=C3=A1n?= Date: Thu, 7 Feb 2019 13:53:14 -0800 Subject: [PATCH 5/5] Rename 0003-npm-link-changes.md to 0011-npm-link-changes.md --- accepted/{0003-npm-link-changes.md => 0011-npm-link-changes.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename accepted/{0003-npm-link-changes.md => 0011-npm-link-changes.md} (100%) diff --git a/accepted/0003-npm-link-changes.md b/accepted/0011-npm-link-changes.md similarity index 100% rename from accepted/0003-npm-link-changes.md rename to accepted/0011-npm-link-changes.md