npm install doesn't wait for dependencies to be installed #6170

jbuck · 2014-09-11T22:07:46Z

We found a bug over in filerjs/filer#285 where if you have a dependency (bower in this case) listed in your root package.json as well as another dependency's package.json (filer in this case), it's very easy to hit a race condition where the filer postinstall would fail because npm wouldn't wait for bower to be installed in the root.

STR create a package.json with the following content:

{
  "name": "a-poor-app",
  "version": "1.0.0",
  "dependencies": {
    "bower": "1.3.9",
    "filer": "0.0.28"
  }
}

Then run npm install; echo "*************************** noooooooooooooo npm install why aren't you patient **************************"; npm install and receive the following output:

npm WARN package.json a-poor-app@1.0.0 No description
npm WARN package.json a-poor-app@1.0.0 No repository field.
npm WARN package.json a-poor-app@1.0.0 No README data

> filer@0.0.28 postinstall /Users/jbuck/Documents/Github/a-poor-app/node_modules/filer
> bower install

sh: bower: command not found
npm ERR! filer@0.0.28 postinstall: `bower install`
npm ERR! Exit status 127
npm ERR!
npm ERR! Failed at the filer@0.0.28 postinstall script.
npm ERR! This is most likely a problem with the filer package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     bower install
npm ERR! You can get their info via:
npm ERR!     npm owner ls filer
npm ERR! There is likely additional logging output above.

npm ERR! System Darwin 13.3.0
npm ERR! command "node" "/usr/local/bin/npm" "install"
npm ERR! cwd /Users/jbuck/Documents/Github/a-poor-app
npm ERR! node -v v0.10.31
npm ERR! npm -v 1.4.23
npm ERR! code ELIFECYCLE
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /Users/jbuck/Documents/Github/a-poor-app/npm-debug.log
npm ERR! not ok code 0
*************************** noooooooooooooo npm install why aren't you patient **************************
npm WARN package.json a-poor-app@1.0.0 No description
npm WARN package.json a-poor-app@1.0.0 No repository field.
npm WARN package.json a-poor-app@1.0.0 No README data

> filer@0.0.28 postinstall /Users/jbuck/Documents/Github/a-poor-app/node_modules/filer
> bower install

bower mocha#1.17.1              cached git://github.com/visionmedia/mocha.git#1.17.1
bower mocha#1.17.1            validate 1.17.1 against git://github.com/visionmedia/mocha.git#1.17.1
bower chai#1.9.0                cached git://github.com/chaijs/chai.git#1.9.0
bower chai#1.9.0              validate 1.9.0 against git://github.com/chaijs/chai.git#1.9.0
bower chai#1.9.0               install chai#1.9.0
bower mocha#1.17.1             install mocha#1.17.1

chai#1.9.0 bower_components/chai

mocha#1.17.1 bower_components/mocha
filer@0.0.28 node_modules/filer
└── base64-arraybuffer@0.1.2

The text was updated successfully, but these errors were encountered:

othiym23 · 2014-09-15T22:00:58Z

We believe that this issue has been addressed in npm@2.0.0 (backported into npm@1.4.28). Please try one of those two and let us know if it does not address the problem. Thanks!

jbuck · 2014-09-17T16:37:41Z

@othiym23 This error still occurs with npm@1.4.28 and npm@2.0.0:

Jons-MacBook-Pro:test jbuck$ cat package.json
{
  "name": "a-poor-app",
  "version": "1.0.0",
  "dependencies": {
    "bower": "1.3.9",
    "filer": "0.0.28"
  }
}
Jons-MacBook-Pro:test jbuck$ npm install; echo "*************************** noooooooooooooo npm install why aren't you patient **************************"; npm install
npm WARN package.json a-poor-app@1.0.0 No description
npm WARN package.json a-poor-app@1.0.0 No repository field.
npm WARN package.json a-poor-app@1.0.0 No README data

> filer@0.0.28 postinstall /Users/jbuck/Documents/Github/test/node_modules/filer
> bower install

sh: bower: command not found
npm ERR! Darwin 13.3.0
npm ERR! argv "node" "/usr/local/bin/npm" "install"
npm ERR! node v0.10.32
npm ERR! npm  v2.0.0
npm ERR! code ELIFECYCLE

npm ERR! filer@0.0.28 postinstall: `bower install`
npm ERR! Exit status 127
npm ERR!
npm ERR! Failed at the filer@0.0.28 postinstall script.
npm ERR! This is most likely a problem with the filer package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     bower install
npm ERR! You can get their info via:
npm ERR!     npm owner ls filer
npm ERR! There is likely additional logging output above.
*************************** noooooooooooooo npm install why aren't you patient **************************
npm WARN package.json a-poor-app@1.0.0 No description
npm WARN package.json a-poor-app@1.0.0 No repository field.
npm WARN package.json a-poor-app@1.0.0 No README data

> filer@0.0.28 postinstall /Users/jbuck/Documents/Github/test/node_modules/filer
> bower install

bower mocha#1.17.1              cached git://github.com/visionmedia/mocha.git#1.17.1
bower mocha#1.17.1            validate 1.17.1 against git://github.com/visionmedia/mocha.git#1.17.1
bower chai#1.9.0                cached git://github.com/chaijs/chai.git#1.9.0
bower chai#1.9.0              validate 1.9.0 against git://github.com/chaijs/chai.git#1.9.0
bower chai#1.9.0               install chai#1.9.0
bower mocha#1.17.1             install mocha#1.17.1

chai#1.9.0 bower_components/chai

mocha#1.17.1 bower_components/mocha


┌───────────────────────────────────────────┐
│ Update available: 1.3.10 (current: 1.3.9) │
│ Run npm update -g bower to update.        │
└───────────────────────────────────────────┘

filer@0.0.28 node_modules/filer
└── base64-arraybuffer@0.1.2
Jons-MacBook-Pro:test jbuck$
Jons-MacBook-Pro:test jbuck$ npm install; echo "*************************** noooooooooooooo npm install why aren't you patient **************************"; npm install
npm WARN package.json a-poor-app@1.0.0 No description
npm WARN package.json a-poor-app@1.0.0 No repository field.
npm WARN package.json a-poor-app@1.0.0 No README data

> filer@0.0.28 postinstall /Users/jbuck/Documents/Github/test/node_modules/filer
> bower install

sh: bower: command not found
npm ERR! filer@0.0.28 postinstall: `bower install`
npm ERR! Exit status 127
npm ERR!
npm ERR! Failed at the filer@0.0.28 postinstall script.
npm ERR! This is most likely a problem with the filer package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     bower install
npm ERR! You can get their info via:
npm ERR!     npm owner ls filer
npm ERR! There is likely additional logging output above.

npm ERR! System Darwin 13.3.0
npm ERR! command "node" "/usr/local/bin/npm" "install"
npm ERR! cwd /Users/jbuck/Documents/Github/test
npm ERR! node -v v0.10.32
npm ERR! npm -v 1.4.28
npm ERR! code ELIFECYCLE
npm ERR! not ok code 0
*************************** noooooooooooooo npm install why aren't you patient **************************
npm WARN package.json a-poor-app@1.0.0 No description
npm WARN package.json a-poor-app@1.0.0 No repository field.
npm WARN package.json a-poor-app@1.0.0 No README data
\
> filer@0.0.28 postinstall /Users/jbuck/Documents/Github/test/node_modules/filer
> bower install

bower chai#1.9.0                cached git://github.com/chaijs/chai.git#1.9.0
bower chai#1.9.0              validate 1.9.0 against git://github.com/chaijs/chai.git#1.9.0
bower mocha#1.17.1              cached git://github.com/visionmedia/mocha.git#1.17.1
bower mocha#1.17.1            validate 1.17.1 against git://github.com/visionmedia/mocha.git#1.17.1
bower chai#1.9.0               install chai#1.9.0
bower mocha#1.17.1             install mocha#1.17.1

chai#1.9.0 bower_components/chai

mocha#1.17.1 bower_components/mocha
filer@0.0.28 node_modules/filer
└── base64-arraybuffer@0.1.2

othiym23 · 2014-09-17T16:48:49Z

Sorry for the hair trigger, @jbuck – we've been cleaning out a bunch of issues related #6043 and press-ganged yours prematurely.

I'm actually not sure what the right thing to do in your case is. Is there a reason you're depending on bower directly? (I'm assuming that there is, and you're using a reduced package.json here.) If you just install filer alone, it installs without issue because it installs bower as its own dependency. It seems like filer should have a peerDependency relationship with bower (and probably grunt), because it does depend on them being installed for its scripts to run. I think this is probably a bug in filer's packaging, but this is also relevant to the work that @iarna is doing on multi-stage install, so I'm sending it her way as well.

jbuck · 2014-09-18T16:07:06Z

Sorry, I left one part out of my example. a-poor-app and filer both run bower install in their postinstall scripts.

I'm fine with putting bower in as a peerDependency to ensure that it's installed before the postinstall script is run, but I think that should be documented somewhere as a valid use case for peerDependency. I guess it's just weird how the dependency chain is changed when you run npm install in a-poor-app vs filer.

othiym23 · 2014-09-20T21:10:37Z

It's definitely weird, and it's something that npm should handle better than it does. I'm going to add this to the multi-stage install milestone, because one of the things that @iarna is hoping to address with that work is making the whole interaction between installation and lifecycle scripts simpler and more deterministic.

chrisskilton · 2014-11-27T13:25:45Z

Great thread. has there been any movement on this, or is the peerDependency solution the way to go?

othiym23 · 2014-12-01T22:40:45Z

@drvanhalen The behavior @jbuck is seeing is a bug, but my hope is that multi-stage install / npm@3 will fix this as a byproduct of it being more or less a rewrite of the installer process, and that will solve the problem. It may or may not get fixed prior to the release of npm@3 (which, to be clear, is in the relatively near future).

chrisskilton · 2014-12-02T09:25:54Z

@othiym23 thanks very much.

illegalnumbers · 2014-12-02T17:42:51Z

+1

iarna · 2014-12-15T10:31:38Z

This'll be solved by #6926, so I'm going to close this ticket and further discussion should occur over there.

othiym23 closed this as completed Sep 15, 2014

othiym23 reopened this Sep 17, 2014

othiym23 added bug peerdependencies labels Sep 17, 2014

othiym23 added this to the multi-stage install milestone Sep 20, 2014

iarna closed this as completed Dec 15, 2014

iarna mentioned this issue Dec 15, 2014

Don't run "install"/"postinstall" lifecycle scripts until after dependencies are installed #6926

Closed

npm locked and limited conversation to collaborators Jun 24, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

npm install doesn't wait for dependencies to be installed #6170

npm install doesn't wait for dependencies to be installed #6170

jbuck commented Sep 11, 2014

othiym23 commented Sep 15, 2014

jbuck commented Sep 17, 2014

othiym23 commented Sep 17, 2014

jbuck commented Sep 18, 2014

othiym23 commented Sep 20, 2014

chrisskilton commented Nov 27, 2014

othiym23 commented Dec 1, 2014

chrisskilton commented Dec 2, 2014

illegalnumbers commented Dec 2, 2014

iarna commented Dec 15, 2014

npm install doesn't wait for dependencies to be installed #6170

npm install doesn't wait for dependencies to be installed #6170

Comments

jbuck commented Sep 11, 2014

othiym23 commented Sep 15, 2014

jbuck commented Sep 17, 2014

othiym23 commented Sep 17, 2014

jbuck commented Sep 18, 2014

othiym23 commented Sep 20, 2014

chrisskilton commented Nov 27, 2014

othiym23 commented Dec 1, 2014

chrisskilton commented Dec 2, 2014

illegalnumbers commented Dec 2, 2014

iarna commented Dec 15, 2014