Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically install missing dependencies, part 2 #805

Merged
merged 39 commits into from
Mar 18, 2018
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
866e2b7
Automagically npm install missing dependencies
davidnagli Feb 14, 2018
60e160d
Changed automatic install to use installPackage helper
davidnagli Feb 14, 2018
1cacb59
Merge branch 'master' into master
davidnagli Feb 14, 2018
bcbb07d
Added back yarn.lock check, improved yarn.lock & project root resolution
davidnagli Feb 15, 2018
0e80fe3
Merge remote-tracking branch 'origin/master'
davidnagli Feb 15, 2018
184e6f9
Bug fix
davidnagli Feb 16, 2018
1d30d0c
refactor/cleanup code to fix the issues @brandon93s pointed out
davidnagli Feb 16, 2018
8b98fee
Dev mode check
davidnagli Feb 16, 2018
9f27b5d
Added autoinstall cli flags
davidnagli Feb 17, 2018
2bd5410
Merge remote-tracking branch 'upstream/master'
davidnagli Feb 17, 2018
9308335
Fixed Node 6 Compatiblity Issues
davidnagli Feb 18, 2018
8354b61
Fixed refactoring mistake
davidnagli Feb 21, 2018
b3ba099
Removed recursive config file search
davidnagli Feb 21, 2018
b076b7d
Merge branch 'master' of https://github.com/davidnagli/parcel into da…
devongovett Feb 22, 2018
8a3dc51
autoinstall improvements
devongovett Feb 23, 2018
9680030
Merge branch 'master' of github.com:parcel-bundler/parcel into davidn…
devongovett Mar 6, 2018
6f4a194
Merge branch 'davidnagli-master' into devon-autoinstall
devongovett Mar 6, 2018
949c1df
Removed redundent variable projectRootLocation
davidnagli Mar 6, 2018
b3ba28e
Added back reccursive yarn.lock resolution
davidnagli Mar 6, 2018
63c34f5
CHANGED CONFIG.RESOLVE() TO CHECK THE DIRECORY BEFORE GOING TO PARENT
davidnagli Mar 6, 2018
b26cb28
Changed localrequire to pass in 'basedir' instead of 'path' to install()
davidnagli Mar 6, 2018
11a5c35
Added initial unit tests
davidnagli Mar 6, 2018
9c91de6
Whoops... removed test.only() filter from autoinstall unit tests
davidnagli Mar 6, 2018
9e06479
Merge branch 'master' of https://github.com/davidnagli/parcel
davidnagli Mar 6, 2018
bdb6485
Added support for absolute and tilde paths
davidnagli Mar 6, 2018
2356c2a
Refactored parameters into config object
davidnagli Mar 6, 2018
4cf5d2d
Fixed refactored usages, cleaned up unit tests
davidnagli Mar 7, 2018
06eecdd
Renamed 'config' to 'configObj'
davidnagli Mar 7, 2018
40efbb3
Merge branch 'master' of https://github.com/davidnagli/parcel into de…
devongovett Mar 7, 2018
89a668d
Cleanup / Refactoring - Split code into seperate functions
davidnagli Mar 8, 2018
674d458
Cleaned up unit tests
davidnagli Mar 8, 2018
2030211
Retrigger CI
davidnagli Mar 8, 2018
eb9c263
Merge branch 'master' into master
davidnagli Mar 13, 2018
b32e731
Merge branch 'master' of https://github.com/davidnagli/parcel into de…
devongovett Mar 17, 2018
37c412c
Merge branch 'master' of github.com:parcel-bundler/parcel into devon-…
devongovett Mar 18, 2018
3343f28
Autoinstall improvements
devongovett Mar 18, 2018
6e07e6d
Fix tests
devongovett Mar 18, 2018
c462a4e
Remove explicit package-manager argument for now
devongovett Mar 18, 2018
d2716cd
Fix line counting on windows
devongovett Mar 18, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions src/Bundler.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ const config = require('./utils/config');
const emoji = require('./utils/emoji');
const loadEnv = require('./utils/env');
const PromiseQueue = require('./utils/PromiseQueue');
const installPackage = require('./utils/installPackage');
const bundleReport = require('./utils/bundleReport');
const prettifyTime = require('./utils/prettifyTime');

Expand Down Expand Up @@ -96,7 +97,9 @@ class Bundler extends EventEmitter {
? options.sourceMaps
: !isProduction,
hmrHostname: options.hmrHostname || '',
detailedReport: options.detailedReport || false
detailedReport: options.detailedReport || false,
autoinstall: (options.autoinstall || false) && !isProduction,
packageManager: options.packageManager
};
}

Expand Down Expand Up @@ -332,14 +335,28 @@ class Bundler extends EventEmitter {
let thrown = err;

if (thrown.message.indexOf(`Cannot find module '${dep.name}'`) === 0) {
let isLocalFile = dep.name.startsWith('.');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the dependency be an absolute path, which would still be a local file? Trying to install in that scenario would blow up on npm install.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya you’re right.

I would also have to add support for tilde paths etc now that #850 landed.

Copy link
Member

@devongovett devongovett Feb 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll need something like let isLocalFile = /^[/~.]/.test(dep.name); in order to support the new resolver. See https://github.com/parcel-bundler/parcel/blob/master/src/Resolver.js#L88-L114

// Attempt to install missing npm dependencies
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mind adding tests for this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. I’ve tried a few times but I’m gonna need to wait for #881 to land. Atm the second I add tests which use the test /input directory it causes a bunch of other flaky tests to go crazy.

Copy link
Contributor Author

@davidnagli davidnagli Mar 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done :)

if (!isLocalFile && this.options.autoinstall) {
logger.status(emoji.progress, `Installing ${dep.name}...`);
await installPackage(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be time for a config object to installPackage... 5 parameters!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Path.dirname(asset.name),
[dep.name],
false,
false,
this.options.packageManager
);
return await this.resolveAsset(dep.name, asset.name);
}

if (dep.optional) {
return;
}

thrown.message = `Cannot resolve dependency '${dep.name}'`;

// Add absolute path to the error message if the dependency specifies a relative path
if (dep.name.startsWith('.')) {
if (isLocalFile) {
const absPath = Path.resolve(Path.dirname(asset.name), dep.name);
err.message += ` at '${absPath}'`;
}
Expand Down
12 changes: 12 additions & 0 deletions src/cli.js
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ program
.option('--no-hmr', 'disable hot module replacement')
.option('--no-cache', 'disable the filesystem cache')
.option('--no-source-maps', 'disable sourcemaps')
.option('--no-autoinstall', 'disable autoinstall')
.option(
'-m, --package-manager <packageManager>',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this option really necessary? Our approach to inferring the package manager seems pretty solid.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s true, but I don’t really see any downsides to giving the users manual control if necessary.

One scenario that I would imagine will be particularly problematic is if a user is starting a brand new project with Parcel and has no dependencies yet. They try to add their first import statement, and Parcel automatically uses npm because it doesn’t see a yarn.lock, even though the user would rather use yarn. In that case they would want to use the -m yarn flag to force it to use yarn.

(I mean I suppose they could just touch yarn.lock to generate an empty one, but that’s kinda hacky and just another point for confusion)

I can remove it if you think we should, but I just don’t see any downsides to leaving it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to remove it for now. I think our approach is pretty solid. If someone wants to open a bug with a good reason to add an option for it, they can.

'set the package manger for autoinstall (npm or yarn)',
/^(npm|yarn)$/
)
.option(
'-t, --target [target]',
'set the runtime environment, either "node", "browser" or "electron". defaults to "browser"',
Expand All @@ -65,6 +71,12 @@ program
.option('--no-hmr', 'disable hot module replacement')
.option('--no-cache', 'disable the filesystem cache')
.option('--no-source-maps', 'disable sourcemaps')
.option('--no-autoinstall', 'disable autoinstall')
.option(
'-m, --package-manager <packageManager>',
'set the package manger for autoinstall (npm or yarn)',
/^(npm|yarn)$/
)
.option(
'-t, --target [target]',
'set the runtime environment, either "node", "browser" or "electron". defaults to "browser"',
Expand Down
59 changes: 52 additions & 7 deletions src/utils/installPackage.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,67 @@ const config = require('./config');
const path = require('path');
const promisify = require('./promisify');
const resolve = promisify(require('resolve'));
const commandExists = require('command-exists').sync;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used in an async method. Should use the promise version exposed by command-exists with await.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya that’s exactly what I originally wanted to do that 😃

But... because of the way the command-exists package is written, it’s doesn’t really work out. 😕

Pretty much, commandExists() conveys the result of whether or not it found the command by resolving the returned promise for “package found”, and rejecting the promise for “package not found”. This is a problem if we’re using async/await because await handles promise rejections by throwing. So pretty much whenever yarn isn’t found it’ll throw an error instead of nicely informing us.

I guess it’s possible to do it using a try/catch, but that’s really messy. The only other way to use the async version of command-exists is to just directly use the promise, but that’s really complicated since promise.finally hasn’t landed in Node yet and it would probably require nesting all the rest of the install code.

So I can still use the async version... just it won’t be as clean, and I don’t really think there’s that much of a performance loss from doing it synchronously, especially considering that the function it’s running in is gonna be executing asynchronously anyway.

Your call.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're already inside of a promise chain (return new Promise...) so you could use the Promise API with a bit of refactoring. Alternatively, you could create a hasYarnInstalled method:

hasYarnInstalled = async () => {
	try {
		return await commandExists('yarn');
	} catch (err) {
		return false;
	}
}

if (await hasYarnInstalled() && fs.exists(...)) {
   ...
}

const logger = require('../Logger');
const fs = require('./fs');

async function install(dir, modules, installPeers = true) {
let location = await config.resolve(dir, ['yarn.lock', 'package.json']);
async function install(
dir,
modules,
installPeers = true,
saveDev = true,
packageManager
) {
let projectRootLocation = dir;

return new Promise((resolve, reject) => {
let configFileLocation = await config.resolve(dir, [
'yarn.lock',
'package.json'
]);

if (configFileLocation)
projectRootLocation = path.dirname(configFileLocation);

return new Promise(async (resolve, reject) => {
let install;
let options = {
cwd: location ? path.dirname(location) : dir
cwd: projectRootLocation
};

if (location && path.basename(location) === 'yarn.lock') {
install = spawn('yarn', ['add', ...modules, '--dev'], options);
let packageManagerToUse;
if (packageManager) {
packageManagerToUse = packageManager;
} else {
// If no package manager specified, try to figure out which one to use:
// Default to npm
packageManagerToUse = 'npm';
// If the yarn command exists and we find a yarn.lock, use yarn
if (commandExists('yarn')) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (commandExists('yarn') && await fs.exists(...))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There’s a reason I separated them. I only want the else to apply if await fs.exists(...) is false AND commandExists(‘yarn’) is true.

Pretty much only if the user has yarn, but no yarn.lock, then I want to trigger the warning. If they don’t have yarn, then no need to warn them.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. The warning may be unnecessary. If they haven't explicitly asked for yarn, and there is no yarn.lock, then using npm without a warning seems valid - expected even.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty much the same edge case that I mentioned before applies. If you’re starting a new project, you’d want to know that you need to create a yarn.lock / --yarn and that Parcel isn’t using yarn.

I see your point though... if you want, I can remove it

if (await fs.exists(path.join(projectRootLocation, 'yarn.lock'))) {
packageManagerToUse = 'yarn';
} else {
logger.warn(
"Using NPM instead of Yarn. No 'yarn.lock' found in project directory, use the --package-manager flag to explicitly specify which package manager to use."
);
}
}
}

let commandToUse;
if (packageManagerToUse === 'npm') {
commandToUse = 'install';
} else {
install = spawn('npm', ['install', ...modules, '--save-dev'], options);
commandToUse = 'add';
}

let args = [commandToUse, ...modules];

if (saveDev) {
args.push('-D');
}

install = spawn(packageManagerToUse, args, options);

install.stdout.pipe(process.stdout);
install.stderr.pipe(process.stderr);

Expand Down