Skip to content

Avoid installing dev dependencies when --production - #2944

Closed
juanca wants to merge 6 commits into
yarnpkg:masterfrom
juanca:fix_hoisting_in_production
Closed

juanca wants to merge 6 commits into
yarnpkg:masterfrom
juanca:fix_hoisting_in_production

Conversation

@juanca

@juanca juanca commented Mar 17, 2017

Copy link
Copy Markdown
Contributor

Summary

I'm not sure if this is a substantial feature request. However, I believe the issue is severe and should be addressed immediately.

As of yarn v0.18.0, yarn will sometimes install development dependencies when using the production flag (--production). This PR introduces a failing spec from an existing issue.

Test plan

Test driven development: See Implement failing spec for accidentally hoisting dev dependencies

Halp

I would appreciate some guidance on the files to look at for possible fixes. I've dived a bit and it looks like hoisting might be the reason for installing dev dependencies: when resolving the dependency tree, dev dependencies are still considered -- which is not straightforward -- and used in the install process.

My initial thought is to avoid any dev dependencies and use the lockfile's resolved list of packages (not in dev dependencies) when invoking yarn install --production -- and essentially avoid any type of hoisting.

Comment thread src/cli/commands/install.js Outdated

pushDeps('dependencies', {hint: null, optional: false}, true);
pushDeps('devDependencies', {hint: 'dev', optional: false}, !this.config.production);
if (!this.config.production) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks 11 other tests... However, I question the validity of at least one of the breaking specs:

 FAIL  __tests__/commands/import.js
  ● import missing dev deps in production

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I've seen this import test to fail some times I think


test.concurrent('--production flag ignores dev dependencies', () => {
return runInstall({production: true}, 'install-production', async (config) => {
return runInstall({production: true}, 'install-production-without-dev', async (config) => {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better to add a new test rather than change existing one if the new one covers a different case

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

The first test was too basic. This new integration test covers a more common case with nested dependencies and dev dependencies.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basic tests are good :) If they break, we don't have to debug which part of the test is failing

@bestander

Copy link
Copy Markdown
Member

@juanca, we have merged recently a new fix to hoisting algorithm.
Do you want to rebase and try again?

@juanca

juanca commented Apr 7, 2017

Copy link
Copy Markdown
Contributor Author

Will do! Thanks for the update.

@juanca

juanca commented Apr 7, 2017 •

Copy link
Copy Markdown
Contributor Author

Well, I seem to be breaking less tests with the updated code.

I might just:

  • revert my attempted fix
  • keep basic test
  • keep non-basic test

@juanca

juanca commented Apr 7, 2017

Copy link
Copy Markdown
Contributor Author

Also, when you have a minute of spare time, do you mind linking the hoisting fix PR? I'm interested in knowing more about yarn's hoisting mechanisms.

@bestander

Copy link
Copy Markdown
Member

This PR fixed dev dependency hoisting bug #2921

@juanca

juanca commented Apr 7, 2017

Copy link
Copy Markdown
Contributor Author

I got an intermittent failure. :/ Any way to rebuild from here?

@bestander

Copy link
Copy Markdown
Member

You mean Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.?
It started happening on CI, as long as Travis is green it should be fine.

@bestander

Copy link
Copy Markdown
Member

The test passes, does it mean that your case now works?

@@ -0,0 +1,1219 @@
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is quite a big dependency graph, it would increase the test run time.
Is it possible to get cover the case with a smaller example?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I'll try to come up with something else. Ensure that is fails pre-fix and ensure it is green post-fix.

@juanca

juanca commented Apr 7, 2017

Copy link
Copy Markdown
Contributor Author

Yeah, the fix made my case work!

We will be testing in staging (then production) servers once the fix is released.

I'll try to come up with a smaller / optimized use cases for tests. Most likely going to make extra packages in e2e test repository.

@bestander

Copy link
Copy Markdown
Member

Awesome, thanks for investing in stability

@bestander

Copy link
Copy Markdown
Member

@juanca thanks for the PR.
I'll close it as it can't be merged this way yet.
A PR with a more isolated test will be very much welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants