`yarn install --production` doesn't install correct dependencies

[donovan-graham]

When running yarn install --production it does not install the required dependencies of forever. This seem to be related to havingnodemon in devDependencies.

Error response:

> forever app.js
module.js:457
    throw err;
    ^
Error: Cannot find module 'minimatch'

I've created a test application here:
https://github.com/donovan-graham/yarn-example-app

#  Steps to reproduce error
git clone https://github.com/donovan-graham/yarn-example-app.git
cd yarn-example-app
yarn install --production
npm start

#  temporary step to bypass error
rm -rf node_modules
yarn remove nodemon
yarn install --production
npm start

[sendilkumarn]

@Daniel15 I guess this is because of nodemon is having latest version of minimatch.

The linker function currently takes in both deps and dev deps into it. For argument production, this should be prevented.

Even on normal yarn install with out production argument. Only latest version is installed in the actual path. This has to checked too.

[liorbrauer]

I'm getting a similar problem when running yarn install --production and then trying to run my build using webpack (running yarn install works fine).

> NODE_ENV=production webpack -p --config webpack/production.config.js

module.js:457
    throw err;
    ^

Error: Cannot find module 'graceful-fs'
    at Function.Module._resolveFilename (module.js:455:15)
    at Function.Module._load (module.js:403:25)
    at Module.require (module.js:483:17)
    at require (internal/module.js:20:19)

and if I recall correctly, previous attempts showed similar errors with another package (not only graceful-fs)

[comfroels]

I'm getting very similar as well... yarn install works just fine. but with the --production flag I get this:

> yarn install --production

yarn install v0.15.1
error npm-shrinkwrap.json found. This will not be updated or respected. See [TODO] for more information.
[1/4] Resolving packages...
[2/4] Fetching packages...
warning lodash@2.4.2: The engine "rhino" appears to be invalid.
warning fsevents@1.0.14: The platform "win32" is incompatible with this module.
info "fsevents@1.0.14" is an optional dependency and failed compatibility check. Excluding it from installation.
warning benchmark@1.0.0: The engine "rhino" appears to be invalid.
warning lodash@1.0.2: The engine "rhino" appears to be invalid.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
[1/1] ⠐ node-sass:     at Module.require (module.js:367:17)
[-/1] ⠐ waiting...
[-/1] ⠐ waiting...
[-/1] ⠐ waiting...
error C:\vagrant\ebroker-quoteengine\node_modules\node-sass: Command failed.
Exit code: 1
Command: C:\WINDOWS\system32\cmd.exe
Arguments: /d /s /c node scripts/install.js
Directory: C:\vagrant\ebroker-quoteengine\node_modules\node-sass
Output:
module.js:341
    throw err;
    ^

Error: Cannot find module 'tough-cookie'
    at Function.Module._resolveFilename (module.js:339:15)
    at Function.Module._load (module.js:290:25)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at Object.<anonymous> (C:\vagrant\ebroker-quoteengine\node_modules\node-sass\node_modules\request\lib\cookies.js:3:13)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Module.require (module.js:367:17)
    at SpawnError (C:\Users\nathan.white\AppData\Roaming\npm\node_modules\yarnpkg\lib\errors.js:18:1)
    at ChildProcess.<anonymous> (C:\Users\nathan.white\AppData\Roaming\npm\node_modules\yarnpkg\lib\util\child.js:107:15)
    at emitTwo (events.js:100:13)
    at ChildProcess.emit (events.js:185:7)
    at maybeClose (internal/child_process.js:827:16)
    at Socket.<anonymous> (internal/child_process.js:319:11)
    at emitOne (events.js:90:13)
    at Socket.emit (events.js:182:7)
    at Pipe._onclose (net.js:471:12)

[hybrist]

Can reproduce a similar issue with:

npm init --yes
yarn add --dev nodemon
yarn add gulp
rm -rf node_modules
yarn install --production

This will install is-glob but not its dependency is-extglob:

> yarn why is-glob
yarn why v0.16.0
# ...
info Reasons this module exists
   - "nodemon#chokidar" depends on it
   - "gulp#liftoff#findup-sync" depends on it

> yarn why is-extglob
yarn why v0.16.0
#  ...
info This module exists because "nodemon#chokidar#is-glob" depends on it.

It seems to "forget" the gulp#liftoff dependency path while traversing..?

EDIT: Smaller example:

npm init --yes
yarn add --dev chokidar@1.6.1
yarn add liftoff@2.3.0
rm -rf node_modules
yarn --prod
node -e "require('is-glob')"

Also confirmed that removing devDependencies before running yarn --prod installs the correct dependency tree.

[ArkadyDR]

My software team ran into this issue, specifically with the package prr which is a dependency of both less and pouchdb. Many other packages were also missing from the --production build but prr was the first to cause a failure in our product. This issue has been a showstopper for us as the size of our installer would significantly increase if we included the dev packages, so we've returned to using npm.

[gihrig]

FWIW: I can work around the issue by deleting devDependencies section from package.json prior to running yarn in production.

[royriojas]

as @gihrig said running npm prune --production can remove the devDependencies which helps to workaround this issue.

[tanx]

as @gihrig said running npm prune --production can remove the devDependencies which helps to workaround this issue.

Yarn's main advantage over npm is a deterministic node_modules dir i.e. same in dev, CI, and production out of the box. Does running npm prune --production yield that same behavior?

My current workaround is to just install devDependencies in production as well. Disk is cheap (especially on AWS) and a deterministic install is much more important to me than disk space. So my "workaround" is to just act like yarn --production does not exist right now .

[royriojas]

@tanx npm prune --production just remove the devDependencies. And in my tests always the same modules were removed. On the other hand, yes disk space is cheap, so maybe acting as if yarn --production does not exits is a better workaround :)

[tanx]

@tanx npm prune --production just remove the devDependencies. And in my tests always the same modules were removed.

This is precisely the "works on my machine" mentality described in the Yarn blog post. The problem is you're letting npm change the state of node_modules without yarn's integrity check via theyarn.lock file.

[kylecordes]

Hopefully the workarounds discussed will be rendered moot soon by an yarn update to respect dev vs production deps. In the meantime, indeed there is much to groan about with the "npm prune" post-processing hack.

[hybrist]

The yarn why thing described above is unrelated afaict. It looks to be just a side effect of how the why code is searching for the packages.

[hybrist]

Tried to find a nice way of doing this without adding an additional pass to propagate the visibility after the graph was walked once. Not sure if it would be acceptable to just split out the visibility into a separate step..?

[hybrist]

There's some interesting edge cases as well, it's not just about properly resolving visibility:

• A is an optional dependency of a production dependency
• B is a non-optional dependency of a dev dependency
• C is a non-optional dependency of both

In that case the optional flag of C depends on dev vs. prod. In dev it would be non-optional, in prod it would be optional. Just inheriting the optional flag from one of the parents (or always inheriting it from the prod parent) could lead to weirdness.