Repository navigation
[BUG] npm install deletes an existing symlinked node_modules directory #3669
Description
Is there an existing issue for this?
- I have searched the existing issues
Current Behavior
If a package contains a node_modules that is symlinked then an npm install will issue the message
npm WARN reify Removing non-directory ....
and delete the node_modules directory.
Expected Behavior
A symlinked node_modules directory should be used as is and not recreated - many people symlink node_modules to a local filesystem so that it does not get synced to cloud storage (like Dropbox).
I suspect that the npm code is just checking if any existing file object named node_modules is a directory and does not also check to see if the non-directory file object is actually a symlinked directory.
Steps To Reproduce
cd /tmp
mkdir symlinked
mkdir z
cd z
npm init -y
ln -s /tmp/symlinked node_modules .
npm install lodash
Environment
- OS: Ubuntu 20.04.1
- Node: 16.7
- npm: 7.21.0
Reacted by Vicary, Da CodeKid, Bruno Gomes, combondrn, Alvin Ramskogler, Joshua Barnett, htorohn, Girish Ramakrishnan, Oleg Kalachev, Eric Esquibel and 11 more
Activity
vicary commentedon Oct 7, 2021
This behavior is introduced between npm 7.15 and 7.21.
The new npm install behavior is to always replace a symlinked node_modules with an actual directory, which breaks the usage of node_modules.nosync and a few existing npm modules that favors this behavior, e.g. nosync-icloud
If this breaking change is intended, it should be introduced with a major version upgrade, shouldn’t it?
Redirected to here from a long trip,
Commenting in the hope for Google to index this and aid future lost souls to land here.
dejanceltra commentedon Oct 7, 2021
This still works in
7.20.0, so it had to be introduced in later versions.This also breaks when developing using Docker, where code is mounted from the host machine, but
node_modulesis symlinked to a different location, to improve performance.vicary commentedon Oct 7, 2021
Confirming a new lower bound of
7.20.6, the lower bound in my previous comment comes from a local cached version of the binary which is incorrect.dejanceltra commentedon Oct 7, 2021
Seems like this was added with
7.21.0with upgrade ofnpm/arborist: npm/arborist@f2b0ceeRelated: https://github.com/npm/arborist/issues/328
vicary commentedon Oct 7, 2021
From the commit message it reads,
Sounds like an intentional incompatibility with
yarnexisting tools in the ecosystem.Although skippable with
--force, it piggybacks a lot of useful checks under the same flag.In the spirit of semantic versioning it should be at least a major upgrade so the community can be prepared, shouldn't it?
dejanceltra commentedon Oct 7, 2021
It works with
--force, but this flags also does other things, which might not be desirable:isaacs commentedon Oct 7, 2021
This is by design, and prevents a class of RCE security exploits. GHSA-gmw6-94gg-2rc2
MartinMystikJonas commentedon Dec 17, 2021
Could there option to keep node_modules as symlink? It basically broke our CI becuse we used node_modules symlinked to shared folder to optimize build - now build that took 2-3 minutes takes about an hour and CI fails on timeout. Reason is it again and again needs to install/copy node_modules in each job.
blaumeise20 commentedon Jan 20, 2022
Please add an option to disable this! I used node_modules.nosync with iCloud, and the folder has a size of 300 MB. This uses all of my iCloud storage.
Using
--forcedoesn't work either, so I can't really use my project until this is fixed.htorohn commentedon Feb 3, 2022
Has somebody find a solution or a workaround about this issue? I'm having the same issue and had to downgrade my node version on the server, but it is not the ideal solution.
vicary commentedon Feb 4, 2022
@htorohn This is not about node, this is an npm design flaw. You may either downgrade, or switch away, from npm.
8 remaining items
ljharb commentedon Mar 25, 2022
Seems like the fix is to avoid a publicly known security attack by not trying to do this unwise idea in the first place.
mk-pmb commentedon Mar 25, 2022
@ljharb That part was explained in the comment above the close. I understand how it might be dangerous if npm, and thus potential scripts, can modify or replace the symlink during install. The remaining mystery is why we need to defend against symlinks that npm cannot modify, due to lack of permission or lack of filesystem capability.
ljharb commentedon Mar 25, 2022
@mk-pmb npm's inability to modify them might not be permanent, and such a pattern in one system might end up in another system where those guarantees don't exist.
mk-pmb commentedon Mar 25, 2022
I see a common pattern: Security defenses that are usually useful, except in edge cases where external safeguards prevent the attack. There should thus be an option to give npm a list of vulnerability IDs (CVEs etc.) that it does not need to defend because the admin has decided it does not apply to their edge case.
ljharb commentedon Mar 25, 2022
I totally agree! see npm/rfcs#422.
That, however, doesn't mean that npm itself should make decisions that are unsafe in nonzero scenarios.
xiromoreira commentedon Mar 31, 2022
@ljharb deleting user files without being asked for is by far an "unsafe in nonzero scenarios" decision, no program should do it. The sane option is to refuse working and/or display a visible warning explaining why.
ljharb commentedon Mar 31, 2022
Sure, eagerly failing loudly would be better also. The outcome is still the same - you wouldn’t be able to do the thing you want, you just would be informed faster.
mk-pmb commentedon Mar 31, 2022
Which can be crucial. In my case, removal of the symlink would have caused backup cron jobs to fail if I had not noticed early enough.
Worst case scenario would probably be somehow breaking the path to someone's SSH authorization mechanism. I can't imagine what such a setup would look like, but from my tech support experience, I'm sure there are people who will have invented a fragile enough setup.
xiromoreira commentedon Apr 1, 2022
Not sure the outcome is still the same. If you delete the symlink but continue the job:
For me the reasonable decision is: the symlink should not be removed, the command should not continue and should exit with non-zero status. It is doing the exact opposite.
vicary commentedon Apr 1, 2022
Being a few package maintainer myself, this very decision creates wondrous moments with
npm link.rohan-v8rma commentedon Feb 26, 2023
@GrumpyLittleTed
Check dropboxignore out if you just want
node_modulesnot to get synced to dropbox, works like a charm.jan-hudec commentedon Dec 12, 2023
I disagree this actually fixes anything, because the preinstall script can do the same damage in myriad of other ways besides replacing
node_moduleswith a symlink. Because it is ultimately running with the same permissions asnpm.To prevent a preinstall script from doing damage, it would have to be sandboxed, and then that sandbox can prevent it from, among many other things, replacing
node_modules(and touching the subdirectories of any other packages than itself).mk-pmb commentedon Dec 15, 2023
Good point. In before "just disable npm scripts".
MerzDaniel commentedon Mar 12, 2025
I'd rather be enabled to consciously disable this security feature, than being forced to accept this...
On my machine there are >100 repos to be looked into from time to time. The aggregated node_modules sizes actually has a central impact on my laptops remaining storage. Having a possibility to have a central package repository starts to become a requirement.
This should be possible with the standard tool NPM and not force users to migrate to e.g. pnpm
mk-pmb commentedon Jul 18, 2025
I finally was annoyed enough to read about how pnpm organizes packages, and their strategy is way smarter than what I had used before. Thanks for recommending!