From bc9af80dcd3456b763bdbbcd30b6269699e27a7b Mon Sep 17 00:00:00 2001 From: Dwood Date: Sat, 28 Apr 2018 19:08:24 -0600 Subject: [PATCH 1/3] Add Custom RFCS file --- accepted/0000-drop-in-replacements.md | 35 +++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 accepted/0000-drop-in-replacements.md diff --git a/accepted/0000-drop-in-replacements.md b/accepted/0000-drop-in-replacements.md new file mode 100644 index 000000000..6f256f5d8 --- /dev/null +++ b/accepted/0000-drop-in-replacements.md @@ -0,0 +1,35 @@ +# {{TITLE: a human-readable title for this RFC!}} + +## Summary + +{{A concise, one-paragraph description of the change.}} + +## Motivation + +{{Why are we doing this? What pain points does this resolve? What use cases does it support? What is the expected outcome? Use real, concrete examples to make your case!}} + +## Detailed Explanation + +{{Describe the expected changes in detail, }} + +## Rationale and Alternatives + +{{Discuss 2-3 different alternative solutions that were considered. This is required, even if it seems like a stretch. Then explain why this is the best choice out of available ones.}} + +## Implementation + +{{Give a high-level overview of implementaion requirements and concerns. Be specific about areas of code that need to change, and what their potential effects are. Discuss which repositories and sub-components will be affected, and what its overall code effect might be.}} + +{{THIS SECTION IS REQUIRED FOR RATIFICATION -- you can skip it if you don't know the technical details when first submitting the proposal, but it must be there before it's accepted}} + +## Prior Art + +{{This section is optional if there are no actual prior examples in other tools}} + +{{Discuss existing examples of this change in other tools, and how they've addressed various concerns discussed above, and what the effect of those decisions has been}} + +## Unresolved Questions and Bikeshedding + +{{Write about any arbitrary decisions that need to be made (syntax, colors, formatting, minor UX decisions), and any questions for the proposal that have not been answered.}} + +{{THIS SECTION SHOULD BE REMOVED BEFORE RATIFICATION}} From 74003a517e9ded30697c7a2a02348d2aae99c568 Mon Sep 17 00:00:00 2001 From: Dason Woodhouse Date: Sat, 28 Apr 2018 23:14:10 -0600 Subject: [PATCH 2/3] Initial rfcs commit --- accepted/0000-drop-in-replacements.md | 156 ++++++++++++++++++++++++-- 1 file changed, 149 insertions(+), 7 deletions(-) diff --git a/accepted/0000-drop-in-replacements.md b/accepted/0000-drop-in-replacements.md index 6f256f5d8..8382eed7a 100644 --- a/accepted/0000-drop-in-replacements.md +++ b/accepted/0000-drop-in-replacements.md @@ -1,35 +1,177 @@ -# {{TITLE: a human-readable title for this RFC!}} +# Drop In Replacements ## Summary -{{A concise, one-paragraph description of the change.}} +This proposal introduces to npm the concept of marking packages compatible as drop-in-replacements, in place of another package. ## Motivation -{{Why are we doing this? What pain points does this resolve? What use cases does it support? What is the expected outcome? Use real, concrete examples to make your case!}} +As npm has grown, many packages have been created, developed, and quite a few, even popular packages, have been abandoned completely. They, or even their dependencies become outdated as both npm and nodejs progresses and updates their respective api +'s. These packages, or their dependencies which are similarly abandoned, may have security issues. + +Often times, these outdated packages/dependencies have been forked, updated, their security bugs fixed, new features/more documentation. + +On top of this, it may be difficult to find repositories or npm packages that function as drop-in replacements allowing minimal upgrade headache, nor is there a way to specify a drop-in for a dependency's dependency. + +This proposal seeks to help npm users: + +Find out how old their dependencies (and those dependency's dependencies) are. +Find packages which can function drop-in replacements for another, outdated package +Explicitly use an alternative/fork of a package in place of another, regardless of how deep the modules tree they need to go. + +While a package's number of, or most recent updates list is not a positive indicator of a package's security, showing users and organizations packages which are no longer being maintained, and helping them find drop-in replacements can help them be more proactive in tracking down bugs, security issues, and keeping up-to-date with the latest browser, node, npm features and optimisations. ## Detailed Explanation -{{Describe the expected changes in detail, }} +### Finding Unmaintained packages - `npm audit` flags + +#### `npm audit --unmaintained` + +Output: Display packages of concern in gridlike fastion, ordered firstly by potential security severity, then, by the time of the most recent git commits//updates to the npm site's package. + +Only packages that haven't been updated in > X months ago are shown, where X is some arbitrary number chosen based on a metric, such as node/npm lts cycles. + +Causes npm to poll the site/registry/github for all package the current package/repository depends on, and attempts to find compatible repositories of the current package. + +Formula to order packages is based on a combination of the potential security threat of relying on a particular unmaintained packages could be and npm package + git activity. Low package + git activity are shown first, where higher activity packages are shown last. + +Example: + +``` +$ npm audit --unmaintained + === npm audit report of unmaintained dependencies === + +# Run `npm audit --find-forks` to find active forks of this package, or, instead: +# `npm audit --drop-ins` to find packages marked as drop-ins of this package. +# To see a module in this repository's tree, use: `npm ls ` +┌──────────────────┬─────────────┬───────────────┬─────────────────────┬───────────────────┐ +│ Package Name │ Last Update │ Sec. Severity │ Dependency of │ Dependency-Depth │ +├──────────────────┼─────────────┼───────────────┼─────────────────────┼───────────────────┤ +│ http-proxy-agent │ Apr 4, 2017 │ High │ npm-profile │ 3 │ +├──────────────────┼─────────────┼───────────────┼─────────────────────┼───────────────────┤ + (...) +├──────────────────┼─────────────┼───────────────┼─────────────────────────────────────────┤ +│ blessed │ Jan 4, 2016 │ Low │ │ 0 │ +└──────────────────┴───────────────────────────────────────────────────────────────────────┘ +``` + +### Finding Drop-In Replacements + +### Additions & Changes to `npm install` + +Unless indicated otherwise, npm's Standard Operating Protocol, or (SOP) for versioning and dependency resolution should remain the same. + +The new `npm install` flags are as follows: + +`npm install --replace= --deep` + +#### `` + +Essentially the same as a regular install, except `package` is either: + +1) A direct fork of ``, as indicated by git, with the `package.json` attribute `"forkOf":`, described below. + +Or: + +2) A package with a `package.json` containing a `"replaces":` attribute , described below. + +#### `--replace=` + +Tells npm to install this module as a replacement for top-level references (all manually-installed modules) whose versions requirements on map with . + +If ``'s versioning does not match with ``, npm should emit an error, such as: + +```Installation Error! Replacement Package: is not marked as a replacement for ``` + +#### `--deep-replace` or `--deep` + +`--deep` tells npm to attempt to resolve all references to `` to be resolved to ``. regardless of depth. + +Should a dependency specify their own `"replace":` package.json, `--deep` or `--deep-replace` will prefer the current user's package, regardless of the package's lockfile. If the dependency's versioning is not marked as compatible. In such cases, npm will use standard versioning and dependency protocol. + +If `--deep` or `--deep-replace` is specified and `--replace=` is not, npm should emit an error such as: + +```--deep-replace does not make sense without --replace=``` + +### Changes to package.json and registry + +Additional attributes for package.json files, package-lock.json, and npm's registry. + +#### The `forkOf` package.json attribute + + `'forkOf': '@'` + + Indicates the version of `` which was forked, and, consequently, the last version of `` which is an exact copy of ``'s version tag. + + Side note: Up to this point, all commit sha's will likely have the the exact same signatures. + +#### package.json, registry attributes: `forkOf`, `dropInFor` + +No changes on the part of the package being replaced should required. + +Forked (updated/maintained) `package.json`: +``` + "name": "dwood15/blessed", +{{ ... }} + "replaces": { + "package":"blessed", + "isFork": "true", + "minVersion": "^0.0.1", + "maxVersion": "^0.0.1", + "forkedFrom": "https://github.com/chjj/blessed@0.1.0" + } + "preferGlobal": false, + "repository": "git://github.com/dwood15/blessed.git", +{{...}} +``` + +### package-lock.json attribute + +... Is a package-lock.json flag/attribute actually necessary? + +During a regular install, while npm resolves dependencies, npm references the package-lock.json and checks +For recording exact versioning and dependency replacement. ## Rationale and Alternatives -{{Discuss 2-3 different alternative solutions that were considered. This is required, even if it seems like a stretch. Then explain why this is the best choice out of available ones.}} +1) Symlinking + +Requires the most effort on the part of the user. + +2) Package Aliasing + +Package aliasing may work as drop-in replacement, but doesn't solve majority of the issues with outdated npm packages/dependencies. + +Scope is limited. Does not allow finding outdated dependencies regardless of depth, nor does it allow to search npm for potential replacements of an outdated package + +3) Fork outdated repository, apply fixes, add to npm registry, then npm install updated repository. + +This is the most common methodology of fixing bugs in outdated repositories, but requires more steps the further down the dependency tree a user has to go. + +It's a lot of steps to fork an outdated repository, apply the fixes, add the new package by npm install . This creates a silly number of repositorys to have to wade through when searching the repository, and only exacerbates the issues of outdated repositories. We end up with a massive number of forked packages which fix that user's specific bug, and then the new package is never updated again. + +--- + +First-class Aliasing support only works on a per-project, top-level basis, and so if my issue is multiple levels down the dependency tree, I have to go one above the defective dependency, do #3 where the fix is to alias the package, and then re-publsh the repository to npm's registry. + +This solution, while the most complex, encourages users to pick up and update old packages by making it easier or others to find their better-supported/more commonly updated modules. It also encourages users to be more proactive with respect to security and semantic versioning. ## Implementation {{Give a high-level overview of implementaion requirements and concerns. Be specific about areas of code that need to change, and what their potential effects are. Discuss which repositories and sub-components will be affected, and what its overall code effect might be.}} -{{THIS SECTION IS REQUIRED FOR RATIFICATION -- you can skip it if you don't know the technical details when first submitting the proposal, but it must be there before it's accepted}} - ## Prior Art {{This section is optional if there are no actual prior examples in other tools}} {{Discuss existing examples of this change in other tools, and how they've addressed various concerns discussed above, and what the effect of those decisions has been}} +As far as I know, no dependency management tool offers features like this. + ## Unresolved Questions and Bikeshedding +To what extent can a user solve these issues, + {{Write about any arbitrary decisions that need to be made (syntax, colors, formatting, minor UX decisions), and any questions for the proposal that have not been answered.}} {{THIS SECTION SHOULD BE REMOVED BEFORE RATIFICATION}} From ef165599ff3079c9caab194b6e97f5f5df8fc752 Mon Sep 17 00:00:00 2001 From: Dwood Date: Wed, 2 May 2018 21:58:10 -0600 Subject: [PATCH 3/3] Reduce scope somewhat. --- accepted/0000-drop-in-replacements.md | 179 ++++++++------------------ 1 file changed, 56 insertions(+), 123 deletions(-) diff --git a/accepted/0000-drop-in-replacements.md b/accepted/0000-drop-in-replacements.md index 8382eed7a..8806841a8 100644 --- a/accepted/0000-drop-in-replacements.md +++ b/accepted/0000-drop-in-replacements.md @@ -2,176 +2,109 @@ ## Summary -This proposal introduces to npm the concept of marking packages compatible as drop-in-replacements, in place of another package. +This proposal introduces the ability to mark packages compatible as drop-in-replacements, or packages which can be used in place of another package. ## Motivation -As npm has grown, many packages have been created, developed, and quite a few, even popular packages, have been abandoned completely. They, or even their dependencies become outdated as both npm and nodejs progresses and updates their respective api -'s. These packages, or their dependencies which are similarly abandoned, may have security issues. +Replacing and updating outdated dependencies is difficult, ensuring nested dependencies are the latest and most-maintained version is basically impossible. -Often times, these outdated packages/dependencies have been forked, updated, their security bugs fixed, new features/more documentation. +Users need a way to find old and outdated packages in their own package.json file, they also need to be able to replace packages that may have bugs or may not fit their needs, with another package, regardless of the dependency depth. -On top of this, it may be difficult to find repositories or npm packages that function as drop-in replacements allowing minimal upgrade headache, nor is there a way to specify a drop-in for a dependency's dependency. +It may be difficult to find repositories or packages that can function as drop-in replacements, allowing minimal upgrade headache, nor is there a way to specify a drop-in for a dependency's dependency. -This proposal seeks to help npm users: +Outdated packages are often forked, updated, their security bugs fixed, new features/more documentation could be added, etc. If a user makes a project a fork of an existing one, or if they deliberately have the same api as another package, there is no way to communicate this compatibility to other developers. -Find out how old their dependencies (and those dependency's dependencies) are. -Find packages which can function drop-in replacements for another, outdated package -Explicitly use an alternative/fork of a package in place of another, regardless of how deep the modules tree they need to go. +#### This proposal explicitly seeks to help users find packages that are drop-in replacements for another package. -While a package's number of, or most recent updates list is not a positive indicator of a package's security, showing users and organizations packages which are no longer being maintained, and helping them find drop-in replacements can help them be more proactive in tracking down bugs, security issues, and keeping up-to-date with the latest browser, node, npm features and optimisations. +#### It does not seek to add aliasing regardless of depth, or help users find unmaintained packages. ## Detailed Explanation -### Finding Unmaintained packages - `npm audit` flags +### Marking Packages as Compatible -#### `npm audit --unmaintained` +The key to this feature is the ability to mark packages as compatible. + +This is expected to be a flag set by a package's author in their own package.json. -Output: Display packages of concern in gridlike fastion, ordered firstly by potential security severity, then, by the time of the most recent git commits//updates to the npm site's package. +This flag, or package.json entry should be able to indicate: -Only packages that haven't been updated in > X months ago are shown, where X is some arbitrary number chosen based on a metric, such as node/npm lts cycles. - -Causes npm to poll the site/registry/github for all package the current package/repository depends on, and attempts to find compatible repositories of the current package. - -Formula to order packages is based on a combination of the potential security threat of relying on a particular unmaintained packages could be and npm package + git activity. Low package + git activity are shown first, where higher activity packages are shown last. - -Example: - -``` -$ npm audit --unmaintained - === npm audit report of unmaintained dependencies === - -# Run `npm audit --find-forks` to find active forks of this package, or, instead: -# `npm audit --drop-ins` to find packages marked as drop-ins of this package. -# To see a module in this repository's tree, use: `npm ls ` -┌──────────────────┬─────────────┬───────────────┬─────────────────────┬───────────────────┐ -│ Package Name │ Last Update │ Sec. Severity │ Dependency of │ Dependency-Depth │ -├──────────────────┼─────────────┼───────────────┼─────────────────────┼───────────────────┤ -│ http-proxy-agent │ Apr 4, 2017 │ High │ npm-profile │ 3 │ -├──────────────────┼─────────────┼───────────────┼─────────────────────┼───────────────────┤ - (...) -├──────────────────┼─────────────┼───────────────┼─────────────────────────────────────────┤ -│ blessed │ Jan 4, 2016 │ Low │ │ 0 │ -└──────────────────┴───────────────────────────────────────────────────────────────────────┘ -``` - -### Finding Drop-In Replacements - -### Additions & Changes to `npm install` - -Unless indicated otherwise, npm's Standard Operating Protocol, or (SOP) for versioning and dependency resolution should remain the same. - -The new `npm install` flags are as follows: - -`npm install --replace= --deep` - -#### `` - -Essentially the same as a regular install, except `package` is either: - -1) A direct fork of ``, as indicated by git, with the `package.json` attribute `"forkOf":`, described below. - -Or: - -2) A package with a `package.json` containing a `"replaces":` attribute , described below. - -#### `--replace=` - -Tells npm to install this module as a replacement for top-level references (all manually-installed modules) whose versions requirements on map with . - -If ``'s versioning does not match with ``, npm should emit an error, such as: - -```Installation Error! Replacement Package: is not marked as a replacement for ``` - -#### `--deep-replace` or `--deep` - -`--deep` tells npm to attempt to resolve all references to `` to be resolved to ``. regardless of depth. - -Should a dependency specify their own `"replace":` package.json, `--deep` or `--deep-replace` will prefer the current user's package, regardless of the package's lockfile. If the dependency's versioning is not marked as compatible. In such cases, npm will use standard versioning and dependency protocol. - -If `--deep` or `--deep-replace` is specified and `--replace=` is not, npm should emit an error such as: - -```--deep-replace does not make sense without --replace=``` - -### Changes to package.json and registry - -Additional attributes for package.json files, package-lock.json, and npm's registry. - -#### The `forkOf` package.json attribute - - `'forkOf': '@'` - - Indicates the version of `` which was forked, and, consequently, the last version of `` which is an exact copy of ``'s version tag. + * Earliest version {{currentPackage}} is a suitable drop-in for. - Chosen by author + * Latest version {{currentPackage}} is suitable drop-in for. - Chosen by author + * Type of compatibility - whether or not the package is a direct fork. + * isFork (optional) - detected/verified by npm CLI / by npm site on publish or repository location change. - Side note: Up to this point, all commit sha's will likely have the the exact same signatures. - -#### package.json, registry attributes: `forkOf`, `dropInFor` +#### package.json attributes: `compatibleWith`. `forkOf`, or `dropInFor` No changes on the part of the package being replaced should required. -Forked (updated/maintained) `package.json`: + `package.json` additions: + ``` - "name": "dwood15/blessed", -{{ ... }} - "replaces": { - "package":"blessed", - "isFork": "true", + "compatibleWith": { + "package": "blessed", + "isFork": true, "minVersion": "^0.0.1", - "maxVersion": "^0.0.1", + "maxVersion": "^1.0.0", "forkedFrom": "https://github.com/chjj/blessed@0.1.0" } "preferGlobal": false, "repository": "git://github.com/dwood15/blessed.git", -{{...}} ``` ### package-lock.json attribute -... Is a package-lock.json flag/attribute actually necessary? - -During a regular install, while npm resolves dependencies, npm references the package-lock.json and checks -For recording exact versioning and dependency replacement. - + * Is a package-lock.json flag/attribute necessary? + + ## Rationale and Alternatives -1) Symlinking + * Searching Github or another website for forks which are decently maintained is a less-than-pleasant experience. + +Npm does not currently allow searching for packages based on compatibility. -Requires the most effort on the part of the user. +First-class Aliasing support only works on a per-project, top-level basis, and so if my issue is multiple levels down the dependency tree, I have to go one above the defective dependency, do #3 where the fix is to alias the package, and then re-publsh the repository to npm's registry. -2) Package Aliasing +This solution encourages users to pick up and update old packages by making it easier or others to find their better-supported/more commonly updated modules. It also encourages users to be more proactive with respect to security and semantic versioning. -Package aliasing may work as drop-in replacement, but doesn't solve majority of the issues with outdated npm packages/dependencies. +## Implementation -Scope is limited. Does not allow finding outdated dependencies regardless of depth, nor does it allow to search npm for potential replacements of an outdated package +The development phase of a project or package should be all that's really marked, as this feature is scaffolding for search as well as additions to an implementation feature. -3) Fork outdated repository, apply fixes, add to npm registry, then npm install updated repository. +## Prior Art -This is the most common methodology of fixing bugs in outdated repositories, but requires more steps the further down the dependency tree a user has to go. +As far as I know, no dependency management tool offers this feature. -It's a lot of steps to fork an outdated repository, apply the fixes, add the new package by npm install . This creates a silly number of repositorys to have to wade through when searching the repository, and only exacerbates the issues of outdated repositories. We end up with a massive number of forked packages which fix that user's specific bug, and then the new package is never updated again. +The closest is git's "fork of", but navigating that feature in large and commonly-used repositories to find active and currently-maintained offshoots of a package is nigh-impossible. ---- +## Unresolved Questions and Bikeshedding -First-class Aliasing support only works on a per-project, top-level basis, and so if my issue is multiple levels down the dependency tree, I have to go one above the defective dependency, do #3 where the fix is to alias the package, and then re-publsh the repository to npm's registry. +### Short story//Disclaimer//Prior Art(?) -This solution, while the most complex, encourages users to pick up and update old packages by making it easier or others to find their better-supported/more commonly updated modules. It also encourages users to be more proactive with respect to security and semantic versioning. +I used chjj's blessed repository for minor personal projects in linux. The repository is stable, does what it intended to do, and is relatively speedy. -## Implementation +On the other hand, the repository (as of May 2. 2018) hasn't been updated in 3 years, and has only been accumulating PR's and Issues. -{{Give a high-level overview of implementaion requirements and concerns. Be specific about areas of code that need to change, and what their potential effects are. Discuss which repositories and sub-components will be affected, and what its overall code effect might be.}} +After forking, I integrated a number of open pull requests from chjj/blessed's repo and fixed some dependency problems, giving blessed Windows support. -## Prior Art +My package, has had virtually no traffic despite resolving some bugs and incompatibilities with Windows, making more of blessed work out of the box with the OS. -{{This section is optional if there are no actual prior examples in other tools}} +Renaming the package and striking out on a different lane wasn't the goal - I had nothing unique to offer other than being willing to integrate a few PR's and fix some bugs. -{{Discuss existing examples of this change in other tools, and how they've addressed various concerns discussed above, and what the effect of those decisions has been}} +The repository owner has not responded to my email or any other comment. In the end, I made my own account and package on the registry. The ordeal felt pretty wasteful. -As far as I know, no dependency management tool offers features like this. -## Unresolved Questions and Bikeshedding +### The fridge - These items may need a little more time to bake. -To what extent can a user solve these issues, +#### Finding Drop-In Replacements -{{Write about any arbitrary decisions that need to be made (syntax, colors, formatting, minor UX decisions), and any questions for the proposal that have not been answered.}} +A few options for helping users find drop-in replacements. + + 1) Notice/tab on a package page that is relatively unmaintained indicating a package has been forked. + 1) Tab/notice should bring up a list of compatible pkgs/forks, ordered by some combination of code quality/security/maintenance metrics. + 2) + +#### The `forkOf` package.json attribute -{{THIS SECTION SHOULD BE REMOVED BEFORE RATIFICATION}} + `'forkOf': '@'` + + Indicates the version of `` which was forked, and, consequently, the last version of `` which is an exact copy of ``'s version tag. Easily verified via sha1 checksum comparisons.