fix(pd001,pd002): resolve imports per workspace member and ignore type-only imports - #1114
Conversation
…e-only imports Two independent false-positive sources in the phantom-dependency rules, both on the same comparison in the PD detectors. Workspace roots: imports were collected from the whole tree but declarations were resolved against the root manifest only, so scanning a monorepo root reported every dependency a member declares and imports in its own source as a transitive-only phantom. buildOverrideContext now discovers workspace members (root `workspaces`, pnpm-workspace.yaml) and their declared packages, and PD001/PD002 resolve each importing file against the nearest enclosing member before falling back to the root. The finding lists only the files whose owning package leaves the import undeclared. Type-only imports: the usage scanner is a regex pass over raw file text, so a JSDoc annotation such as `/** @type {import('postcss-load-config') .Config} */` counted as a dynamic import, and `import type { X } from 'pkg'` counted as a runtime import. Comments are now blanked out (string-aware) before matching, and `import type` / `export type` / all-`type` specifier lists are skipped. This applies to every consumer of the scanner: PD001, PD002, the OA009 guard, and the --usage filter, where a type-only reference to a vulnerable package no longer counts as usage. Closes OWASP#966
sonukapoor
left a comment
There was a problem hiding this comment.
Thanks for this, and for the diagnosis in particular. You read #966 correctly on both halves, and a few things here are better than what the issue asked for.
undeclaredImportFiles filtering the file list rather than short-circuiting the whole package is the right call. A package imported in five members and declared in three now reports only the two that are genuinely undeclared, and the "Imported in:" line stays honest. I also checked the case I most expected to be wrong, and it is not: import { type A, b } from 'p' is correctly treated as a runtime import while import type { A } from 'p' is erased. That distinction trips most people up. The ${member.dir}/ suffix stopping apps/web from claiming apps/web-admin is a nice catch too, and you tested it.
I checked the tests would actually fail if the fix were reverted rather than assuming it, and they do.
There is one thing I need fixed before this can land, and it is not in the part of the change you were focused on.
stripComments has no regex-literal state, and the top-level scan has no escape handling even though the string branch right below it does. That means the very common /\// idiom gets misread as the start of a line comment, and everything after it on that line is blanked. /[/*]/ is worse: it opens a false block comment that runs to the next */ or to the end of the file.
The reason I care about this more than the false positives we are fixing is that scanAllImports is not only feeding PD001 and PD002. It also feeds --only-used, which suppresses findings. So a lost import there does not produce noise, it produces silence. Concretely, with this branch:
const isAbs = /^\//.test(p);
const minimist = require('minimist');put those on one line and minimist is reported as unused, so --only-used prints "No known vulnerabilities found" while minimist@1.2.5 still carries a critical. Removing just the regex from that line brings the finding back. A scanner that quietly hides a critical is a worse outcome than one that reports a phantom, so I would rather take the false positives for another week than ship this.
The fix is small. The string branch at line 63 already consumes \ plus the next character as a pair. The top level needs the same thing, so a \ before a / cannot open a comment. Full regex-literal tracking would be more correct, but escape awareness covers every case I could construct.
Three things then:
- escape handling at the top level of
stripComments, covering both the//and/*branches - a fixture with
/\//,/https?:\/\//and/[/*]/followed by a real import, both on the same line and on later lines - an end to end
--only-usedtest asserting a genuinely required vulnerable package is still reported. There is no test for that path at all today, which is how this got past CI, and given it now gates a suppression decision it should have one regardless of this PR
Everything else I found is follow-up rather than a change request, and I will file it separately so it does not sit on you:
- OA009's guard still resolves declarations against the root manifest, so it is now out of step with PD001 in a workspace. Worth noting the pd001 doc page this PR edits still describes the two as paired.
packages/**andpkg-*workspace globs are not matched by the existing pattern expansion, so those monorepos keep the false positives. Pre-existing, not yours.stripCommentsbuilds its result with per-character concatenation, which is roughly 11x slower than the old pass. Not urgent, but #837 is open on performance.
One design question I would like your view on rather than a change. Dropping import type is unambiguously right for --usage and --only-used, which model runtime reachability. I am less sure it is right for PD001 and PD002, which model declaration hygiene: a type-only import of an undeclared transitive package really will break tsc the moment the parent drops it, which is close to what PD002's own message warns about. The comment and JSDoc half of this has no such tension and is simply correct. If you think the cleaner shape is for scanAllImports to record whether an import was type-only and let each caller apply its own policy, I would be open to that, either here or as a follow-up.
To be clear on credit: @alamb-hex diagnosed both defects in #966 down to the exact comparison, and he will be credited as reporter in the release notes alongside you as author.
|
Filed the three follow-ups I mentioned, so none of them sit on this PR:
All three are ours, not yours. #1118 and #1120 both want to land after this PR since they touch code it introduces. The only thing still on you is the regex handling in |
# Conflicts: # src/overrides/context.ts
stripComments had no regex-literal state, so a `/` inside a regex read as the start of a comment. The worst case is a character class containing both slash and star: in `const sep = /[/*]/;` the inner `/*` opened a block comment that ran to the next `*/` or to end of file, blanking every import below it. That silently dropped imports for PD001, PD002, OA009 and --usage / --only-used, since all four read the same pass. Reproduced on a four-line file where both imports below a glob regex disappeared. A `/` now opens a regex literal only where an expression may begin, judged from the last significant character and, for the keyword cases like `return /x/`, the identifier run ending there. Character classes are tracked so a slash inside one does not terminate the literal, and scanning stops at a newline because a regex literal cannot span one, which bounds a misjudged division to a single line. `}` stays ambiguous without a parser and is treated as expression position. Adds the test the review asked for: the regex case, alongside assertions that division, line comments, block comments and JSDoc import annotations all still behave.
|
@osfv - I have pushed two commits to this branch rather than leave it sitting, and I want to be upfront that I would normally ask you to do it rather than doing it myself. The reasoning: the review went up on 10 September, the branch had drifted into conflict, and it closes #966, which @alamb-hex reported back in August and which affects a real monorepo of his. It also holds What I pushed. A merge of current Then the regex-literal fix, which was the blocking item. You were closer than the review made it sound, and the failure was more severe than I described at the time. In A I also added the test you were asked for, covering the regex case alongside division, line comments, block comments and JSDoc annotations. It fails if the regex branch is disabled, so it is pinning the behaviour rather than just passing. 2102 tests green, build clean. Your work and your authorship are intact, and the substance here is yours: the per-workspace-member resolution and the type-only import handling are the fix, and they were right. #966's reporter credit goes to @alamb-hex in the release notes, and you will be credited as the author. If you would rather I had left it alone, say so and I will take that as the default next time. |
sonukapoor
left a comment
There was a problem hiding this comment.
Approved.
All three threads from the 10 September review are addressed by the commits now on this branch, so I have resolved them:
- the regex-literal state in
stripComments - the character-class handling, which was the blast-radius concern on that thread
- the regex case in
tests/usage.test.ts
2102 tests green against current main, build clean.
Thanks @osfv. The per-workspace-member resolution and the type-only import handling are the fix here, and both were right. Reporter credit for #966 goes to @alamb-hex in the release notes.
|
Merged - thank you @osfv! This closes #966, and both defects in it are now fixed: PD001 and PD002 resolve declarations against the nearest enclosing workspace member rather than the root manifest, and type-only imports no longer count as runtime imports. Credit to @alamb-hex for the report. He diagnosed both defects precisely and measured the type-only-import false positives across 34 projects, and he gets reporter credit in the release notes alongside @osfv as author. Two follow-ups of ours land on top of this: #1118 for OA009's declaration guard, which now diverges from PD001, and #1120 for the |
What changed and why
Fixes both defects reported in #966. They land on the same comparison in the PD detectors, so they are in one change as suggested there.
A - workspace roots
Imports were collected from the whole tree while declarations were resolved against the root manifest only, so scanning a monorepo root reported every dependency a member declares and imports in its own source as a PD002 phantom (or PD001 when an override happened to exist).
buildOverrideContextnow discovers workspace members (rootworkspaces,pnpm-workspace.yaml) via a newreadWorkspaceMemberManifestsinsrc/utils/package-json.ts, reusing the same pattern expansionreadDirectDependencyNamesalready uses for the CVE scan, and exposesctx.workspaceMembers(dir + declared set).phantom-utils.undeclaredImportFiles.B - type-only imports counted as runtime
scanAllImports/scanProjectForPackageUsageare a regex pass over raw text, so/** @type {import('postcss-load-config').Config} */counted as a dynamic import andimport type { X } from 'pkg'counted as a runtime import.//,/* */) are blanked before matching, string-aware so'https://...'is not treated as a comment.import type ...,export type ..., and specifier lists where every entry istype-prefixed are skipped. A default import that happens to be namedtype, or a mixed list (import { type A, b }), still counts.This applies to every consumer of the scanner, as discussed in the issue: PD001, PD002, the OA009 guard, and the
--usagereachability filter. One existing assertion intests/usage.test.tstreatedimport typeas usage and was updated accordingly; a type-only reference to a vulnerable package no longer counts as "used".Verification
Against the minimal repro from the issue (pnpm workspace root,
apps/webdeclares and importsjs-yaml) plus a stockpostcss.config.mjswith the JSDoc annotation,mainreports two PD002 findings and this branch reports none. New unit coverage intests/usage.test.ts,tests/overrides/detectors/pd00{1,2}.test.ts, andtests/overrides/context-builder.test.ts. Rule docs for PD001/PD002 gained a short "What counts as an import" section.Note: #966 is assigned to @alamb-hex. The maintainer's check-in on Sept 6 had no reply, so I went ahead; happy to close this in favour of theirs if they are still working on it.
Closes #966