[dogancanbakir]

Proposed changes

Closes #7431

Since the metadata-cache refactor (#6630), tag filtering moved into index.Filter but the stat increment + log the old TagFilter path performed were not carried over. Templates dropped by default .nuclei-ignore tags (e.g. local) now vanish with no warning at any verbosity, leaving only [FTL] no templates provided for scan.

This re-feeds the existing TemplatesExcludedStats machinery at the filter drop sites, guarded to actual exclude-tag exclusions so include-filter mismatches (-tags, -severity) are not mislabeled.

Proof

Running -code -t <local-tagged code template>:

Before:

[FTL] Could not run nuclei: no templates provided for scan

After:

[WRN] Excluded 1 template[s] with known weak matchers / tags excluded from default run using .nuclei-ignore
[FTL] Could not run nuclei: no templates provided for scan

A template dropped by an include filter (no excluded tag) prints no .nuclei-ignore line.

Checklist

• Pull request is created against the dev branch
• All checks passed (lint, unit/integration/regression tests etc.) with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

Summary by CodeRabbit

• Bug Fixes
  • Enhanced tracking of excluded templates: Templates filtered out by tag rules (such as .nuclei-ignore) are now properly counted and logged with warnings for better visibility into exclusion reasons.

[coderabbitai]

Walkthrough

Added slices import and introduced a noteExcludedByTag helper closure that detects when an index filter rejects a template due to an excluded tag, incrementing stats and optionally logging a warning. The helper is called at two filter-evaluation points: when cached metadata fails matching and when parsed metadata fails matching.

Changes

Template Exclusion Tracking by Tags

Layer / File(s)	Summary
Excluded tag tracking helper pkg/catalog/loader/loader.go	slices package imported and noteExcludedByTag closure defined to check whether an excluded tag caused an indexFilter drop, increment TemplatesExcludedStats, and conditionally log an exclusion warning.
Filter evaluation integration points pkg/catalog/loader/loader.go	Helper invoked when cached metadata and parsing-derived metadata are evaluated against indexFilter.Matches to track excluded templates before returning.

---

🎯 2 (Simple) | ⏱️ ~8 minutes

🐰 With tags to skip and note to track,
When filters filter templates back,
We count and log what's tucked away—
No templates lost without display! 🏷️

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The PR title clearly describes the main change: adding warnings when templates are excluded by .nuclei-ignore tags, which directly addresses the core objective.
Linked Issues check	✅ Passed	The code changes implement the core objective from issue #7431: re-introducing warning messages and stats tracking for templates excluded by .nuclei-ignore tags, allowing users to understand why templates are not running.
Out of Scope Changes check	✅ Passed	All changes are scoped to the template loading logic in loader.go and directly address the linked issue without introducing unrelated modifications.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/7431-silent-excluded-template-warning

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/catalog/loader/loader.go`:
- Around line 790-802: The noteExcludedByTag closure currently treats any tag in
indexFilter.ExcludeTags as coming from .nuclei-ignore; change it to only
increment templates.TemplatesExcludedStats and log the “excluded from default
run using .nuclei-ignore” message when the matching exclude tag actually came
from the ignore file: obtain the ignore-file tags (e.g., via
config.ReadIgnoreFile().Tags or a new store field like ignoreFileTags) and
replace the slices.ContainsFunc check against indexFilter.ExcludeTags with a
check against that ignore-file tag set (or check intersection between
metadata.HasTag and ignore-file tags) so CLI-provided exclude tags
(store.config.ExcludeTags / options.ExcludeTags) are not counted or logged as
.nuclei-ignore exclusions; keep all other behavior (stats/logging) unchanged and
only gate them on the ignore-file tag match.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 4e1bd89e-1670-4893-892d-4d891d2adb5b

📥 Commits

Reviewing files that changed from the base of the PR and between 6b4b2d8 and 3bf9e3f.

📒 Files selected for processing (1)

• pkg/catalog/loader/loader.go

[coderabbitai]

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Search for where options.ExcludeTags is set, particularly checking if .nuclei-ignore
# tags are merged with CLI tags or kept separate

# Find where ExcludeTags field is assigned
rg -n -B3 -A3 '\.ExcludeTags\s*='

# Find references to .nuclei-ignore file loading
rg -n -B3 -A3 'nuclei-ignore'

# Check if there's a separate field or filtering for .nuclei-ignore vs CLI tags
ast-grep --pattern $'struct $_ {
  $$$
  ExcludeTags $$$
  $$$
}'

Repository: projectdiscovery/nuclei

Length of output: 9432

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 1) Inspect loader.go around noteExcludedByTag and how indexFilter is built
echo "=== loader.go context (lines ~740-850) ==="
sed -n '740,850p' pkg/catalog/loader/loader.go | nl -ba

# 2) Find where indexFilter.ExcludeTags is assigned in loader.go
echo
echo "=== indexFilter.ExcludeTags assignments in loader.go ==="
rg -n 'indexFilter\.ExcludeTags\s*=' pkg/catalog/loader/loader.go

# 3) Trace how CLI `-exclude-tags` maps into filters.ExcludeTags / options.ExcludeTags
echo
echo "=== CLI flag parsing for exclude-tags ==="
rg -n --hidden --no-ignore-vcs 'exclude-?tags' .

# 4) Locate filters.ExcludeTags source
echo
echo "=== assignments to filters.ExcludeTags ==="
rg -n 'filters\.ExcludeTags\s*=' .

# 5) Locate flag definition (if any) for ExcludeTags and its destination
echo
echo "=== struct fields or options related to ExcludeTags ==="
rg -n 'ExcludeTags\s+(\[\]string|string)' .

Repository: projectdiscovery/nuclei

Length of output: 171

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== loader.go (around noteExcludedByTag) ==="
awk 'NR>=770 && NR<=820 {print NR ":" $0}' pkg/catalog/loader/loader.go

echo
echo "=== indexFilter.ExcludeTags assignments (loader.go) ==="
rg -n 'indexFilter\.ExcludeTags\s*=' pkg/catalog/loader/loader.go

echo
echo "=== flag / parsing for exclude-tags ==="
rg -n --hidden --no-ignore-vcs 'exclude-?tags' .

echo
echo "=== assignments into filters.ExcludeTags ==="
rg -n --hidden --no-ignore-vcs 'filters\.ExcludeTags\s*=' .

echo
echo "=== places where config.ReadIgnoreFile().Tags is appended to exclude tags ==="
rg -n --hidden --no-ignore-vcs 'ReadIgnoreFile\(\)\.Tags' .

Repository: projectdiscovery/nuclei

Length of output: 2069

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

# 1) Find where indexFilter is defined/initialized in loader.go
echo "=== indexFilter occurrences in loader.go ==="
rg -n 'indexFilter' pkg/catalog/loader/loader.go

echo
echo "=== context around indexFilter setup/initialization (approx 650-780) ==="
awk 'NR>=600 && NR<=820 {print NR ":" $0}' pkg/catalog/loader/loader.go

# 2) Trace ExcludeTags assignment into indexFilter in the whole repo (not just loader.go)
echo
echo "=== assignments to *.ExcludeTags used for index filtering ==="
rg -n 'indexFilter\.(ExcludeTags|IncludeTags)\s*=' . || true

# 3) Find CLI flag / option mapping for exclude-tags
echo
echo "=== places defining exclude-tags flag ==="
rg -n --hidden --no-ignore-vcs 'exclude-?tags' .

# 4) Find where filters.ExcludeTags is populated from CLI/options
echo
echo "=== filters.ExcludeTags population ==="
rg -n --hidden --no-ignore-vcs 'filters\.ExcludeTags\s*[:=]' .

echo
echo "=== structs with ExcludeTags field definitions (limit to relevant hits) ==="
rg -n --hidden --no-ignore-vcs 'ExcludeTags\s+(\[\]string|[]string)' .

# 5) Narrow to command-line/options parsing for exclude tags if flag exists
# (helps connect CLI -> filters.ExcludeTags -> options.ExcludeTags -> store.config -> indexFilter)
echo
echo "=== search for '-exclude-tags' / 'ExcludeTags' in cmd/ or main package ==="
rg -n --hidden --no-ignore-vcs 'ExcludeTags' cmd . | head -n 200 || true

Repository: projectdiscovery/nuclei

Length of output: 11225

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== buildIndexFilter in loader.go ==="
rg -n 'buildIndexFilter' pkg/catalog/loader/loader.go
awk 'NR>=130 && NR<=260 {print NR ":" $0}' pkg/catalog/loader/loader.go

echo
echo "=== store.buildIndexFilter definition (full function if not too large) ==="
# Print exact function body by extracting from "func (store *Store) buildIndexFilter" to next "func"
start=$(rg -n 'func \(store \*Store\) buildIndexFilter' -n pkg/catalog/loader/loader.go | head -n1 | cut -d: -f1)
awk -v s="$start" 'NR>=s {print} NR>s && /^func / {exit}' pkg/catalog/loader/loader.go || true

echo
echo "=== index.Filter struct and ExcludeTags usage ==="
rg -n 'type Filter|ExcludeTags|HasTag|Matches\(' pkg/catalog/index/filter.go

echo
echo "=== index/filter.go around ExcludeTags handling ==="
awk 'NR>=300 && NR<=480 {print NR ":" $0}' pkg/catalog/index/filter.go

echo
echo "=== index/filter.go around struct definition ==="
awk 'NR>=1 && NR<=120 {print NR ":" $0}' pkg/catalog/index/filter.go

echo
echo "=== where config.ReadIgnoreFile().Tags gets appended into ExcludeTags ==="
rg -n 'ReadIgnoreFile\(\)\.Tags' -S .

Repository: projectdiscovery/nuclei

Length of output: 13621

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== where options.ExcludeTags is used ==="
rg -n --hidden --no-ignore-vcs 'options\.ExcludeTags' .

echo
echo "=== where filters.ExcludeTags is filled from options ==="
rg -n --hidden --no-ignore-vcs 'filters\.ExcludeTags' lib pkg cmd internal .

echo
echo "=== trace store.config.ExcludeTags assignments ==="
rg -n --hidden --no-ignore-vcs 'ExcludeTags:\s*' lib pkg cmd internal .

echo
echo "=== inspect config/filter struct around ExcludeTags mapping ==="
# show relevant parts of lib/config.go
awk 'NR>=540 && NR<=660 {print NR ":" $0}' lib/config.go

Repository: projectdiscovery/nuclei

Length of output: 8066

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== pkg/catalog/loader/loader.go around ExcludeTags assignment from options (~90-150) ==="
awk 'NR>=70 && NR<=160 {print NR ":" $0}' pkg/catalog/loader/loader.go

echo
echo "=== pkg/catalog/loader/loader.go call sites of noteExcludedByTag (~820-900) ==="
# noteExcludedByTag likely used shortly after load filtering loops
awk 'NR>=800 && NR<=920 {print NR ":" $0}' pkg/catalog/loader/loader.go

echo
echo "=== pkg/types/types.go around ExcludeTags wiring (~450-530) ==="
awk 'NR>=450 && NR<=530 {print NR ":" $0}' pkg/types/types.go

Repository: projectdiscovery/nuclei

Length of output: 13223

---

Fix .nuclei-ignore exclusion stats/log to only count ignore-file tags

noteExcludedByTag (pkg/catalog/loader/loader.go:790-802) increments templates.TemplatesExcludedStats and logs “excluded from default run using .nuclei-ignore” whenever a template matches indexFilter.ExcludeTags, but indexFilter.ExcludeTags is built from store.config.ExcludeTags (store.buildIndexFilter → ExcludeTags: store.config.ExcludeTags), which is populated from options.ExcludeTags (CLI -exclude-tags) and also appended with config.ReadIgnoreFile().Tags (lib/sdk_private.go:140). There’s no check that the matched exclude tag originated from .nuclei-ignore, so CLI -exclude-tags will be misattributed/counted as .nuclei-ignore exclusions.

Separate ignore-file exclude tags from CLI exclude tags in the filter logic (e.g., check intersection only with config.ReadIgnoreFile().Tags) before incrementing/logging.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/catalog/loader/loader.go` around lines 790 - 802, The noteExcludedByTag
closure currently treats any tag in indexFilter.ExcludeTags as coming from
.nuclei-ignore; change it to only increment templates.TemplatesExcludedStats and
log the “excluded from default run using .nuclei-ignore” message when the
matching exclude tag actually came from the ignore file: obtain the ignore-file
tags (e.g., via config.ReadIgnoreFile().Tags or a new store field like
ignoreFileTags) and replace the slices.ContainsFunc check against
indexFilter.ExcludeTags with a check against that ignore-file tag set (or check
intersection between metadata.HasTag and ignore-file tags) so CLI-provided
exclude tags (store.config.ExcludeTags / options.ExcludeTags) are not counted or
logged as .nuclei-ignore exclusions; keep all other behavior (stats/logging)
unchanged and only gate them on the ignore-file tag match.