Skip to content

refactor(templates): centralize opt-in capability gates - #7489

Merged
Mzack9999 merged 1 commit into
devfrom
dwisiswant0/refactor/templates/centralize-opt-in-capability-gates
Jun 23, 2026
Merged

Mzack9999 merged 1 commit into
devfrom
dwisiswant0/refactor/templates/centralize-opt-in-capability-gates

Conversation

@dwisiswant0

@dwisiswant0 dwisiswant0 commented Jun 21, 2026 •

Copy link
Copy Markdown
Member

Proposed changes

Move template execution requirements into a shared
capability table that maps each capability to its
flag, stats key, and template predicate. Use the
shared model from template loading, workflow
parsing, request compilation, and runner stats
display so file, self-contained, headless, code,
DAST, and global matcher gates stay in sync.


Prev. unverified code templates were being counted in overlapping and/or inconsistent ways. Sometimes the top-level loader would increment both the generic unverified counter and the code-template counter for the same skip, and workflow subtemplates did their own thing. The numbers were basically meaningless. Here, that the counters are mutually exclusive now, they actually mean something clear, much easier to reason about.

A few things are worth calling out when I'm working on this:

We have got a vocab mess. The actual terms we use in the code & docs like signed vs. tampered vs. verified, skipped vs. excluded, and similar labels. These words are not interchangeable, but we have been treating them like they are. That's how we end up with confusing stats, contradictory comments, and weird behavior down the line. We should probably just write down clear definitions once (a tiny GLOSSARY dot md would do it) and then actually stick to them.

The user-facing messages are a separate concern. A lot of them feel like they were added randomly over time, slightly different wording, tone, and even spelling for basically the same situation, making the whole thing feel sloppy, even when the logic underneath is solid. We should clean those up and make them consistent, calm, and predictable.

Hence, anything shown to users should use the full, long flag name instead of short flags. I mean they are great for typing a command, but they are terrible for doc or error messages. If the error tells user to do something, it should spell out exactly what they need to do. Users be like "wtf is -duc? -esc?" and then they have to go look it up, which is a bad experience. If it says "you need to enable [INSERT SOME LONG FLAG NAME]" then they know exactly what to do.

I also don't think we should twist ourselves into knots to keep compatibility for internal names that are not part of official lib SDK-facing API. If no external user is supposed to be consuming it, hanging onto the old name just weighs the code down. Compatibility should protect real users, not preserve technical debt forever.

And, currently, [I'm still not sure about this but, for the shake of future-proofing,] a capability needs a certain flag, the capability definition should be the single source of truth for that flag name. So hardcoding the same flag string in messages, stats, checks, everywhere else is just begging for things to get out of sync. The next time we rename a flag or add a new type, we shouldn't have to go on a grep hunt.

Proof

Checklist

  • Pull request is created against the dev branch
  • All checks passed (lint, unit/integration/regression tests etc.) with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features

    • Added capability-based template execution gating (headless, code, DAST, self-contained, global matchers, file).
  • Improvements

    • Standardized template metrics with more granular syntax/runtime warning and skip/exclusion counters.
    • Updated workflow subtemplate filtering and unsigned/unverified skipping behavior to align with capabilities.
    • Improved missing-capability reporting and event-logging behavior.
  • Bug Fixes

    • Fixed a potential file-handle leak during disk matching.
    • Made fuzzing detection resilient to nil request entries.
  • Tests

    • Added/expanded tests for capability gating, workflow parsing, and skip-stat correctness.

@coderabbitai

coderabbitai Bot commented Jun 21, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 6baf00d0-a419-49eb-a1ac-0ba8063d1fbc

📥 Commits

Reviewing files that changed from the base of the PR and between a0e4eca and ed759b7.

⛔ Files ignored due to path filters (3)
  • pkg/templates/tests/workflow-capability-gates.yaml is excluded by !**/*.yaml
  • pkg/templates/tests/workflow-file-template.yaml is excluded by !**/*.yaml
  • pkg/templates/tests/workflow-self-contained-template.yaml is excluded by !**/*.yaml
📒 Files selected for processing (14)
  • internal/runner/runner.go
  • pkg/catalog/disk/find.go
  • pkg/catalog/loader/loader.go
  • pkg/catalog/loader/loader_test.go
  • pkg/templates/capability.go
  • pkg/templates/compile.go
  • pkg/templates/compile_test.go
  • pkg/templates/parser.go
  • pkg/templates/parser_stats.go
  • pkg/templates/stats.go
  • pkg/templates/templates.go
  • pkg/templates/templates_test.go
  • pkg/templates/templates_utils.go
  • pkg/templates/workflows.go
✅ Files skipped from review due to trivial changes (3)
  • pkg/templates/parser.go
  • pkg/templates/templates_utils.go
  • pkg/templates/parser_stats.go
🚧 Files skipped from review as they are similar to previous changes (8)
  • pkg/catalog/disk/find.go
  • pkg/templates/stats.go
  • pkg/templates/workflows.go
  • internal/runner/runner.go
  • pkg/templates/templates_test.go
  • pkg/templates/compile.go
  • pkg/templates/templates.go
  • pkg/templates/compile_test.go

Walkthrough

Introduces a Capability type and CapabilitySet abstraction for template feature-gating (headless, code, DAST, self-contained, global-matchers, file), replacing scattered types.Options field checks in compile, loader, workflow, and runner paths. Stats constants are renamed to canonical Template*/Skipped*/Excluded* forms with deprecated backward-compatible aliases. Adds resource cleanup to disk catalog file opening.

Changes

Template Capability System and Stats Refactor

Layer / File(s) Summary
Capability type, registry, and Template methods
pkg/templates/capability.go
Defines Capability string type, six exported constants (headless, code, DAST, self-contained, global-matchers, file), capabilityDefinitions registry (stat/flag/kind/enable/require predicates per capability), AllCapabilities(), CapabilitySet, CapabilitiesFromOptions(), and Template.RequiredCapabilities() / MissingCapabilities() / MissingLoadCapabilities() with self-contained and global-matchers detection helpers.
Stats constants rename and deprecated aliases
pkg/templates/parser_stats.go, pkg/templates/stats.go
Adds canonical TemplateSyntaxErrorStats, TemplateSyntaxWarningStats, TemplateRuntimeWarningStats, SkippedUnverifiedTemplateStats, SkippedUnverifiedCodeTemplateStats, SkippedRequestSignatureTemplateStats, and capability-specific ExcludedHeadlessTemplateStats / ExcludedCodeTemplateStats / ExcludedSelfContainedTemplateStats / ExcludedGlobalMatchersTemplateStats / ExcludedWeakMatcherTemplateStats / ExcludedFileTemplateStats / ExcludedDASTTemplateStats constants; redirects all deprecated constants to the new names as aliases; refactors init() to register stats from a templateStatEntries slice.
Compile, IsEnabledFor, and IsFuzzableRequest capability integration
pkg/templates/compile.go, pkg/templates/templates.go, pkg/templates/templates_utils.go
compileProtocolRequests derives a CapabilitySet and gates headless/code compilation via caps.Has(...); isGlobalMatchersEnabled delegates to requiresGlobalMatchers(); IsEnabledFor delegates to MissingCapabilities() + toCapabilitySet(); nil-guards added to IsFuzzableRequest() for HTTP and headless request iteration; deprecated notices added to Requirements / Capabilities types and Requirements() method.
Parser stat key updates
pkg/templates/parser.go
Switches LoadTemplate and LoadWorkflow stat increments from deprecated SyntaxErrorStats / SyntaxWarningStats to canonical TemplateSyntaxErrorStats / TemplateSyntaxWarningStats.
Workflow subtemplate capability gating
pkg/templates/workflows.go
parseWorkflowTemplate computes a CapabilitySet, skips subtemplates with missing load-blocking capabilities via noteMissingCapabilities, updates skip counters to canonical identifiers (SkippedUnverifiedTemplateStats, SkippedUnverifiedCodeTemplateStats, SkippedRequestSignatureTemplateStats), and removes explicit HasCodeRequest / EnableCodeTemplates branching.
Loader capability gating and weak-matcher exclusion tracking
pkg/catalog/loader/loader.go
Adds noteMissingCapabilities helper; updates LoadTemplateTags, LoadTemplatesOnlyMetadata, and LoadTemplatesWithTags to compute caps via CapabilitiesFromOptions(), gate by MissingLoadCapabilities(), track weak-matcher exclusions via ExcludedWeakMatcherTemplateStats, and use canonical request-signature skip stat.
Runner stat key and display updates
internal/runner/runner.go
RunEnumeration validation gate and displayExecutionInfo switch to canonical TemplateSyntax* / TemplateRuntimeWarningStats keys; excluded-template warnings loop via AllCapabilities() instead of a fixed set.
File descriptor cleanup in disk catalog
pkg/catalog/disk/find.go
findFileMatches defers file handle close to ensure resource cleanup regardless of early exit paths.
Capability, stats, compile, and loader tests
pkg/templates/templates_test.go, pkg/templates/compile_test.go, pkg/catalog/loader/loader_test.go
Adds tests for CapabilitiesFromOptions(), deprecated stat aliases, AllCapabilities() ordering, capability metadata (Stat(), Flag(), TemplateKind(), MissingFlagMessage()), MissingCapabilities() across request scenarios, MissingLoadCapabilities() for global-matchers, deprecated Requirements() / IsEnabledFor() wrapper behavior, IsFuzzableRequest() nil guard, workflow capability-gate fixture parsing, and unsigned-code-template skip stat isolation in both compile and loader paths.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • projectdiscovery/nuclei#7452: Both PRs change pkg/catalog/loader/loader.go to alter how templates excluded by .nuclei-ignore/exclude-tags are counted and logged.

Suggested reviewers

  • dogancanbakir
  • Mzack9999

Poem

🐇 Hoppity-hop through the capability gate,
Each template now tells me what features it needs!
No more scattered flag-checks — just Has() and Stat(),
The weak-matcher misses get tracked to the seed.
Old constants still work, they just alias the new,
A cleaner nuclei garden for me and for you! 🌿

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 13.64% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely describes the main refactoring: centralizing opt-in capability gates into a shared capability system.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch dwisiswant0/refactor/templates/centralize-opt-in-capability-gates

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/templates/stats.go`:
- Around line 36-38: In the ExcludedSelfContainedTemplateStats description
field, replace the short flag reference `-esc` with the appropriate long flag
name to maintain consistency with the long-flag messaging standard and ensure
the user-facing message aligns with the actual capability metadata for
self-contained template execution.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0f3a3751-2b51-4aa3-b4c9-6d2a2a2dfc98

📥 Commits

Reviewing files that changed from the base of the PR and between a86c4fa and d566585.

⛔ Files ignored due to path filters (3)
  • pkg/templates/tests/workflow-capability-gates.yaml is excluded by !**/*.yaml
  • pkg/templates/tests/workflow-file-template.yaml is excluded by !**/*.yaml
  • pkg/templates/tests/workflow-self-contained-template.yaml is excluded by !**/*.yaml
📒 Files selected for processing (13)
  • internal/runner/runner.go
  • pkg/catalog/loader/loader.go
  • pkg/catalog/loader/loader_test.go
  • pkg/templates/capability.go
  • pkg/templates/compile.go
  • pkg/templates/compile_test.go
  • pkg/templates/parser.go
  • pkg/templates/parser_stats.go
  • pkg/templates/stats.go
  • pkg/templates/templates.go
  • pkg/templates/templates_test.go
  • pkg/templates/templates_utils.go
  • pkg/templates/workflows.go

Comment thread pkg/templates/stats.go Resolved
@dwisiswant0
dwisiswant0 force-pushed the dwisiswant0/refactor/templates/centralize-opt-in-capability-gates branch from d566585 to a0e4eca Compare June 21, 2026 20:09
Move template execution requirements into a shared
capability table that maps each capability to its
flag, stats key, and template predicate. Use the
shared model from template loading, workflow
parsing, request compilation, and runner stats
display so file, self-contained, headless, code,
DAST, and global matcher gates stay in sync.

Signed-off-by: Dwi Siswanto <git@dw1.io>
@dwisiswant0
dwisiswant0 force-pushed the dwisiswant0/refactor/templates/centralize-opt-in-capability-gates branch from a0e4eca to ed759b7 Compare June 21, 2026 22:48
@dwisiswant0
dwisiswant0 requested a review from Mzack9999 June 23, 2026 02:31
@dwisiswant0

Copy link
Copy Markdown
Member Author

@coderabbitai create an issue for this.

@coderabbitai

coderabbitai Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

gh issue create \
  --repo "projectdiscovery/nuclei" \
  --title "Follow-up: Address vocabulary inconsistencies, user-facing message quality, and capability flag naming" \
  --body "## Overview

This issue tracks follow-up improvements identified during the capability gates refactor in `#7489`. The refactor centralized capability definitions into a shared table, but several broader code quality and UX concerns were noted that are out of scope for that PR.

---

## 1. Vocabulary inconsistency

Terms like **\"signed\"**, **\"tampered\"**, **\"verified\"**, **\"skipped\"**, and **\"excluded\"** are used interchangeably across the codebase despite having distinct meanings. This contributes to confusion in code, documentation, and observable behavior.

**Goal:** Establish clear, consistent definitions for each term and apply them uniformly across the codebase, docs, and user-facing output.

---

## 2. User-facing message quality

User-facing messages (warnings, errors, skip notices) lack consistency in wording, tone, and spelling across similar scenarios. Messages appear ad-hoc rather than intentional.

**Goal:** Audit and standardize user-facing messages for consistency in tone, spelling, and format.

---

## 3. Flag naming in user-facing messages

User-facing output currently uses short CLI flags (e.g., \`-c\`) in error messages and skip notices. Short flags are efficient for command-line typing but create confusion in error messages and documentation.

**Goal:** Replace short flags with full long flag names (e.g., \`--code\`) in all user-facing messages to provide clear, actionable guidance.

---

## 4. Single source of truth for capability flag names

Capability flag names are hardcoded in multiple places (messages, stats, checks). This increases the risk of drift when flags are renamed or refactored.

**Goal:** Ensure capability definitions (introduced in `#7489`) are the single authoritative source for associated flag names, so that renaming a flag only requires a change in one place.

---

## References

- PR `#7489` — Refactor template capability gates into a centralized capability table

/cc `@dwisiswant0`" \
  --assignee "dwisiswant0"

Length of output: 2394



@dwisiswant0: Done! I've created #7497, tracking the four follow-up concerns identified in this PR:

  1. Vocabulary inconsistency — standardizing "signed", "tampered", "verified", "skipped", "excluded"
  2. User-facing message quality — consistent wording, tone, and spelling
  3. Flag naming in messages — using full long flag names instead of short flags
  4. Single source of truth — capability definitions as the authoritative reference for flag names

The issue has been assigned to you and links back to this PR.

@dwisiswant0 dwisiswant0 linked an issue Jun 23, 2026 that may be closed by this pull request
@Mzack9999
Mzack9999 merged commit 638be44 into dev Jun 23, 2026
19 checks passed
@Mzack9999
Mzack9999 deleted the dwisiswant0/refactor/templates/centralize-opt-in-capability-gates branch June 23, 2026 20:21
social4hyq pushed a commit to social4hyq/homebrew-core that referenced this pull request Sep 20, 2026
nuclei 3.10.0

Created-by: HarmonybrewBot
Commit-by: HarmonybrewBot
Merged-by: HarmonybrewBot
Description: Created by `brew bump`

---

Created with `brew bump-formula-pr`.<details>
  <summary>release notes</summary>
  <pre>## What's Changed

### 🎉 New Features
* Added per-host HTTP client pooling by @Mzack9999 in projectdiscovery/nuclei#7301

### 🐞 Bug Fixes
* Fixed handling in hosterrorscache to automatically skip hosts that consistently time out by @knakul853 in projectdiscovery/nuclei#7455
* Fixed preservation of explicit target port in network templates (fixes #7323) by @XananasX7 in projectdiscovery/nuclei#7465
* Fixed connection reuse and improved port pre-flight handling by @Mzack9999 in projectdiscovery/nuclei#6715
* Fixed code template signature validation before DAST loading by @dogancanbakir in projectdiscovery/nuclei#7472
* Fixed gating of MySQL allowAllFiles option to require `-lfa` flag by @dogancanbakir in projectdiscovery/nuclei#7473
* Fixed ASCII-section regex to properly escape literal `.` by @snicket2100 in projectdiscovery/nuclei#7476
* Fixed recording of decoded bytes for debug dumps by @snicket2100 in projectdiscovery/nuclei#7478
* Fixed proper escaping of dbname in lib/pq URLs by @dwisiswant0 in projectdiscovery/nuclei#7479
* Fixed network policy enforcement prior to LDAP dialing by @dwisiswant0 in projectdiscovery/nuclei#7494
* Fixed normalization and rejection of trace file DSN options in Oracle by @dwisiswant0 in projectdiscovery/nuclei#7480
* Fixed proper escaping of MSSQL database names in connection URLs by @dwisiswant0 in projectdiscovery/nuclei#7481
* Fixed krbforge to reject unsandboxed ccache writes by @dwisiswant0 in projectdiscovery/nuclei#7482
* Fixed rejection of request-condition(s) during fuzzing by @dwisiswant0 in projectdiscovery/nuclei#7466
* Fixed: YAML now correctly rejects recursive include chains by @dwisiswant0 in projectdiscovery/nuclei#7492
* Fixed resource leaks by @Mzack9999 in projectdiscovery/nuclei#7502

### Other Changes
* Updated govaluate dependency to prevent slice-bounds panic on invalid UTF-8 input by @XananasX7 in projectdiscovery/nuclei#7464
* Updated goja dependency by @Mzack9999 in projectdiscovery/nuclei#7467
* Updated dependencies to remove unused packages by @dwisiswant0 in projectdiscovery/nuclei#7457
* Refactored templates to centralize opt-in capability gating by @dwisiswant0 in projectdiscovery/nuclei#7489
* Added fuzzing parser harnesses for raw requests and templates by @dwisiswant0 in projectdiscovery/nuclei#7459
* Refactored template rendering boundary by @dwisiswant0 in projectdiscovery/nuclei#7499

## New Contributors
* @XananasX7 made their first contribution in projectdiscovery/nuclei#7465
* @snicket2100 made their first contribution in projectdiscovery/nuclei#7476

**Full Changelog**: https://github.com/projectdiscovery/nuclei/compare/v3.9.0...v3.10.0</pre>
  <p>View the full release notes at <a href="https://github.com/projectdiscovery/nuclei/releases/tag/v3.10.0">https://github.com/projectdiscovery/nuclei/releases/tag/v3.10.0</a>.</p>
</details>
<hr>

See merge request: Harmonybrew/homebrew-core!13187
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[MAINTENANCE] Capability gates refactor

2 participants