Skip to content

fix(validate): fail --strict on requirements over the length limit - #2020

Merged
clay-good merged 2 commits into
mainfrom
fix/requirement-length-strict
Oct 1, 2026
Merged

clay-good merged 2 commits into
mainfrom
fix/requirement-length-strict

Conversation

@clay-good

@clay-good clay-good commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #1976

Status: LGTM. Finishes what #1978 started, including the CI gate before archive and how to split an existing long requirement.

What was wrong: #1976 asked for three things. #1978 shipped two of them: agents are now told about the 500-character limit, and the validator message explains how to split. Two gaps stayed open: the finding was INFO, so openspec validate --all --strict exited 0 and CI had no way to enforce the limit; and there was no documented way to split an existing long requirement (the instruction only said not to split under MODIFIED).

How it was fixed:

Surface Before After
Finding level INFO WARNING
openspec validate --all exit 0 exit 0 (unchanged)
openspec validate --all --strict exit 0 exit 1
openspec validate <change> --strict with an overlong ADDED requirement exit 0 (failed only after archive) exit 1
openspec archive unaffected unaffected (archive validates non-strict)

ADDED requirements in a change get the same warning, measured by the same shared body reader, so a change fails --strict at exactly the length its main spec would after archive. MODIFIED is not checked: its text is the existing requirement, which the instruction says to keep whole.

The specs instruction (and its docs-lab page, which quotes it) now says how to split an existing long requirement in a change made for that purpose: keep the MODIFIED header and every scenario, cut the description to one behavior, and add each removed behavior as its own ADDED requirement.

This is the same normal-passes, strict-fails split the SHALL/MUST keyword warning already uses (cli-validate spec). The specs instruction and its docs-lab page now say "a warning … --strict fails on it" instead of "an informational hint".

Replication / proof:

  • A new test (fails strict validation, but not normal validation, on an overlong requirement) and the updated boundary test both fail on main and pass here.
  • CLI repro with one main spec holding a 537-character requirement: validate --all exits 0; validate --all --strict printed [WARNING] requirements[0]: Requirement text is very long… and exited 1 (it exited 0 before).
  • New delta tests: an ADDED requirement at 501 characters passes normal and fails strict validation (fails on the previous head), 500 passes, and an overlong MODIFIED is not flagged.
  • End to end from dist: a change adding a 501-character requirement exits 1 under validate <change> --strict and still archives; the documented split (MODIFIED keeping its scenarios + two ADDED) validates strict, archives, and the main spec then passes validate --specs --strict.
  • Full suite: 6,390 passed, 2 failed. Both failures (artifact-workflow "creates skills for Cursor tool", config-profile "confirmed project apply") also fail on main. tsc --noEmit and eslint are clean.

Notes / nits:

  • Behavior change for --strict users: projects that already have overlong requirements will see validate --strict start failing after upgrading. That is what the issue asks for, and the message says how to fix it. The changeset calls it out.
  • Not in scope: MODIFIED requirements are not length-checked in the change; an existing overlong requirement is still reported on the main spec.
  • Length is measured in UTF-16 units (pre-existing), so emoji-heavy text counts more than its visible characters.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Validation
    • Descriptions over 500 characters in ADDED requirements and the main spec now trigger a warning. Standard validation remains valid when this is the only finding; strict validation fails. MODIFIED requirements are not subject to this length check.
  • Guidance
    • Spec-writing instructions clarify where the limit applies and how to split a long existing requirement when requested, preserving its header and scenarios while keeping its meaning.

A requirement description over 500 characters was an INFO finding, so
`openspec validate --all --strict` still exited 0 and CI could not hold
the limit. It is now a WARNING: normal validation and archive still pass,
while strict mode fails, the same split the SHALL/MUST keyword warning
already uses. The specs instruction and its docs page say so.

Closes #1976

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@clay-good
clay-good requested review from a team and TabishB as code owners October 1, 2026 17:06
@coderabbitai

coderabbitai Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: Fission-AI/OpenSpec/.coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 5f4cb969-dfd1-427d-a45c-6f8e6bc4dd41

📥 Commits

Reviewing files that changed from the base of the PR and between c3534c1 and d08aa74.

📒 Files selected for processing (6)
  • .changeset/requirement-length-strict.md
  • docs-lab/reference/schemas/spec-driven/index.md
  • schemas/spec-driven/schema.yaml
  • src/core/validation/validator.ts
  • test/core/templates/requirement-length-guidance.test.ts
  • test/core/validation.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • .changeset/requirement-length-strict.md

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

Overlong descriptions in main specs and ADDED requirements now produce warnings. Normal validation remains valid, while strict validation fails on these warnings. Guidance describes how to split an existing requirement when requested.

Changes

Requirement length handling

Layer / File(s) Summary
Requirement-length validation
src/core/validation/validator.ts, test/core/validation.test.ts, .changeset/requirement-length-strict.md
Main-spec and ADDED requirement descriptions over 500 characters produce warnings. Tests cover normal and strict validation, the limit boundary, and the absence of a length warning for MODIFIED requirements.
Requirement authoring guidance
schemas/spec-driven/schema.yaml, docs-lab/reference/schemas/spec-driven/index.md, test/core/templates/requirement-length-guidance.test.ts
The guidance describes strict-validation behavior and explains how to split an existing long requirement when requested, while preserving its header and scenarios.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: ⚪ Minimal · up to d08aa

Overlong main-spec and ADDED requirement descriptions now warn in normal validation and fail strict validation; MODIFIED requirements remain unchecked as intended. No material merge-blocking risk is established.

Security Architecture Review

Security architecture risk: ⚪ Minimal · up to d08aa

The change strengthens strict validation without expanding filesystem access or privileges. Ordinary validation remains non-blocking for these warnings. Existing projects with overlong requirements may need corrections before strict checks pass.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — For the inspected paths, supplied requirement text can now make a strict validation report invalid through its length. The changed checks do not grant access to additional project roots or filesystem operations.

Trust Boundaries and Controls

  • observed — Inspected command callers retain their existing boundary controls: general validation resolves the project or store root and checks identifiers before constructing paths, while legacy change validation checks the change-directory location. The warning changes do not alter these controls.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR satisfies the coding objectives in #1976. applySpecRules reports overlong main-spec requirements as WARNING, so normal validation remains advisory and strict validation can fail. `validateC…
Out of Scope Changes check ✅ Passed The changed files support #1976. The validator implements the warning and strict-mode behavior. The tests verify the behavior and guidance. The schema, documentation, and changeset describe the same r…
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 3…
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: strict validation now fails for requirements over the length limit.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@openspec-cloud

openspec-cloud Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

No PR-relevant drift confirmed.

AI-generated · A citation proves the line exists, not that it makes the case — verify before acting.
No issue was confirmed at c3534c1; provider billing could not be confirmed, so the scan stopped with total cost unknown.
This is not a full-repository clean result; see the check for coverage and any broader findings.
View results · Click Refresh, then Scan again in the check. Or comment /openspec-cloud.

alfred-openspec
alfred-openspec previously approved these changes Oct 1, 2026

@alfred-openspec alfred-openspec left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the validation behavior, regression coverage, changeset, and canonical docs update. Normal validation remains advisory, strict validation now enforces the documented warning, and CI is green.

With the length finding now failing --strict, a change could still add an
overlong requirement and pass `validate <change> --strict`; CI only went red
after archive merged it into the main spec. ADDED requirements now get the
same warning, using the shared body reader so the limit matches the main
spec exactly. MODIFIED is left alone, since its text is the existing
requirement the instruction says to keep whole.

The specs instruction (and its docs-lab page) now says how to split an
existing long requirement in a dedicated change: keep the MODIFIED header and
every scenario, cut the description to one behavior, and add each removed
behavior as its own ADDED requirement. Verified end to end: the split change
validates strict, archives, and the main spec then passes --strict.

Closes the rest of #1976.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@alfred-openspec alfred-openspec left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed the new head. The ADDED-requirement check uses the shared body reader, matches the main-spec 500/501 boundary, preserves normal/archive behavior, and intentionally leaves MODIFIED requirements unchanged. The splitting guidance and regression coverage align with #1976, and the full hosted matrix is green.

@clay-good
clay-good added this pull request to the merge queue Oct 1, 2026
Merged via the queue into main with commit 760584b Oct 1, 2026
17 checks passed
@clay-good
clay-good deleted the fix/requirement-length-strict branch October 1, 2026 20:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Requirement length guidance is not exposed to coding agents or enforced by strict validation

2 participants