Repo upgrade

[Malcolmnixon]

Pull Request

Description

This PR upgrades the repository to the latest agents and scripts.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Code quality improvement

Related Issues

Closes #

Pre-Submission Checklist

Before submitting this pull request, ensure you have completed the following:

Build and Test

• Code builds successfully: dotnet build --configuration Release
• All unit tests pass: dotnet test --configuration Release
• Code produces zero warnings

Code Quality

• Code formatting is correct: dotnet format --verify-no-changes
• New code has appropriate XML documentation comments
• Static analyzer warnings have been addressed

Quality Checks

Please run the following checks before submitting:

• All linters pass: ./lint.sh (Unix/macOS) or cmd /c lint.bat / ./lint.bat (Windows)

Testing

• Added unit tests for new functionality
• Updated existing tests if behavior changed
• All tests follow the AAA (Arrange, Act, Assert) pattern
• Test coverage is maintained or improved

Documentation

• Updated README.md (if applicable)
• Added code examples for new features (if applicable)
• Updated requirements.yaml (if applicable)
• Updated XML documentation comments for changed APIs
• Updated ARCHITECTURE.md (if applicable)

Additional Notes

[Malcolmnixon]

@copilot Please use the lint-fix agent to fix linting issues. Inform it that long pipe-tables that it has a hard time collapsing to fit to the line-length limit can be converted to bullet-lists, or if they need to stay as tables they can be grid-tables.

[Copilot]

Pull request overview

This PR upgrades the repo’s Continuous Compliance tooling and documentation pipeline by standardizing on PowerShell-based build/lint entrypoints, expanding OTS requirements into dedicated files, and updating CI workflows and standards/agent docs accordingly.

Changes:

• Replace lint.sh/lint.bat and build.sh/build.bat with lint.ps1, fix.ps1, and build.ps1, and wire CI to use them.
• Expand OTS requirements from a single docs/reqstream/ots/ots.yaml into per-tool requirement files (including new FileAssert evidence).
• Refresh documentation/standards/agent guidance and update CI workflow steps for documentation generation and evidence capture.

Reviewed changes

Copilot reviewed 56 out of 57 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
requirements.yaml	Updates includes list to point at new per-OTS requirement YAML files.
pip-requirements.txt	Adds yamlfix for YAML auto-fixing support.
lint.sh	Removes old bash lint entrypoint.
lint.ps1	Adds PowerShell lint entrypoint used by CI and agents.
lint.bat	Removes old Windows batch lint entrypoint.
fix.ps1	Adds PowerShell auto-fix script (yamlfix/markdownlint/dotnet format + YAML LF normalization).
docs/user_guide/definition.yaml	Minor YAML cleanup in Pandoc definition.
docs/reqstream/spdx-model/spdx-model.yaml	Refines requirement wording (e.g., optionality/requiredness in API) and justifications.
docs/reqstream/spdx-model/spdx-license-element.yaml	Refines LicenseElement requirement wording and justification.
docs/reqstream/spdx-model/spdx-helpers.yaml	Refines helper requirements wording and justification.
docs/reqstream/spdx-model/spdx-element.yaml	Refines SpdxElement requirement wording and justification.
docs/reqstream/ots/weasyprint.yaml	Adds dedicated WeasyPrint OTS requirements file.
docs/reqstream/ots/versionmark.yaml	Adds dedicated VersionMark OTS requirements file.
docs/reqstream/ots/sonarmark.yaml	Adds dedicated SonarMark OTS requirements file.
docs/reqstream/ots/sarifmark.yaml	Adds dedicated SarifMark OTS requirements file.
docs/reqstream/ots/reviewmark.yaml	Adds dedicated ReviewMark OTS requirements file.
docs/reqstream/ots/reqstream.yaml	Adds dedicated ReqStream OTS requirements file.
docs/reqstream/ots/pandoc.yaml	Adds dedicated Pandoc OTS requirements file.
docs/reqstream/ots/ots.yaml	Removes old monolithic OTS requirements file.
docs/reqstream/ots/mstest.yaml	Adds dedicated MSTest OTS requirements file.
docs/reqstream/ots/fileassert.yaml	Adds dedicated FileAssert OTS requirements file.
docs/reqstream/ots/buildmark.yaml	Adds dedicated BuildMark OTS requirements file.
build.sh	Removes old bash build entrypoint.
build.ps1	Adds PowerShell build entrypoint for build+test.
build.bat	Removes old Windows batch build entrypoint.
SECURITY.md	Reworks vulnerability reporting guidance and best-practices content.
CONTRIBUTING.md	Updates contributor workflow guidance; documents fix.ps1/lint.ps1 usage and standards.
CODE_OF_CONDUCT.md	Normalizes list formatting and adds an Issues link reference.
AGENTS.md	Updates agent guidance to new scripts, standards matrix, and reporting rules.
.yamlfix.toml	Adds yamlfix configuration aligned to yamllint rules.
.versionmark.yaml	Adds FileAssert to version capture regex list.
.reviewmark.yaml	Adjusts review globs/exclusions and review-set path scopes.
.markdownlint-cli2.yaml	Minor config comment update and noBanner setting documentation.
.gitignore	Minor cleanup/ordering related to docs outputs and agent logs.
.github/workflows/release.yaml	Minor YAML cleanup; adjusts run: scalar style.
.github/workflows/build_on_push.yaml	Minor YAML formatting normalization.
.github/workflows/build.yaml	Switches linters to lint.ps1; adds FileAssert-driven doc validation and restructures doc generation/evidence steps.
.github/standards/testing-principles.md	Clarifies requirement-to-test expectations and permits tests without requirements.
.github/standards/technical-documentation.md	Clarifies recursive subsystem folder patterns and Pandoc collection requirements.
.github/standards/software-items.md	Expands categorization model (adds Software Package) and clarifies placeholder naming conventions.
.github/standards/reviewmark-usage.md	Updates documented ReviewMark invocation and adds --enforce guidance.
.github/standards/requirements-principles.md	Adds explicit requirements principles (unidirectionality, testability, hierarchy).
.github/standards/reqstream-usage.md	Updates ReqStream standards to reference requirements principles and refined structure/semantics.
.github/standards/design-documentation.md	Updates design doc structure to allow recursive subsystem nesting and companion artifact guidance.
.github/standards/csharp-testing.md	Minor wording update to remove “DEMA Consulting” phrasing.
.github/standards/csharp-language.md	Adds reminder to run pwsh ./fix.ps1.
.github/standards/coding-principles.md	Refines literate programming wording and clarifies architecture/traceability rules.
.github/agents/software-architect.agent.md	Adds new software-architect agent definition.
.github/agents/repo-consistency.agent.md	Adds explicit completion-report requirement and updates template wording.
.github/agents/quality.agent.md	Updates quality agent workflow and reporting template to current project practices.
.github/agents/lint-fix.agent.md	Adds new lint-fix agent definition aligned to lint.ps1/fix.ps1.
.github/agents/implementation.agent.md	Updates orchestration state machine (PLANNING/DEVELOPMENT/QUALITY/REPORT) and reporting rules.
.github/agents/formal-review.agent.md	Renames/updates formal review agent guidance and standards references.
.github/agents/developer.agent.md	Updates developer agent workflow to include pre-flight companion artifact planning and fix.ps1 usage.
.fileassert.yaml	Adds FileAssert test catalog used to validate generated HTML/PDF documents and produce TRX evidence.
.cspell.yaml	Updates cspell word list (notably reduced) and ignore paths.
.config/dotnet-tools.json	Adds demaconsulting.fileassert dotnet tool to the manifest.

Comments suppressed due to low confidence (3)

.github/workflows/build.yaml:530

• .github/standards/reviewmark-usage.md now documents dotnet reviewmark --plan ... --enforce, but this workflow still generates the plan/report without --enforce (and even notes it as a TODO). If --enforce is the desired gate to ensure every reviewable file is covered by a review-set, add it here (or update the standard / TODO to clearly explain why enforcement must remain disabled).

      - name: Generate Review Plan and Review Report with ReviewMark
        shell: bash
        # TODO: Add --enforce once reviews branch is populated with review evidence PDFs and index.json
        run: >
          dotnet reviewmark
          --plan docs/code_review_plan/plan.md
          --plan-depth 1
          --report docs/code_review_report/report.md
          --report-depth 1

docs/reqstream/spdx-model/spdx-license-element.yaml:22

• This requirement now asserts that packages/files/snippets carry several license-related fields (concluded license, copyright text, license comments, attribution notices, annotations), but the linked tests primarily validate DeepCopy behavior and only exercise a subset of those properties (e.g., SpdxFile_DeepCopy... doesn’t set/assert any of these fields; SpdxSnippet_DeepCopy... asserts only ConcludedLicense). To make the ReqStream evidence meaningful, either update/add tests to explicitly cover the required fields (and reference them here), or narrow the requirement title to what the existing tests actually demonstrate.
  docs/reqstream/spdx-model/spdx-model.yaml:52
• SpdxModel-Data-NullableReferenceTypes was reworded to an API-design constraint (“explicitly express whether each field is required or optional”), but the linked evidence test (SpdxModel_ReadSpdxJson_Spdx23Example_DeepCopyProducesEquivalentDocument) only checks deep-copy equivalence and doesn’t verify required/optional field semantics. Consider rewriting this requirement into externally observable, testable behavior (e.g., deserialize/validate behavior when optional fields are missing) and linking it to tests that assert those outcomes, or move the API-shaping statement to design documentation instead of ReqStream requirements.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

@copilot Please use the lint-fix agent to fix linting issues. Inform it that long pipe-tables that it has a hard time collapsing to fit to the line-length limit can be converted to bullet-lists, or if...

The lint-fix agent ran and resolved the linting issue — a long line in docs/reqstream/spdx-model/spdx-helpers.yaml was converted to a YAML folded block scalar. Lint now passes cleanly. Committed in 14cec15.

[Copilot]

Pull request overview

This PR modernizes the repo’s compliance/tooling setup by consolidating build/lint automation into PowerShell scripts, expanding OTS requirements coverage, and extending the CI documentation pipeline with FileAssert-based validation and TRX evidence output.

Changes:

• Replace legacy lint.sh/lint.bat and build.sh/build.bat with lint.ps1, fix.ps1, and build.ps1, and wire CI to use them.
• Split monolithic OTS requirements into per-tool requirement files and update requirements/ReviewMark/VersionMark to match.
• Add FileAssert configuration and CI steps to validate generated HTML/PDF documents and emit TRX evidence.

Reviewed changes

Copilot reviewed 56 out of 57 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
requirements.yaml	Updates root requirements includes to reference new per-tool OTS requirement files.
pip-requirements.txt	Adds yamlfix to support YAML auto-formatting in fix.ps1.
lint.sh	Removes legacy bash-based lint entrypoint (replaced by lint.ps1).
lint.ps1	New unified lint script (YAML, npm tooling, compliance tools, dotnet format).
lint.bat	Removes legacy Windows batch lint entrypoint (replaced by lint.ps1).
fix.ps1	New “always exit 0” auto-fix script (yamlfix, markdownlint fix, dotnet format).
docs/user_guide/definition.yaml	Minor YAML cleanup to keep Pandoc definitions tidy/lint-clean.
docs/reqstream/spdx-model/spdx-model.yaml	Adjusts requirement wording for API optionality semantics.
docs/reqstream/spdx-model/spdx-license-element.yaml	Refines license-element requirements to be more behavioral/observable.
docs/reqstream/spdx-model/spdx-helpers.yaml	Refines helper requirements wording (date-time validation and merge preference).
docs/reqstream/spdx-model/spdx-element.yaml	Refines element identity requirement wording.
docs/reqstream/ots/weasyprint.yaml	New OTS requirement file for WeasyPrint with TRX evidence links.
docs/reqstream/ots/versionmark.yaml	New OTS requirement file for VersionMark with evidence links.
docs/reqstream/ots/sonarmark.yaml	New OTS requirement file for SonarMark with evidence links.
docs/reqstream/ots/sarifmark.yaml	New OTS requirement file for SarifMark with evidence links.
docs/reqstream/ots/reviewmark.yaml	New OTS requirement file for ReviewMark with evidence links.
docs/reqstream/ots/reqstream.yaml	New OTS requirement file for ReqStream with enforcement-mode evidence links.
docs/reqstream/ots/pandoc.yaml	New OTS requirement file for Pandoc with FileAssert-backed evidence links.
docs/reqstream/ots/ots.yaml	Removes monolithic OTS requirements file (replaced by per-tool files).
docs/reqstream/ots/mstest.yaml	New OTS requirement file for MSTest with evidence links.
docs/reqstream/ots/fileassert.yaml	New OTS requirement file for FileAssert with self-validation evidence links.
docs/reqstream/ots/buildmark.yaml	New OTS requirement file for BuildMark with evidence links.
build.sh	Removes legacy bash build entrypoint (replaced by build.ps1).
build.ps1	New unified build script for Release build + tests.
build.bat	Removes legacy Windows batch build entrypoint (replaced by build.ps1).
SECURITY.md	Updates and expands vulnerability reporting and policy guidance.
CONTRIBUTING.md	Refreshes contribution guidance and documents the new lint/fix workflow.
CODE_OF_CONDUCT.md	Markdown formatting improvements and adds a reference link for issues.
AGENTS.md	Updates agent guidance to the new lint/fix/build scripts and reporting expectations.
.yamlfix.toml	Adds yamlfix configuration aligned with repository linting expectations.
.versionmark.yaml	Adds FileAssert to version capture configuration.
.reviewmark.yaml	Updates review patterns and review-set paths to match new doc/req layout.
.markdownlint-cli2.yaml	Suppresses markdownlint banner output for cleaner logs.
.gitignore	Minor cleanup/reordering around build notes and agent log ignores.
.github/workflows/release.yaml	Minor YAML hygiene and block-scalar style adjustment.
.github/workflows/build_on_push.yaml	Minor trigger YAML formatting cleanup.
.github/workflows/build.yaml	Switches linting to lint.ps1, restructures doc pipeline, and adds FileAssert TRX generation/usage.
.github/standards/testing-principles.md	Clarifies that every requirement needs evidence, but tests need not map to requirements.
.github/standards/technical-documentation.md	Clarifies Pandoc collection structure and recursive subsystem folder conventions.
.github/standards/software-items.md	Expands categorization (adds “Software Package”) and clarifies naming/path placeholder conventions.
.github/standards/reviewmark-usage.md	Updates ReviewMark guidance (including --enforce guidance) and recursive subsystem patterns.
.github/standards/requirements-principles.md	New requirements principles standard defining flow, quality gates, and test independence.
.github/standards/reqstream-usage.md	Aligns ReqStream guidance with the new requirements principles and structure conventions.
.github/standards/design-documentation.md	Updates design doc structure conventions to allow nested subsystems.
.github/standards/csharp-testing.md	Minor wording cleanup to align with updated standards language.
.github/standards/csharp-language.md	Adds explicit reminder to run the repo’s fix script (pwsh ./fix.ps1).
.github/standards/coding-principles.md	Updates “literate programming” guidance to emphasize intent documentation.
.github/agents/software-architect.agent.md	Adds a new user-invocable software architecture interview agent.
.github/agents/repo-consistency.agent.md	Updates repo-consistency agent to follow new agent reporting requirements.
.github/agents/quality.agent.md	Updates quality agent workflow and reporting template.
.github/agents/lint-fix.agent.md	Adds new lint-fix agent to run fix/lint loops pre-PR.
.github/agents/implementation.agent.md	Updates orchestration state machine and reporting expectations.
.github/agents/formal-review.agent.md	Renames/reframes formal review agent and aligns steps with new standards.
.github/agents/developer.agent.md	Updates developer agent workflow to include pre-flight artifact planning and fix/build usage.
.fileassert.yaml	New FileAssert test definitions for generated documents (HTML/PDF assertions and tagging).
.cspell.yaml	Updates project dictionary word list and ignores.
.config/dotnet-tools.json	Adds demaconsulting.fileassert dotnet tool for CI/local use.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Malcolmnixon]

@copilot The build failed with docs/design/definition.yaml: withBinaryFile: does not exist (No such file or directory)

Please example a sister project such as https://github.com/demaconsulting/FileAssert and use it as a template for the sort of definition.yaml (pandoc project) and title.txt (metadata) needed - and make sure the .fileassert.yaml file has the correct matching content asserts.

[Copilot]

@copilot The build failed with docs/design/definition.yaml: withBinaryFile: does not exist (No such file or directory)

Please example a sister project such as https://github.com/demaconsulting/Fi...

Fixed in commit 363f796. Created the two missing files modelled on the FileAssert sister project:

• docs/design/title.txt: title SpdxModel Software Design, description Software design document for the SpdxModel C# library..., author DEMA Consulting — these match the existing .fileassert.yaml checks (Title contains "Design", Subject contains "design document", Author contains "DEMA Consulting")
• docs/design/definition.yaml: pandoc project definition listing all 23 design .md files in logical order (introduction → model types → io → transform)

[Copilot]

Pull request overview

This PR upgrades the repository’s Continuous Compliance tooling and documentation pipeline by modernizing lint/build entrypoints, expanding OTS requirements coverage, and adding FileAssert-driven document validation.

Changes:

• Replaces legacy lint.sh/lint.bat and build.sh/build.bat with PowerShell-based lint.ps1, fix.ps1, and build.ps1, and wires CI to use them.
• Splits OTS requirements into per-tool files and updates requirements.yaml includes accordingly.
• Adds FileAssert configuration and CI steps to validate generated HTML/PDF outputs (Pandoc/WeasyPrint/document artifacts).

Reviewed changes

Copilot reviewed 62 out of 63 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
requirements.yaml	Updates includes to new OTS req files
pip-requirements.txt	Adds yamlfix dependency
lint.sh	Removes legacy bash linter entrypoint
lint.ps1	Adds unified PowerShell lint runner
lint.bat	Removes legacy batch linter entrypoint
fix.ps1	Adds unified auto-fix script
docs/user_guide/title.txt	Updates User Guide PDF title metadata
docs/user_guide/definition.yaml	Removes stray blank line in inputs
docs/requirements_report/title.txt	Updates Trace Matrix PDF title metadata
docs/requirements_doc/title.txt	Updates Requirements PDF title metadata
docs/reqstream/spdx-model/spdx-model.yaml	Renames/rewrites requirement for optionality
docs/reqstream/spdx-model/spdx-license-element.yaml	Rewords license element requirements
docs/reqstream/spdx-model/spdx-helpers.yaml	Clarifies helper requirements wording
docs/reqstream/spdx-model/spdx-element.yaml	Rewords element identifier requirement
docs/reqstream/ots/weasyprint.yaml	Adds WeasyPrint OTS requirements file
docs/reqstream/ots/versionmark.yaml	Adds VersionMark OTS requirements file
docs/reqstream/ots/sonarmark.yaml	Adds SonarMark OTS requirements file
docs/reqstream/ots/sarifmark.yaml	Adds SarifMark OTS requirements file
docs/reqstream/ots/reviewmark.yaml	Adds ReviewMark OTS requirements file
docs/reqstream/ots/reqstream.yaml	Adds ReqStream OTS requirements file
docs/reqstream/ots/pandoc.yaml	Adds Pandoc OTS requirements file
docs/reqstream/ots/ots.yaml	Removes monolithic OTS requirements file
docs/reqstream/ots/mstest.yaml	Adds MSTest OTS requirements file
docs/reqstream/ots/fileassert.yaml	Adds FileAssert OTS requirements file
docs/reqstream/ots/buildmark.yaml	Adds BuildMark OTS requirements file
docs/design/title.txt	Adds Design doc Pandoc metadata
docs/design/definition.yaml	Adds Design doc Pandoc definition
docs/code_quality/title.txt	Updates Code Quality PDF title metadata
build.sh	Removes legacy bash build entrypoint
build.ps1	Adds unified PowerShell build runner
build.bat	Removes legacy batch build entrypoint
SECURITY.md	Updates vulnerability reporting policy text
CONTRIBUTING.md	Refreshes contribution workflow and tooling
CODE_OF_CONDUCT.md	Normalizes list formatting and links
AGENTS.md	Updates agent guidance and tooling entrypoints
.yamlfix.toml	Adds yamlfix configuration aligned to yamllint
.versionmark.yaml	Adds FileAssert tool version capture
.reviewmark.yaml	Refines review patterns and review-set scoping
.markdownlint-cli2.yaml	Adds clarification comment for banner suppression
.gitignore	Minor ordering/cleanup for ignored docs/artifacts
.github/workflows/release.yaml	Minor YAML cleanup (`run:
.github/workflows/build_on_push.yaml	Normalizes trigger YAML formatting
.github/workflows/build.yaml	Switches CI linting/doc pipeline; adds FileAssert usage
.github/standards/testing-principles.md	Updates test/requirement linkage guidance
.github/standards/technical-documentation.md	Clarifies Pandoc collection requirements
.github/standards/software-items.md	Expands software item taxonomy and guidance
.github/standards/reviewmark-usage.md	Updates ReviewMark usage guidance (--enforce)
.github/standards/requirements-principles.md	Adds new requirements principles standard
.github/standards/reqstream-usage.md	Updates ReqStream structure and command guidance
.github/standards/design-documentation.md	Updates structure to allow nested subsystems
.github/standards/csharp-testing.md	Minor wording normalization
.github/standards/csharp-language.md	Adds pwsh ./fix.ps1 reminder
.github/standards/coding-principles.md	Clarifies intent documentation expectations
.github/agents/software-architect.agent.md	Adds new interactive architecture agent
.github/agents/repo-consistency.agent.md	Updates reporting requirement guidance
.github/agents/quality.agent.md	Refines QA process/reporting template
.github/agents/lint-fix.agent.md	Adds dedicated lint-fix agent
.github/agents/implementation.agent.md	Updates orchestration state machine/docs
.github/agents/formal-review.agent.md	Renames/updates formal review agent guidance
.github/agents/developer.agent.md	Updates developer agent workflow/reporting
.fileassert.yaml	Adds FileAssert document validation suite
.cspell.yaml	Prunes/adjusts custom spelling word list
.config/dotnet-tools.json	Adds demaconsulting.fileassert tool

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Malcolmnixon]

@copilot I thought we'd fixed the metadata problem but the build just failed with File 'D:\a\SpdxModel\SpdxModel\docs/SpdxModel Trace Matrix.pdf' PDF metadata 'Subject' does not contain 'Traceability'. Please review ALL .fileassert.yaml checks against the pandoc title.txt metadata files and make sure the metadata is correct and the asserts will pass.

[Copilot]

@copilot I thought we'd fixed the metadata problem but the build just failed with `File 'D:\a\SpdxModel\SpdxModel\docs/SpdxModel Trace Matrix.pdf' PDF metadata 'Subject' does not contain 'Traceability...

Fixed in commit 5f0f3a7. Reviewed all .fileassert.yaml PDF Subject checks against every title.txt description field — all passed except the Trace Matrix where the description had traceability (lowercase t) but the check required contains: "Traceability" (capital T). Updated docs/requirements_report/title.txt description to Requirements Traceability Matrix... to match.