Skip to content

Enhance backlog assessment pilot and improve documentation - #491

Merged
robgruen merged 15 commits into
mainfrom
dev/robgruen/typechat_factory
Oct 30, 2026
Merged

robgruen merged 15 commits into
mainfrom
dev/robgruen/typechat_factory

Conversation

@robgruen

Copy link
Copy Markdown
Contributor

This pull request updates repository ownership and workflow configuration to reflect a change in the project owner and to enhance GitHub Actions capabilities. The main changes are the reassignment of ownership from the @microsoft/typechat team to @robgruen, updates to documentation to match this change, and improvements to the backlog assessment workflow by enabling new tools and graders.

Ownership and Documentation Updates:

  • Changed code and documentation ownership from @microsoft/typechat to @robgruen in .github/CODEOWNERS and updated references in .github/copilot-instructions.md to reflect the new project owner. [1] [2]

Workflow Enhancements (backlog-assessment):

  • Added permissions for the github(search_issues) and github(search_pull_requests) tools in .github/workflows/backlog-assessment.lock.yml to expand the available GitHub Actions functionality. [1] [2] [3]
  • Updated job display names and timing eligibility to include a new graders job in the workflow environment variables.
  • Enabled built-in graders (such as loops, tool-failure-count, tool-success-rate, and trajectory-efficiency) in the grader manifest for more comprehensive workflow evaluation.

robgruen and others added 15 commits October 26, 2026 17:15
Add a manually dispatched, report-only backlog assessment with scoped repository reads and shared Copilot guidance. Pin GPT-6.1-sol with max reasoning effort for the main assessment, preserve detector effort defaults, and add strict compilation and contract checks with dated rollout documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…_factory

# Conflicts:
#	AUTOMATION.md

Co-authored-by: robgruen <25374553+robgruen@users.noreply.github.com>
Co-authored-by: robgruen <25374553+robgruen@users.noreply.github.com>
Use plain-language reports while preserving evidence and human review. Raise the main assessment allowance to 400 AI credits, document report retention, and update the generated workflow and contract tests. Detection, permissions, and retention settings remain unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Compare the complete Markdown link destination against the existing TypeChat.NET project URL. This addresses the CodeQL PR comment by ensuring the assertion validates the full destination, not merely link text or a partial match.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve conflicts from the earlier squash merge while preserving the approved assessment limits, regression tests, and documentation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document the successful October 29 run and active execution limits, preserve the original dated draft with provenance and review limits, and record subsequent human merges. Keep control hardening and approval-gated workflows as future work.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolve documentation conflicts in favor of the completed pilot record, retain the original assessment snapshot, and preserve the already-merged Python and website changes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Record the owner-approved switch to @robgruen, restrict model-visible tools, and replace transport-derived tool grades with native-only diagnostics. Document enabled Copilot reviews while preserving existing branch protections, budgets, and secrecy restrictions.

Validated with 42 automation tests, reproducible strict compilation, and offline replay of the recorded assessment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 30, 2026 03:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The ownership and approval-policy changes require explicit human owner authorization, while the inaccessible pinned grader runtime leaves its execution contract unverified.

No open issues 🎉

What changed in this PR

This PR updates TypeChat’s project ownership and strengthens the read-only backlog assessment pilot with restricted tool access, native-event diagnostics, and a preserved assessment report.

Changes:

  • Names @robgruen as project owner across policy, contributor guidance, and CODEOWNERS.
  • Restricts model-visible tools and replaces four built-in metrics with three native-event diagnostics, with regression coverage.
  • Archives the October 29 assessment and documents its evidence limits, automation status, and human-review requirements.
File Description
tools/​tests/​backlog-assessment.test.mjs Adds ownership, tool-boundary, grader-isolation, and native-metric checks.
reports/​backlog-assessments/​2026-10-29-40962028077.md Preserves the draft inventory of 21 issues and 25 PRs with coverage limits.
README.md Updates ownership and summarizes pilot progress and review status.
GOALS.md Updates the direction owner and clarifies approval versus enforcement.
AUTOMATION.md Documents run results, archive provenance, diagnostics, and review controls.
.github/​workflows/​backlog-assessment.md Adds tool filtering, native graders, and access-denial guidance.
.github/​workflows/​backlog-assessment.lock.yml Regenerates the workflow with tool restrictions and an isolated graders job.
.github/​copilot-instructions.md Updates the named policy owner.
.github/​CODEOWNERS Assigns policy and automation-control review to @robgruen.

Review effort: Balanced

@robgruen
robgruen merged commit 9a8844c into main Oct 30, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants