Skip to content

ISS-14112: Default Codex workers to gpt-5.6-sol from one source - #431

Closed
shafty023 wants to merge 21 commits into
mainfrom
vibe/iss-14112-codex-model
Closed

shafty023 wants to merge 21 commits into
mainfrom
vibe/iss-14112-codex-model

Conversation

@shafty023

@shafty023 shafty023 commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

vibe started every Codex writer, reviewer, verifier and operational worker, and every cl-sweep ticket owner, on gpt-6-sol, while the evals policy and the fleet configuration said gpt-5.6-sol. Operator rulings (2026-11-12): "why are we using gpt-6-sol? we are supposed to be using gpt-5.6-sol" and "the models should be defaults, not mandatory like we have them".

The default moved, and has one source:

  • The default Codex model for those roles and for cl-sweep ticket owners is now gpt-5.6-sol. Explorers stay on gpt-5.6-terra, mechanical support on gpt-6-luna.
  • plugins/vibe/evals/codex/model-policy.json (codexModel) is the one source. The evals runner and the nightly policy already read it; skills/cl-sweep/scripts/model-policy.mjs now reads it too instead of carrying its own literal, so the vibe agents' Codex binding (worker-model.mjs codex <name>) and the cl-sweep session scripts follow it.
  • Production code therefore depends on a file under evals/. It ships in the plugin payload (the payload rule in CLAUDE.md excludes package manifests, lockfiles, node_modules and TypeScript sources, not evals/). When the file is missing, unreadable, not JSON or has no usable codexModel, loading the policy is refused with eval-codex-model-policy-invalid. No model is built in as a fallback.

cl-sweep ticket owners honour the per-machine override:

  • A new session starts on, in order: the model given at init (App Server path, and the generic orchestrator's explicit model), the Codex writer model in ~/.codex/vibe/models.json, then the default.
  • --requested-model (App Server path) accepts any compact model identifier. It used to accept only the one required model. The CLI fallback has no init model option; it takes the override, else the default.

Saved sessions, one rule (this changes asserted wording and existing assertions):

  • A saved ticket-owner session is never refused for its model. A turn already in flight finishes on the saved model, and the session file is never rewritten.
  • When the saved model differs from the model that resolves for that session, the session is replaced at its next idle boundary before another turn. A model explicitly requested at init resolves to itself; every other session resolves to the Codex writer override, else the default. This is the path gpt-5.5 sessions already used; status reports modelReplacement with the resolved model as target and a new turn is refused with the existing idle-boundary sentence.
  • So a session saved on gpt-6-sol stays readable and recoverable, finishes its in-flight turn, and is replaced at idle. A session whose saved model equals the resolved model (for example gpt-5.6-terra set as the override) is not replaced.
  • A session started with an explicit --requested-model keeps it: the session file carries requestedModelExplicit: true, the session runs turns on that model and is not replaced because the default or the override differs. The marker is written only then; a session without it, which is every session saved by earlier code and every CLI fallback session, is treated as having no explicit request. Earlier readers ignore unknown keys in a session file, so the marker is additive.
  • A new lease generation re-attached to an earlier thread is initialized with that thread's saved model as --requested-model, so a thread saved on gpt-6-sol re-attaches and stays on it.
  • This replaces the earlier rule that treated gpt-5.5 and every gpt-5.6-* model as legacy and refused any other saved model outright. Without it, moving the default alone would have made every saved gpt-6-sol session fail validation on every command.

Init and status hardening:

  • The model a new session resolves is checked before the session file is written, on the App Server path and the CLI fallback. An override the session format cannot store (for example ft:gpt-5.6-sol:org) is refused at init with the existing "must be a compact model identifier" sentence instead of leaving a session that fails every later command.
  • status still answers when the per-machine overrides file cannot be read: modelReplacement is null and a new modelReplacementError field carries the error's message.
  • One model-name validator (validateModelName in model-policy.mjs) serves all three session scripts, and the vibe-bench eval policy takes the model from model-policy.mjs instead of reading the policy file a second time.

For a human to look at:

  • The CLI fallback now refuses a session that must be replaced before it marks a pending callback correction, so the session file is left byte for byte.
  • The policy file is read when model-policy.mjs loads, so a missing or invalid file stops every command that imports it (worker-model.mjs claude, set and unset, both session scripts' status, supervise and reconcile, and sweep-root-state.mjs) with the clear refusal. That follows from "refuse, never fall back"; resolving the file lazily is possible if wanted.

Native transport: the generic orchestrator no longer refuses a new gpt-5.5 session (operator ruling, 2026-11-13: "remove it"), so an explicit or override model works the same on all three transports. The refusal dated from #216 with no recorded reason.

Other transports: the generic orchestrator and the fleet session script have no replace-at-idle rule: a model given at launch is saved and used for every later turn, so an explicit model is already honoured there; the marker applies only to cl-sweep ticket-owner sessions.

Renames and removals:

  • LEGACY_COMPATIBLE_WORKER_MODEL is removed from app-server-worker-session.mjs; only tests imported it.
  • app-server-worker-session.mjs exports DEFAULT_WORKER_MODEL in place of REQUIRED_WORKER_MODEL; its probe output field is defaultWorkerModel in place of requiredWorkerModel. No other file read either name.
  • isLegacyTicketOwnerModel is removed; ticketOwnerModel and ticketOwnerNeedsReplacement in model-policy.mjs replace it.
  • CodexWorkerRole moves to skills/vibe/scripts/agent-models.mjs, which gains resolveRoleModel. The Claude launcher bundle inlines agent-models.mjs, so this keeps the policy file read out of the bundle; the bundle is rebuilt and only dist/claude-worker.mjs changed.

Docs: the 1.1.87 CHANGELOG.md entry was regenerated with /update-documentation after the code was final. That command's README step starts a background agent, which the session's writer cannot start, so the README was verified by hand: the skill and agent counts (17 each) match disk and the version line reads 1.1.87. The cl-sweep skill, its execution-surfaces.md and cli-worker-protocol.md references, the cl-execute plan-review and support-lane references, the Codex harness notes and the README name gpt-5.6-sol and state the saved-session rule in place of the legacy model list. The README says cl-sweep ticket-owner sessions use the Codex writer model, including an override.

Existing assertions that changed, and why:

  • Default gpt-6-sol to gpt-5.6-sol, and gpt-5.6-sol dropped from the refused and legacy lists (first ruling above).
  • "--requested-model must be the required model" and "requestedModel must be the required or a legacy model" replaced by the model-name check and the one saved-session rule (second ruling above).
  • The cl-sweep contract test now asserts the rule wording and that the docs no longer name the legacy set.

No check refuses a reviewer for its model: review-receipts.mjs compares a receipt's model with its own trace only, and the handoff gate has no model check. The cloud import review rule in plugins/vibe-core is a separate rule and is untouched.

Versions: vibe 1.1.99, above #426's 1.1.97 and #433's 1.1.98. The built plugins/vibe-closedloop takes the same version from its build. The provider and vibe-core are not changed.

Stacked on #426 (operator ruling, 2026-11-16): every vibe bug-fix PR stacks on #426, so this PR's base is feat/iss-13446-generic-vibe and it merges #426's head e316b80 (no rebase). Nothing moved to the provider; the ticket ids this branch adds are in exempt cl-sweep test support. The built plugin now keeps evals/codex/model-policy.json, the one default source cl-sweep and worker-model read, as one exception to the build's evals/ exclusion (tools/vibe-closedloop-build/src/build.ts, pinned in build.test.ts); plugins/vibe-closedloop is rebuilt and the vibe-workers bundles reproduce. worker-model AC-1 now copies every module its code imports into its temporary plugin. After the stack, the CHANGELOG word line names all plugin Markdown and states +121 words (orchestrator answer: shipped as recorded). Local checks on the stacked tree pass; main-sync.test.ts timed out at 20 s under load (file untouched by this branch), left to CI.

Files shared with open pull requests: #426 touches CHANGELOG.md only.

Test plan

On the feature tree, one command at a time through the fleet heavy slot:

  • Nine node test files under plugins/vibe/skills (model binding, the three cl-sweep session scripts, the doc contract, four sharing their sources): 175 pass.
  • plugins/vibe/tools/python (610 pass), evals/codex/test_run.py (51 pass), ruff check, pyright.
  • tools/vibe-workers build, typecheck, launcher tests (63 pass), and a rebuild that leaves the committed bundles unchanged; tools/vibe-bench typecheck, plugin validation, full suite (199 pass); tools/plugin-manifests typecheck and tests.

After merging main (8a858bb) and the gpt-5.5 change, on the final tree: the version-pin Python tests, changelog word count, instruction-words, the six cl-sweep and worker-model test files, vibe-triage, plugin-version, targeted-tests-ruling, pr-facts, bench typecheck, reply-checker and validate:plugins, and the manifests tests all pass.

Red on base 80a0585: all five changed test files fail with only the new tests laid over the base. Every new or changed assertion has a mutation proof (35 in all, each restored with a verified tree hash), including restoring the gpt-5.5 refusal, which fails both native AC-2 tests.

Not run locally: e2e (no browser or Desktop surface), the whole uv run pytest plugins/ tree and node lane, and the full tools/vibe-workers suite after one run with three main-sync timeouts under load; CI runs them.

Vibe gate override: ISS-14127: the handoff inventory refuses only with rule duplicate-helper at plugins/vibe/skills/vibe/scripts/dist/claude-worker.mjs:12592, "new helper resolveRoleModel duplicates plugins/vibe/skills/vibe/scripts/agent-models.mjs"; that copy is the generated launcher bundle built from agent-models.mjs, and every other gate check and review passed.

Move every ClosedLoop and symphony-alpha specific line out of vibe and
vibe-core into the new optional vibe-closedloop-provider plugin, which ships
text and data only. With the provider bound, sessions keep today's behavior;
without it, vibe and fleet work from the repository itself: the person's
description, plans in the private git directory, the AGENTS.md test command,
the lockfile install, and a requested PR or named recipient at the end.

- provider binding, provider text assembly and data access in vibe-core
- guards read their rules from the bound provider and stay silent outside
  an activated chat
- main sync gates its provider-only machinery on the provider's recipe
- the recipes opt-in file is validated and withholds recipes with one reason
- the provider is offered for install once, never re-enabled or reinstalled
- generic scan gate, generic walk fixture, eval arms, and generic twins for
  the provider-bound tests
- vibe 1.1.81, vibe-core 1.0.5, vibe-closedloop-provider 1.0.0
- writer ledgers parse capabilities by shape; grants are checked against the
  session binding at register and launch
- the Claude launcher resolves provider text for its roles
- writers inherit the lead's binding through a Provider line, enforced by the
  Codex launcher, the Claude launcher and the Agent guard
- the recipes opt-in file is validated and judged in the session repository
- adopting a session keeps its binding; moved provider releases re-resolve
- every installed provider's plan folder stays private
- generic fleet recovery ids and duties, the worktree exclude rule, the
  guardrails formatter from the verify recipe
- dangling reference paths fixed, provider text free of ticket ids and home
  paths, the inventory snapshot and mapping checked in pytest
- the eval provider arm installs provider-resolved text
- bundles build byte-identically from any working directory
Bind the tracker in the tracker-duty tests, run spawned scripts outside the
real checkout so the cwd repository leg cannot pick it, derive the provider
line in fleet tests from the fixture binding, fix the shadowed prototype
deploymentHost check and rebuild dist, list provider.mjs as a vibe-core
wrapper, and record the changelog word count.
Pass the lead's binding to remote fleet writers, read the recipes opt-in
from the main checkout for older sessions and for a named checkout, know
UI files by extension without an areas recipe, never write a legacy
provider record generic, keep old hook scope records active, restore the
exact ticket pattern for Prior removal, keep clerkOrgId for older vibe,
report a main sync with no check as needing review, require the sole
workflow for an undeclared lane source, keep generic plans in the common
git dir, move the repository E2E text into provider blocks, and stop
offering an install over an unreadable provider.
…c-vibe-v2

# Conflicts:
#	CHANGELOG.md
#	plugins/vibe/.claude-plugin/plugin.json
#	plugins/vibe/.codex-plugin/plugin.json
…ider

- provider.mjs assemble writes a vibe file merged for the session's
  binding, and assemble --blocks gives every bound block of a loaded
  SKILL.md in one call; the reading rules use them
- each generic region and its provider block share their identical lines
  once, so assembled text is unchanged and the provider is smaller
- only an unbound capability falls back to the generic text: a bound
  provider that is gone, a missing block or an unreadable record stops
  with its reason and fix (exit 4) in the CLI, the launcher, every script
  and the hooks, and a guard refuses the action it guards
- session records are written whole and renamed into place
- the 1.1.82 word count is measured at this tree
ISS-14039 (#427) took vibe 1.1.82 and vibe-core 1.0.5. Keep both sides of
the claude-worker launcher (the broken-provider mapping and the review
model), rebuild the bundles, drop the ticket slugs the merge brought into
runtime files, and remeasure the word count against the new main.
ISS-14028 and ISS-14024 (#428) took vibe 1.1.84 and vibe-core 1.0.7. The
unit-status guard keeps the bound provider's rule and gains the chat's
own program board; its tests keep both cases. The fleet-resume header
covers both changes, the bundles are rebuilt, the ticket slugs the merge
brought into runtime comments are dropped, the launcher mark written into
installed launchers stays as an allowlisted compatibility line, and the
word count is remeasured against the new main.
The evals policy file is the one source for the default Codex model; cl-sweep, the vibe agent binding and vibe-bench read it. cl-sweep ticket owners honour the per-machine writer override, and a saved session on another model is replaced at its next idle boundary instead of refused. vibe 1.1.87.
Brings in ISS-14077 (#430). vibe stays 1.1.87 on top of 1.1.86.
Operator ruling 2026-11-13: the generic orchestrator no longer refuses a new gpt-5.5 session, so an explicit or override model works the same on all three transports. Removes the two constants only that refusal used. CHANGELOG regenerated with /update-documentation.
…e check

A bound session now behaves like main. The start is one version check when the
provider skill's description names its calls, and --closedloop/--graph are
accepted again as aliases of --connection; otherwise the read-only provider
probe runs first, and a check with no states records unprobed instead of
binding generic. Each capability binds on its own connection, so a missing
graph keeps the ClosedLoop text and restores the graph. The check writes the
bound text once per repository and binding, and readers open files from that
folder instead of assembling each one. check always exits 0 and prints the
setup block in every state.

Also: precedent stays live-only, a brief without a Provider: line inherits the
lead's binding, the preflight finds the checkout as main did when bound,
vibe-bench binds through the real check, and the eval runner gains a
provider-installed arm.
…-generic

- tools/vibe-closedloop-build assembles plugins/vibe plus the provider into
  the committed plugins/vibe-closedloop (the `vibe` entry), with cold
  deploy and environment sections in their own files, recipes inline, and a
  CI drift check. plugins/vibe is published as `vibe-generic`.
- Retire runtime provider text: the rule paragraph (two generic sentences
  stay, dropped by the build), provider-text.md, the bind-time folder,
  model-run assemble/block/path/tree, the install offer, provider-install,
  provider-match, the skill-listed flag and the provider skill. Discovery
  reads only the provider bundled in vibe.
- Each install finds its own plugin name and marketplace; with both
  installed the ClosedLoop vibe runs the hooks and binds, and the check
  reports both-installed.
- Eval arms install the built folder (with the repository opt-in file in the
  case folder) and plugins/vibe as `vibe`; no harness assembly.
- Resolve run.py's private homes and every main guard to real paths.
…the word-count entry

- provider.mjs knownProviderContracts lists the provider bundled in every
  vibe install or checkout, enabled or not; local-plans uses it so a plan
  written while bound stays private when the session is not, whichever vibe
  the lookup order picks. Binding and lookup orders are unchanged.
- The vibe CHANGELOG entry names the full Markdown scope of its word count
  again, with the generated folder's exclusion stated after it.
… 1.1.97

- The build places the provider's whole-file references (ticket-template,
  closedloop-graph, ticket-units) at main's paths in plugins/vibe-closedloop
  and rewrites provider: names to main's relative forms; provider/ holds only
  provider.json, whose placed map providerPath resolves through. A build test
  requires every Markdown file reference in the built tree to resolve.
- vibe 1.1.97, vibe-core 1.0.13 (VIBE_CORE_MIN raised to it); #432's new
  cloud code follows the branch's generic rules.
Merge feat/iss-13446-generic-vibe (e316b80) per the stack-on-#426 ruling.
The built plugin keeps evals/codex/model-policy.json, the one default source cl-sweep reads; plugins/vibe-closedloop rebuilt.
worker-model AC-1 copies every module its code imports.
@shafty023
shafty023 changed the base branch from main to feat/iss-13446-generic-vibe November 17, 2026 02:36
Base automatically changed from feat/iss-13446-generic-vibe to main November 17, 2026 17:23
shafty023 added a commit that referenced this pull request Nov 18, 2026
Folds #431 (ISS-14112) into this branch: one vibe 1.1.98 release and one changelog entry.

ISS-14136: a reviewer can mark a criterion as having no surface, with a reason, and the audit then asks for a named test row instead of an app run.
ISS-14127: the duplicate-helper rule skips the launcher bundle built from the flagged source.
ISS-14112: Codex workers default to gpt-6.2-sol, with gpt-6.2-astra for planning and review lanes on HIGH tickets, falling back to gpt-5.6-sol when the running Codex does not list the model.
@shafty023

Copy link
Copy Markdown
Collaborator Author

Folded into #433 (one version, one changelog set, one CI, one merge). ISS-14112 ships there at vibe 1.1.101, including the gpt-6.2 model switch. Closing this PR; the branch stays.

@shafty023 shafty023 closed this Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant