Skip to content

Harness 3.2 Model picker respects the harness [both repos] #6544

Open

Description

@chelojimenez

Part of #6535.

  • Today a GPT model on a Claude Code client passes preflight and fails after the box boots (BB-245).
  • Filter pickers by harness using the support table.
  • Refuse incompatible models when a client or environment is saved, with an actionable error. The places to add this:
    • ensureHostConfigV2 (convex/lib/hostConfigV2.ts)
    • validateLaunchableModel (convex/projectEnvironments.ts)
  • Make the eval dry run check the environment's model override (checkEvalHarnessStaticAdmission, server/routes/v1/evals.ts ~4916).
  • Regression check only: assistant-turn.ts (~675-705) already refuses an incompatible harness model instead of running the emulated engine. Add or keep a test that pins this behavior.

Delivery rules

  • Fresh worktree and branch; each PR at most 400 handwritten changed lines; stack when necessary.
  • Verify the actual failing boundary and run repository checks.
  • Preserve broker-only model auth, no venue/engine fallback, and separate infrastructure failures.
  • Human-only: production deployment, flags/env changes, rebuilds/backfills, signed/published packs and large paid runs.

Status: planned; completion requires the acceptance evidence above.

Pull request stack

The implementation hub records merge order, current verification and remaining release gates. PR publication does not mark the deployed or packaged acceptance matrix complete.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions