Skip to content

Add provider-local environment overlays - #1719

Merged
christso merged 2 commits into
mainfrom
av-f7w4-provider-env-overlays
Jul 7, 2026
Merged

Add provider-local environment overlays#1719
christso merged 2 commits into
mainfrom
av-f7w4-provider-env-overlays

Conversation

@christso

@christso christso commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add providers[].environment support for inline recipes and file:// refs in eval/provider catalog/config provider declarations.
  • Compose candidate provider overlays with suite/test/case environments, preserve layered provenance, and emit explicit conflicts for unsafe setup/cwd/Docker overlaps.
  • Prepare grader provider-local environments separately for grader invocation, without applying them to candidate execution.
  • Include provider-local environment refs in task/eval bundle serialization and docs for the next contract.

Validation

  • bun test packages/core/test/evaluation/environment-host-runtime.test.ts packages/core/test/evaluation/loaders/environment-recipe.test.ts packages/core/test/evaluation/loaders/config-loader.test.ts
  • bun test apps/cli/test/commands/eval/task-bundle.test.ts
  • bun run lint
  • bun run typecheck
  • bun run build
  • Live dogfood via local OpenAI-compatible proxy at http://127.0.0.1:10531/v1:
    • candidate provider: CLI wrapper that calls the proxy and reads a provider-local environment marker from cwd
    • grader provider: live OpenAI-compatible provider selected with --grader-provider live-grader
    • result: 1/1 passed at 100%

Evidence

  • Private evidence branch: agentv-private:evidence/av-f7w4-provider-env-overlays
  • Evidence commit: b838ec7
  • Evidence includes the source eval/providers/environment recipes, run bundle, artifact tree, and provenance sidecar showing distinct base/provider composition layers.

Bead: av-f7w4

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 7, 2026

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: f043763
Status: ✅  Deploy successful!
Preview URL: https://ea513bd6.agentv.pages.dev
Branch Preview URL: https://av-f7w4-provider-env-overlay.agentv.pages.dev

View logs

@christso
christso force-pushed the av-f7w4-provider-env-overlays branch from 0866c4f to f043763 Compare July 7, 2026 08:12
@christso
christso merged commit a48e426 into main Jul 7, 2026
9 checks passed
@christso
christso deleted the av-f7w4-provider-env-overlays branch July 7, 2026 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant