Repository navigation
Run OpenCode one-shots through a session so free-tier models work - #703
cpa01cmz-beep wants to merge 1 commit into
Conversation
OpenCode's free tier refuses the sessionless generate route
(`/api/experimental/generate`) with "free tier can only be used from within
OpenCode" — a policy on that endpoint, enforced by the provider gateway, not by
the model, the database, or the client. The session-scoped route
(`/api/session/{id}/generate`) returns the same `/data/text` shape and is
allowed, so one-shots now create a throwaway session, generate on it, and
delete it. Servers that predate the session route keep using the sessionless
one.
This fixes titles, starter prompts, skill-import screening, and compaction
summaries for anyone whose chat model is a free-tier OpenCode model.
[Medium impact] Adds a fallback code generation path for free-tier models. The PR appears safe to merge, with non-blocking cleanup and test improvements. FindingsFix with agent prompt### Issue 1
src/local/harness/opencode_v2.rs:914-918
`session_generate` deletes the session inside the generation timeout. If generation times out after creating a session, deletion never runs. `NativeStore::Isolated` uses a shared, persistent database, so killing the child leaves that session behind. A stalled deletion can also turn a successful reply into a timeout.
Keep the session ID outside the timed operation and give cleanup its own short deadline before killing the child.
### Issue 2
src/local/harness/opencode_v2.rs:1095-1099
Both fallback tests return 404 from session creation. Neither covers the compatibility case this change promises: session creation succeeds, but `/api/session/{id}/generate` is missing.
Add a test that creates a session, returns 404 from its generate route, and checks that the session is deleted and the sessionless route supplies the reply. Otherwise this separate fallback branch can break while both tests still pass.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.SummaryRoutes OpenCode one-shot requests through a throwaway session so free-tier models can work.
|
| let _ = endpoint | ||
| .client | ||
| .delete(format!("{}/api/session/{session}", endpoint.base_url)) | ||
| .send() | ||
| .await; |
There was a problem hiding this comment.
session_generate deletes the session inside the generation timeout. If generation times out after creating a session, deletion never runs. NativeStore::Isolated uses a shared, persistent database, so killing the child leaves that session behind. A stalled deletion can also turn a successful reply into a timeout.
Keep the session ID outside the timed operation and give cleanup its own short deadline before killing the child.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/local/harness/opencode_v2.rs
Line: 914-918
Comment:
**Timeouts skip session cleanup**
`session_generate` deletes the session inside the generation timeout. If generation times out after creating a session, deletion never runs. `NativeStore::Isolated` uses a shared, persistent database, so killing the child leaves that session behind. A stalled deletion can also turn a successful reply into a timeout.
Keep the session ID outside the timed operation and give cleanup its own short deadline before killing the child.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| let app = Router::new() | ||
| .route( | ||
| "/api/session", | ||
| post(|| async { axum::http::StatusCode::NOT_FOUND }), | ||
| ) |
There was a problem hiding this comment.
Fallback branch lacks test coverage
Both fallback tests return 404 from session creation. Neither covers the compatibility case this change promises: session creation succeeds, but /api/session/{id}/generate is missing.
Add a test that creates a session, returns 404 from its generate route, and checks that the session is deleted and the sessionless route supplies the reply. Otherwise this separate fallback branch can break while both tests still pass.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/local/harness/opencode_v2.rs
Line: 1095-1099
Comment:
**Fallback branch lacks test coverage**
Both fallback tests return 404 from session creation. Neither covers the compatibility case this change promises: session creation succeeds, but `/api/session/{id}/generate` is missing.
Add a test that creates a session, returns 404 from its generate route, and checks that the session is deleted and the sessionless route supplies the reply. Otherwise this separate fallback branch can break while both tests still pass.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Closes #702
Problem
With a free-tier OpenCode model selected, orx's internal one-shots (chat titles, starter prompts, skill-import screening, compaction summaries) fail with the gateway's 503
OpenCode's free tier can only be used from within OpenCode, while chat on the same model works.The sessionless route orx posted to (
/api/experimental/generate) is refused per endpoint by the provider gateway. Verified on OpenCode v2.0.32 against the same localopencode serve:POST /api/session/{id}/promptOKPOST /api/experimental/generatePOST /api/session/{id}/generate{"data":{"text":"OK"}}It is not about the database (isolated and real stores both fail), the model's credentials, or client identity (it persists right after a successful prompt on the same server, and with no model in the body).
Change
src/local/harness/opencode_v2.rsonly:generate_text()now creates a throwaway session (POST /api/session), generates on it (POST /api/session/{id}/generate— same/data/textshape), and deletes it (DELETE /api/session/{id}, best-effort).v2_generate_path()) when the server predates/api/session/{id}/generate.session_generate()/decode_generate()split out so the flow is testable without the OpenCode binary.The session is never resumed and carries no work;
/generateis plain text with no tool calls, so untrusted input (skill metadata) still cannot act.Tests
Three new mock-server tests in
local::harness::opencode::v2:one_shot_generation_goes_through_a_session— session route used, sessionless route untouched, session deletedone_shot_generation_falls_back_without_a_session_route— 404 on/api/sessionfalls backone_shot_generation_fallback_uses_the_legacy_path— pre-2.0.4 servers use/api/generateVerification
Also exercised live against OpenCode v2.0.32 with
opencode/nemotron-3.5-super-vl-free: session create 200, session generate 200{"data":{"text":"OK"}}, delete 204.