Skip to content

Run OpenCode one-shots through a session so free-tier models work - #703

Open
cpa01cmz-beep wants to merge 1 commit into
alphaXiv:mainfrom
cpa01cmz-beep:cpa01cmz-beep/opencode-session-generate
Open

cpa01cmz-beep wants to merge 1 commit into
alphaXiv:mainfrom
cpa01cmz-beep:cpa01cmz-beep/opencode-session-generate

Conversation

@cpa01cmz-beep

Copy link
Copy Markdown

Closes #702

Problem

With a free-tier OpenCode model selected, orx's internal one-shots (chat titles, starter prompts, skill-import screening, compaction summaries) fail with the gateway's 503 OpenCode's free tier can only be used from within OpenCode, while chat on the same model works.

The sessionless route orx posted to (/api/experimental/generate) is refused per endpoint by the provider gateway. Verified on OpenCode v2.0.32 against the same local opencode serve:

route result
POST /api/session/{id}/prompt 200, reply OK
POST /api/experimental/generate 503 free-tier rejection
POST /api/session/{id}/generate 200 {"data":{"text":"OK"}}

It is not about the database (isolated and real stores both fail), the model's credentials, or client identity (it persists right after a successful prompt on the same server, and with no model in the body).

Change

src/local/harness/opencode_v2.rs only:

  • generate_text() now creates a throwaway session (POST /api/session), generates on it (POST /api/session/{id}/generate — same /data/text shape), and deletes it (DELETE /api/session/{id}, best-effort).
  • Falls back to the existing sessionless route (v2_generate_path()) when the server predates /api/session/{id}/generate.
  • session_generate() / decode_generate() split out so the flow is testable without the OpenCode binary.

The session is never resumed and carries no work; /generate is plain text with no tool calls, so untrusted input (skill metadata) still cannot act.

Tests

Three new mock-server tests in local::harness::opencode::v2:

  • one_shot_generation_goes_through_a_session — session route used, sessionless route untouched, session deleted
  • one_shot_generation_falls_back_without_a_session_route — 404 on /api/session falls back
  • one_shot_generation_fallback_uses_the_legacy_path — pre-2.0.4 servers use /api/generate

Verification

cargo fmt --all --check                                # clean
cargo test --bin orx local::harness::opencode::v2      # 12 passed
cargo clippy --all-targets -- -D warnings              # clean

Also exercised live against OpenCode v2.0.32 with opencode/nemotron-3.5-super-vl-free: session create 200, session generate 200 {"data":{"text":"OK"}}, delete 204.

OpenCode's free tier refuses the sessionless generate route
(`/api/experimental/generate`) with "free tier can only be used from within
OpenCode" — a policy on that endpoint, enforced by the provider gateway, not by
the model, the database, or the client. The session-scoped route
(`/api/session/{id}/generate`) returns the same `/data/text` shape and is
allowed, so one-shots now create a throwaway session, generate on it, and
delete it. Servers that predate the session route keep using the sessionless
one.

This fixes titles, starter prompts, skill-import screening, and compaction
summaries for anyone whose chat model is a free-tier OpenCode model.
@greptile-apps

greptile-apps Bot commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] Adds a fallback code generation path for free-tier models.

The PR appears safe to merge, with non-blocking cleanup and test improvements.

Findings

  1. P2 Timeouts skip session cleanup ▶
  2. P2 Fallback branch lacks test coverage ▶
Fix with agent prompt
### Issue 1
src/local/harness/opencode_v2.rs:914-918
`session_generate` deletes the session inside the generation timeout. If generation times out after creating a session, deletion never runs. `NativeStore::Isolated` uses a shared, persistent database, so killing the child leaves that session behind. A stalled deletion can also turn a successful reply into a timeout.

Keep the session ID outside the timed operation and give cleanup its own short deadline before killing the child.

### Issue 2
src/local/harness/opencode_v2.rs:1095-1099
Both fallback tests return 404 from session creation. Neither covers the compatibility case this change promises: session creation succeeds, but `/api/session/{id}/generate` is missing.

Add a test that creates a session, returns 404 from its generate route, and checks that the session is deleted and the sessionless route supplies the reply. Otherwise this separate fallback branch can break while both tests still pass.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Routes OpenCode one-shot requests through a throwaway session so free-tier models can work.

  • OpenCode one-shot prompts now run through a temporary session.

Diagram

sequenceDiagram
    participant O as OpenResearch
    participant S as OpenCode server
    O->>S: Create throwaway session
    alt Session created
        O->>S: Generate text on session
        S-->>O: Text, error, or missing route
        O->>S: Delete session
        opt Session-generate route missing
            O->>S: Generate through sessionless route
        end
    else Session creation route missing
        O->>S: Generate through sessionless route
    end
Loading

Reviews (1) · Last reviewed commit: "Run OpenCode one-shots through a session..." · Reviewed by Greptile

Comment on lines +914 to +918
let _ = endpoint
.client
.delete(format!("{}/api/session/{session}", endpoint.base_url))
.send()
.await;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Timeouts skip session cleanup

session_generate deletes the session inside the generation timeout. If generation times out after creating a session, deletion never runs. NativeStore::Isolated uses a shared, persistent database, so killing the child leaves that session behind. A stalled deletion can also turn a successful reply into a timeout.

Keep the session ID outside the timed operation and give cleanup its own short deadline before killing the child.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/local/harness/opencode_v2.rs
Line: 914-918

Comment:
**Timeouts skip session cleanup**

`session_generate` deletes the session inside the generation timeout. If generation times out after creating a session, deletion never runs. `NativeStore::Isolated` uses a shared, persistent database, so killing the child leaves that session behind. A stalled deletion can also turn a successful reply into a timeout.

Keep the session ID outside the timed operation and give cleanup its own short deadline before killing the child.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment on lines +1095 to +1099
let app = Router::new()
.route(
"/api/session",
post(|| async { axum::http::StatusCode::NOT_FOUND }),
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Fallback branch lacks test coverage

Both fallback tests return 404 from session creation. Neither covers the compatibility case this change promises: session creation succeeds, but /api/session/{id}/generate is missing.

Add a test that creates a session, returns 404 from its generate route, and checks that the session is deleted and the sessionless route supplies the reply. Otherwise this separate fallback branch can break while both tests still pass.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/local/harness/opencode_v2.rs
Line: 1095-1099

Comment:
**Fallback branch lacks test coverage**

Both fallback tests return 404 from session creation. Neither covers the compatibility case this change promises: session creation succeeds, but `/api/session/{id}/generate` is missing.

Add a test that creates a session, returns 404 from its generate route, and checks that the session is deleted and the sessionless route supplies the reply. Otherwise this separate fallback branch can break while both tests still pass.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

OpenCode free-tier models are rejected for orx one-shot tasks

1 participant