Repository navigation
feat(review): route public reviews through LiteLLM auto #2621
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
c661581
5ff59e4
7d293e7
7f5ba4a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2656,6 +2656,7 @@ jobs: | |
| update-environment: false | ||
|
|
||
| - name: Provision contextual-orchestrator review sidecar | ||
| if: needs.validate-pr-metadata.outputs.is_private == 'true' | ||
| env: | ||
| SIDECAR_PYTHON: ${{ steps.sidecar_python.outputs.python-path }} | ||
| BYTEZ_API_KEY: ${{ secrets.BYTEZ_API_KEY }} | ||
|
|
@@ -3757,6 +3758,9 @@ jobs: | |
| OPENCODE_FAILED_CHECK_EVIDENCE_FILE: ${{ runner.temp }}/opencode-failed-check-evidence.md | ||
| OPENCODE_CHANGED_FILES_FILE: ${{ runner.temp }}/opencode-changed-files.txt | ||
| OPENCODE_SOURCE_WORKDIR: ${{ runner.temp }}/opencode-pr-head | ||
| EXPECTED_IS_PRIVATE: ${{ needs.validate-pr-metadata.outputs.is_private }} | ||
| LLM_GATEWAY_API_KEY: ${{ needs.validate-pr-metadata.outputs.is_private == 'false' && secrets.LLM_GATEWAY_API_KEY || '' }} | ||
| LLM_GATEWAY_MODEL: ${{ vars.LLM_GATEWAY_MODEL || '' }} | ||
| run: | | ||
| set -euo pipefail | ||
| mkdir -p "$OPENCODE_REVIEW_WORKDIR" | ||
|
|
@@ -4099,11 +4103,44 @@ jobs: | |
| cp "$GITHUB_WORKSPACE/ci-review-prompt.md" "${OPENCODE_REVIEW_WORKDIR}/ci-review-prompt.md" | ||
| cp "$GITHUB_WORKSPACE/code-reviewer-prompt.md" "${OPENCODE_REVIEW_WORKDIR}/code-reviewer-prompt.md" | ||
|
|
||
| jq -n '{ | ||
| if [ "$EXPECTED_IS_PRIVATE" = true ]; then | ||
| review_model="contextual-orchestrator/orchestrator/free" | ||
| model_provider="contextual-orchestrator" | ||
| model_id="orchestrator/free" | ||
| provider_name="Contextual Orchestrator" | ||
| base_url="{env:CONTEXTUAL_ORCHESTRATOR_BASE_URL}/v1" | ||
| api_key="{env:CONTEXTUAL_ORCHESTRATOR_TOKEN}" | ||
| model_name="Orchestrator Free (ZDR-first zero-cost pool)" | ||
| elif [ "$EXPECTED_IS_PRIVATE" = false ]; then | ||
| if [ "$LLM_GATEWAY_MODEL" != auto ] || [ -z "$LLM_GATEWAY_API_KEY" ]; then | ||
| echo '::error::Public OpenCode review requires LLM_GATEWAY_MODEL=auto and LLM_GATEWAY_API_KEY.' | ||
| exit 1 | ||
| fi | ||
| printf '::add-mask::%s\n' "$LLM_GATEWAY_API_KEY" | ||
| review_model="litellm-poinnetworks/${LLM_GATEWAY_MODEL}" | ||
| model_provider="litellm-poinnetworks" | ||
| model_id="$LLM_GATEWAY_MODEL" | ||
| provider_name="PoinNetworks LiteLLM" | ||
| base_url="https://litellm.poinnetworks.net/v1" | ||
| api_key='{env:LLM_GATEWAY_API_KEY}' | ||
| model_name="Auto" | ||
| else | ||
| echo '::error::OpenCode review refuses unknown target visibility.' | ||
| exit 1 | ||
| fi | ||
|
|
||
| jq -n \ | ||
| --arg review_model "$review_model" \ | ||
| --arg model_provider "$model_provider" \ | ||
| --arg model_id "$model_id" \ | ||
| --arg provider_name "$provider_name" \ | ||
| --arg base_url "$base_url" \ | ||
| --arg api_key "$api_key" \ | ||
| --arg model_name "$model_name" '{ | ||
| "$schema": "https://opencode.ai/config.json", | ||
| "model": "contextual-orchestrator/orchestrator/free", | ||
| "small_model": "contextual-orchestrator/orchestrator/free", | ||
| "enabled_providers": ["contextual-orchestrator"], | ||
| "model": $review_model, | ||
| "small_model": $review_model, | ||
| "enabled_providers": [$model_provider], | ||
| "lsp": false, | ||
| "mcp": {}, | ||
| "permission": { | ||
|
Comment on lines
4103
to
4146
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift 🔎 Supported by static analysis🏁 Script executed: sed -n '4095,4150p' .github/workflows/opencode-review-dispatch.yml
sed -n '4250,4308p' .github/workflows/opencode-review-dispatch.yml
sed -n '6230,6257p' .github/workflows/opencode-review-dispatch.ymlRepository: ContextualWisdomLab/.github Length of output: 7813 🏁 Script executed: set -e
printf '%s\n' '--- current ADR ---'
nl -ba docs/adr/0003-contextual-orchestrator-vendored-free-zdr.md | sed -n '1,95p;145,170p;295,320p'
printf '%s\n' '--- OpenCode route and callers ---'
nl -ba .github/workflows/opencode-review-dispatch.yml | sed -n '4098,4150p;4248,4298p;6228,6260p'
printf '%s\n' '--- model-pool and diagnosis script bindings ---'
rg -n -F --glob 'scripts/ci/*' -- 'OPENCODE_MODEL_CANDIDATES' scripts/ci || test "$?" -eq 1
rg -n -F --glob 'scripts/ci/*' -- 'MODEL' scripts/ci/run_opencode_review_model_pool.sh scripts/ci/assert_opencode_reasoning_effort.py || true
nl -ba scripts/ci/run_opencode_review_model_pool.sh | sed -n '1,240p'Repository: ContextualWisdomLab/.github Length of output: 33938 🏁 Script executed: set -e
printf '%s\n' '--- ADR amendments after line 320 ---'
nl -ba docs/adr/0003-contextual-orchestrator-vendored-free-zdr.md | sed -n '320,430p'
printf '%s\n' '--- public gateway and cost-policy references ---'
rg -n -i --glob '!vendor/**' --glob '!node_modules/**' \
-e 'public.*(cost|price|spend|budget|zero[- ]cost|free)' \
-e '(cost|price|spend|budget|zero[- ]cost|free).*(public|LiteLLM|LLM_GATEWAY)' \
-e 'LLM_GATEWAY_MODEL|LLM_GATEWAY_API_KEY|litellm-poinnetworks' \
docs .github scripts || test "$?" -eq 1Repository: ContextualWisdomLab/.github Length of output: 20579 공개 OpenCode 경로에 zero-cost admission을 적용하십시오.
같은 검사를 generated configuration, model pool, failed-check diagnosis에 각각 적용해야 합니다. 4115행의 검사만 수정하면 나머지 두 요청 경로는 보호되지 않습니다. 공통 admission gate를 추가하고 세 경로가 실제 모델 요청 전에 그 gate를 통과하도록 수정하십시오. 실제 청구 발생을 전제로 하지 않아도 현재 admission 계약이 보장되지 않는 것이 문제입니다. 🤖 Prompt for AI Agents |
||
|
|
@@ -4179,49 +4216,29 @@ jobs: | |
| } | ||
| } | ||
| }, | ||
| "provider": {} | ||
| }' >"${OPENCODE_REVIEW_WORKDIR}/opencode.jsonc" | ||
|
|
||
| gateway_config="${OPENCODE_REVIEW_WORKDIR}/opencode.gateway.jsonc" | ||
| jq ' | ||
| .model = "contextual-orchestrator/orchestrator/free" | | ||
| .small_model = "contextual-orchestrator/orchestrator/free" | | ||
| .enabled_providers = ["contextual-orchestrator"] | | ||
| .provider = { | ||
| "contextual-orchestrator": { | ||
| "provider": { | ||
| ($model_provider): { | ||
| "npm": "@ai-sdk/openai-compatible", | ||
| "name": "Contextual Orchestrator", | ||
| "options": { | ||
| "baseURL": "{env:CONTEXTUAL_ORCHESTRATOR_BASE_URL}/v1", | ||
| "apiKey": "{env:CONTEXTUAL_ORCHESTRATOR_TOKEN}" | ||
| }, | ||
| "name": $provider_name, | ||
| "options": {"baseURL": $base_url, "apiKey": $api_key}, | ||
| "models": { | ||
| "orchestrator/free": { | ||
| "name": "Orchestrator Free (ZDR-first zero-cost pool)", | ||
| "tool_call": true, | ||
| "reasoning": true, | ||
| "options": { | ||
| "reasoningEffort": "high" | ||
| }, | ||
| "variants": { | ||
| "high": { | ||
| "reasoningEffort": "high" | ||
| } | ||
| }, | ||
| "limit": { | ||
| "context": 200000, | ||
| "output": 32768 | ||
| } | ||
| } | ||
| ($model_id): ( | ||
| {"name": $model_name, "tool_call": true} + | ||
| (if $model_provider == "contextual-orchestrator" then { | ||
| "reasoning": true, | ||
| "options": {"reasoningEffort": "high"}, | ||
| "variants": {"high": {"reasoningEffort": "high"}}, | ||
| "limit": {"context": 200000, "output": 32768} | ||
| } else {} end) | ||
| ) | ||
| } | ||
| } | ||
| } | ||
| ' "${OPENCODE_REVIEW_WORKDIR}/opencode.jsonc" >"$gateway_config" | ||
| mv -- "$gateway_config" "${OPENCODE_REVIEW_WORKDIR}/opencode.jsonc" | ||
| }' >"${OPENCODE_REVIEW_WORKDIR}/opencode.jsonc" | ||
|
|
||
| if ! grep -Fq 'contextual-orchestrator/orchestrator/free' "${OPENCODE_REVIEW_WORKDIR}/opencode.jsonc" \ | ||
| || ! grep -Fq 'CONTEXTUAL_ORCHESTRATOR_BASE_URL' "${OPENCODE_REVIEW_WORKDIR}/opencode.jsonc"; then | ||
| echo '::error::Generated isolated opencode.jsonc is missing the contextual-orchestrator gateway route.' | ||
| if ! grep -Fq "\"$review_model\"" "${OPENCODE_REVIEW_WORKDIR}/opencode.jsonc" \ | ||
| || ! grep -Fq "\"$model_provider\"" "${OPENCODE_REVIEW_WORKDIR}/opencode.jsonc"; then | ||
| echo '::error::Generated isolated opencode.jsonc is missing its visibility-scoped model route.' | ||
| exit 1 | ||
| fi | ||
| printf 'Prepared isolated OpenCode review workspace: %s\n' "$OPENCODE_REVIEW_WORKDIR" | ||
|
|
@@ -4234,7 +4251,8 @@ jobs: | |
| SHARE: "false" | ||
| NPM_CONFIG_IGNORE_SCRIPTS: "true" | ||
| NO_COLOR: "1" | ||
| OPENCODE_MODEL_CANDIDATES: "contextual-orchestrator/orchestrator/free" | ||
| OPENCODE_MODEL_CANDIDATES: ${{ needs.validate-pr-metadata.outputs.is_private == 'true' && 'contextual-orchestrator/orchestrator/free' || format('litellm-poinnetworks/{0}', vars.LLM_GATEWAY_MODEL) }} | ||
| LLM_GATEWAY_API_KEY: ${{ needs.validate-pr-metadata.outputs.is_private == 'false' && secrets.LLM_GATEWAY_API_KEY || '' }} | ||
| # One attempt per model, then fall through to the next model. Retrying | ||
| # the SAME model 5x let a rate-limited/hung leader consume the whole | ||
| # step, so the pool never reached a healthy fallback model. | ||
|
|
@@ -4272,7 +4290,14 @@ jobs: | |
| RUN_ATTEMPT: ${{ github.run_attempt }} | ||
| run: | | ||
| set -euo pipefail | ||
| source "$GITHUB_WORKSPACE/scripts/ci/load_contextual_orchestrator_token.sh" | ||
| if [ "$OPENCODE_MODEL_CANDIDATES" = contextual-orchestrator/orchestrator/free ]; then | ||
| source "$GITHUB_WORKSPACE/scripts/ci/load_contextual_orchestrator_token.sh" | ||
| elif [ "$OPENCODE_MODEL_CANDIDATES" = litellm-poinnetworks/auto ] && [ -n "$LLM_GATEWAY_API_KEY" ]; then | ||
| printf '::add-mask::%s\n' "$LLM_GATEWAY_API_KEY" | ||
| else | ||
| echo '::error::OpenCode model route or credential is unavailable.' | ||
| exit 1 | ||
| fi | ||
| set +e | ||
| bash "$GITHUB_WORKSPACE/scripts/ci/run_opencode_review_model_pool.sh" | ||
| pool_status=$? | ||
|
|
@@ -4852,7 +4877,8 @@ jobs: | |
| COVERAGE_EVIDENCE_SUMMARY: ${{ needs.coverage-evidence.outputs.coverage_summary || 'Coverage evidence job did not run or did not publish coverage evidence.' }} | ||
| OPENCODE_REVIEW_WORKDIR: ${{ runner.temp }}/opencode-review-project | ||
| OPENCODE_SOURCE_WORKDIR: ${{ runner.temp }}/opencode-pr-head | ||
| MODEL: contextual-orchestrator/orchestrator/free | ||
| MODEL: ${{ needs.validate-pr-metadata.outputs.is_private == 'true' && 'contextual-orchestrator/orchestrator/free' || format('litellm-poinnetworks/{0}', vars.LLM_GATEWAY_MODEL) }} | ||
| LLM_GATEWAY_API_KEY: ${{ needs.validate-pr-metadata.outputs.is_private == 'false' && secrets.LLM_GATEWAY_API_KEY || '' }} | ||
| USE_GITHUB_TOKEN: "true" | ||
| NPM_CONFIG_IGNORE_SCRIPTS: "true" | ||
| NO_COLOR: "1" | ||
|
|
@@ -6214,8 +6240,11 @@ jobs: | |
| printf 'Skipping publish-step failed-check OpenCode diagnosis for central review-process self-repair; using collected current-head failed-check logs/SARIF fallback so the publish step stays bounded.\n' >&2 | ||
| return 1 | ||
| fi | ||
| source "$GITHUB_WORKSPACE/scripts/ci/load_contextual_orchestrator_token.sh" | ||
| if [ -z "${CONTEXTUAL_ORCHESTRATOR_TOKEN:-}" ]; then | ||
| if [ "$MODEL" = contextual-orchestrator/orchestrator/free ]; then | ||
| source "$GITHUB_WORKSPACE/scripts/ci/load_contextual_orchestrator_token.sh" | ||
| elif [ "$MODEL" = litellm-poinnetworks/auto ] && [ -n "$LLM_GATEWAY_API_KEY" ]; then | ||
| printf '::add-mask::%s\n' "$LLM_GATEWAY_API_KEY" | ||
| else | ||
| return 1 | ||
| fi | ||
| if ! python3 "$GITHUB_WORKSPACE/scripts/ci/assert_opencode_reasoning_effort.py" \ | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift
🔎 Supported by static analysis
🏁 Script executed:
Repository: ContextualWisdomLab/.github
Length of output: 6083
🏁 Script executed:
Repository: ContextualWisdomLab/.github
Length of output: 9699
🏁 Script executed:
Repository: ContextualWisdomLab/.github
Length of output: 12596
공개 Noema 요청에 zero-priced admission을 추가하세요.
REQUIRE_ZDR=false분기는LLM_GATEWAY_MODEL=auto와 비어 있지 않은 키만 확인한 뒤 외부 LiteLLM URL을 사용합니다.call_llm은 이 값을model=auto로 전송합니다. ADR은 Noema가 zero-priced route만 허용한다고 유지합니다. 현재 경로에는 zero-price 또는 spend admission이 없습니다.auto와 키를 admission 증거로 사용하지 마세요. 공개 분기에서 zero-priced admission 결과를 검증하고, 증거가 없으면 모델 요청 전에 실패하도록 수정하세요. 이 수정은 required public review가 승인된 무료 supplier admission 없이 모델 요청을 수행하는 것을 방지합니다. 실제 billing 발생이나 public ZDR 위반은 이 코드만으로 입증되지 않습니다.🤖 Prompt for AI Agents