Skip to content

strix openai-direct fallback 404s at runtime despite gpt-5.4 contract (blocks macos_utility_packs#3) #1327

Open

Description

@seonghobae

Evidence

Run 32815579699 attempt 2 (macos_utility_packs, head 8348e4f, trusted strix @8fd471a):

  1. Primary NIM nemotron-3-super-120b-a12b → provider failure-signal.
  2. Fallback openai-direct/gpt-5.4 → LLM CONNECTION FAILED · Error: 404 page not found after 4s.
  3. An inner pass had already reported Vulnerabilities 0, but no report artifact was produced → gate correctly fails closed.

Hypotheses (ordered)

  1. openai_direct/ is not a recognized strix provider slug; the runner maps openai-direct/gpt-5.4→openai_direct/gpt-5.4 (.github/workflows/strix.yml:784-788) and the unknown-provider default base URL 404s.
  2. Base URL for direct OpenAI missing /v1 segment when constructed from STRIX_OPENAI_API_BASE_FILE.
  3. gpt-5.4 not exposed on the direct OpenAI API for this key tier (model-id mismatch vs gpt-5.4-turbo style alias).

Ask

Fix provider slug/base-url routing so the contracted final fallback completes; then re-run Strix Security Scan on macos_utility_packs#3 head 8348e4f. Related: #1317, #1318.

Activity

  1. seonghobae commented on Aug 25, 2026

    @seonghobae
    ContributorAuthor

    Root cause confirmed and fixed in #1326: resolved_llm_api_base_for_model had no openai-direct branch, so nvidia_nim-primary runs passed LLM_API_BASE=https://integrate.api.nvidia.com/v1 to the openai-direct/gpt-5.4 fallback attempt. litellm posted openai/gpt-5.4 to NVIDIA's inference endpoint → Go-style '404 page not found' → STRIX_PROVIDER_UNAVAILABLE. The fix returns no custom base for explicit direct-OpenAI models so litellm routes to api.openai.com with the OpenAI fallback key (key selection already existed). Awaiting Checks on #1326 head; macos_utility_packs#3 strix should pass on the next scheduler rerun after merge.

  2. seonghobae commented on Aug 25, 2026

    @seonghobae
    ContributorAuthor

    Fresh cross-product reproduction on 2026-08-25 confirms this is still a central Strix provider/fallback boundary, not a leaf-code finding.

    ContextualWisdomLab/pg-erd-cloud#704 exact head 2c4f891bd71ed74c5a7cff8e511c3b2d746a667a has every other live required context terminal-success, but required Strix run 32830460915 / job 97747751275 failed on trusted Strix 8fd471a31399a914d9cb22a840f4a4c68e010ea6:

    • primary nvidia_nim/nvidia/nemotron-3-super-120b-a12b repeatedly hit HTTP 429;
    • the NVIDIA 49B fallback completed a zero-vulnerability penetration-test summary, but provider-failure evidence from the attempt kept the fail-closed gate non-passing;
    • final openai-direct/gpt-5.4 then failed with the same 404 page not found signature documented here.

    I re-ran only the failed Strix job against the unchanged leaf head; the replacement job is 97852761442 and is currently queued, so it is not passing evidence. No pg-erd-cloud source change was made for this control-plane/provider failure.

    This gives #1326 a second independent consumer canary for the resolved_llm_api_base_for_model repair. After #1326 integrates normally, re-run Strix on the then-unchanged leaf head and require terminal success before treating the leaf as merge-ready.

  3. seonghobae commented on Aug 25, 2026

    @seonghobae
    ContributorAuthor

    Fresh downstream canary from ContextualWisdomLab/OriginWeave#124 confirms the same central Strix fallback defect on an exact-current leaf head and adds one distinct transient/tool-boundary symptom that should be preserved during the owner fix.

    • OriginWeave PR docs: record aFIPC required workflow coverage gap #124 exact head: 6dd2023ca2f9fd5003564d1603565c05f4d9f111
    • protected live base: b05d5acca82b9d916ada2c8e82f59f92a89817e1
    • required Strix run/job: 32884789734 / 97922586136
    • Strix report artifact: 9577973496, digest sha256:2d0134a7ec1e8f474af480b98f6cac46921740f53850daa31528871a9ef389cf
    • native CI 32884793521, SAST 32884793442, and Security Scan 32884793441 all completed successfully on the same head.

    The Strix cascade was:

    1. primary nvidia_nim/nvidia/nemotron-3-super-120b-a12b repeatedly returned HTTP 429 and exhausted its bounded retry budget;
    2. the NVIDIA 49B fallback later hit loginAsGuest failed after 10 attempts / curl exit 7 against the local 127.0.0.1:48080 proxy boundary;
    3. final openai-direct/gpt-5.4 reproduced the same 404 page not found signature already traced here to resolved_llm_api_base_for_model routing.

    No OriginWeave source workaround is appropriate: this is the organization-required Strix provider/runtime boundary and the leaf gate correctly remains fail-closed. Please use this exact-head canary when validating #1326: preserve bounded 429 handling and fail-closed semantics, ensure the direct-OpenAI fallback no longer inherits the NVIDIA API base, and separately classify the local proxy-readiness failure rather than masking it as a clean scan. After the central fix integrates normally, rerun Strix against the then-unchanged OriginWeave #124 head; only terminal exact-head evidence should count.

  4. seonghobae commented on Aug 26, 2026

    @seonghobae
    ContributorAuthor

    Fresh ScopeWeave canary confirms the same central Strix provider/fallback defect on an exact PR head; no ScopeWeave source workaround is appropriate.

    • leaf PR: ContextualWisdomLab/scopeweave#602
    • exact leaf head: 5f9dc371fb3ba7150a8f1e3693bc64794e50ebee
    • live protected base observed: 2c328875e00e86537df3e965170be80532571cad
    • required Strix run/job: 32819824316 / 97715410727
    • trusted .github Strix source: 8fd471a31399a914d9cb22a840f4a4c68e010ea6
    • the trusted workflow correctly fetched and scanned PR head 5f9dc371fb3ba7150a8f1e3693bc64794e50ebee; this is not a synthetic-checkout failure.

    Observed cascade:

    1. primary nvidia_nim/nvidia/nemotron-3-super-120b-a12b exhausted three bounded attempts with HTTP 429 Too Many Requests;
    2. fallback nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 eventually completed a Vulnerabilities 0 run, but its degraded narrative incorrectly claimed server/server.mjs was absent and full execution was impossible, so the fail-closed gate correctly refused to promote that incomplete/provider-contaminated output;
    3. final openai-direct/gpt-5.4 reproduced LLM CONNECTION FAILED · Error: 404 page not found after ~5 seconds;
    4. no authoritative structured vulnerability report completed, so the required check correctly ended STRIX_PROVIDER_UNAVAILABLE/failure rather than false-green.

    Use this as another exact-head acceptance canary for the existing owner repair. RED should preserve the NIM-primary→direct-OpenAI fallback chain with a non-OpenAI primary base and assert that the direct-OpenAI attempt cannot inherit https://integrate.api.nvidia.com/v1. GREEN requires a valid direct-OpenAI route/key/base, a completed authoritative structured report on the unchanged leaf head, no provider-failure markers, and terminal required-check success without weakening fail-closed behavior. Also retain a fixture proving that a model-produced 0 vulnerabilities summary which simultaneously reports inability to execute the target is not merge evidence.

  5. seonghobae commented on Aug 26, 2026

    @seonghobae
    ContributorAuthor

    Fresh ScopeWeave consumer evidence adds a distinct direct-OpenAI fallback contract failure that is not covered by the existing NVIDIA-base 404 diagnosis, so the central owner path needs to preserve both regressions.

    Consumer tuple:

    • ContextualWisdomLab/scopeweave#610
    • exact head fb381e86cdb0c6bcc6397a278070cb26e964a4d6
    • protected live base 2c328875e00e86537df3e965170be80532571cad
    • required Strix run/job 32976934806 / 98203848191

    The trusted workflow validated the exact target/base/head and entered Strix normally. The primary nvidia_nim/nvidia/nemotron-3-super-120b-a12b attempt then hit HTTP 429. The final direct fallback was resolved as openai_direct/gpt-5.4, but failed before authoritative analysis with a LiteLLM/OpenAI BadRequestError: the configured reasoning_effort was rejected because the runtime classified the fallback as not a GPT-5-series model. No authoritative Strix report was produced, so the required gate correctly remained fail-closed. There is no ScopeWeave-local causal repair for this provider/runtime contract.

    Please extend the existing owner repair/validation rather than treating #1326's API-base fix alone as sufficient. A useful permanent RED contract is: force the primary NIM path to a bounded 429, exercise the same installed Strix/LiteLLM runtime and trusted fallback configuration, and prove openai_direct/gpt-5.4 reaches model execution without either (a) inheriting the NVIDIA API base or (b) failing provider/model-specific reasoning_effort validation. The smallest remedy should align provider slug/model normalization and reasoning-effort options with what the installed runtime actually supports; do not remove the fail-closed/no-artifact guard or downgrade provider failures.

    Required GREEN evidence: focused central fallback contracts, full exact-head central gates/reviews, then an unchanged-head rerun of ScopeWeave #610 proving the fallback can reach target analysis and emit authoritative exact-head Strix evidence. This new canary is complementary to the existing macos_utility_packs/pg-erd-cloud/OriginWeave 404 canaries.

  6. seonghobae commented on Aug 26, 2026

    @seonghobae
    ContributorAuthor

    Fresh convergence check on the three live repair vehicles: #1350 (045f1789...), #1352 (dbf8ae19...), and #1353 (6c26f754...) all target the direct-OpenAI Strix reasoning contract, but #1353 currently has the narrowest demonstrated provider boundary for the ScopeWeave canary. Its first revision was correctly challenged for suppressing configured effort on custom OpenAI-compatible endpoints; exact head 6c26f754... now limits none to explicit direct-OpenAI attempts using the native/default https://api.openai.com/v1 authority and adds regressions proving a custom endpoint preserves configured effort. The thread is resolved/outdated, Strix Changed Path Quality CI is terminal success, while CodeQL/Python Security are still queued and therefore non-authorizing.

    By contrast, #1350 keys the override specifically to the GPT-5.4 model spelling and #1352 also edits the workflow-level default plus gate propagation; they overlap the same causal path. Please converge on one owner branch rather than integrating multiple competing variants. Preserve #1353's native-vs-custom endpoint regression plus the existing NIM 429/410 → direct OpenAI fallback regression, and require a post-integration unchanged-head downstream canary on scopeweave#610@fb381e86cdb0c6bcc6397a278070cb26e964a4d6. No downstream success should be claimed until that rerun reaches authoritative Strix analysis/artifact generation.

  7. seonghobae commented on Aug 26, 2026

    @seonghobae
    ContributorAuthor

    Fresh downstream reproduction from BandScope shows the same canonical Strix fallback lane has moved past the earlier 404 but is still broken at the request-contract boundary.

    Exact evidence (2026-08-26 UTC):

    • consumer: ContextualWisdomLab/bandscope#1024
    • exact PR head: c03001e3af3452057d4ec21dec402ce39fc2fa04
    • independently resolved live base: develop@749511c3ad4000090048718f685c6bee6b3d2c25
    • trusted central source: ContextualWisdomLab/.github/main@e00bd7964f332b69cf7b430b0cb5ad486eef8258
    • Strix run: 32982165723
    • failing job: 98221210650

    Observed attempt sequence from the current job log:

    1. nvidia_nim/nvidia/nemotron-3-super-120b-a12b -> HTTP 429 Too Many Requests.
    2. fallback nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 -> HTTP 410 Gone: Model is no longer available as of Aug 26, 09Z. Please use an alternative model. This fallback is now permanently EOL, not a transient provider failure.
    3. final fallback openai_direct/gpt-5.4 reaches OpenAI, but fails HTTP 400: reasoning_effort is not supported with this model (param: reasoning_effort). Strix then exhausts the pool without an authoritative finding/no-finding report and correctly fails closed.

    First causal boundary is central provider/model routing and request-envelope capability handling; there is no correct BandScope-local source fix for this exact failure. Please keep the fail-closed gate and repair the central lane rather than asking the consumer to suppress/retry around it.

    Acceptance test for this reproduction:

    • remove/replace the EOL NIM fallback with a currently supported model under the existing bounded provider policy;
    • make request construction capability-aware so openai_direct/gpt-5.4 does not receive unsupported reasoning_effort (or route it through the officially supported endpoint/envelope for that capability);
    • add focused regressions covering 410-permanent-model-retirement and model-specific unsupported request parameters without converting real provider failures/findings to success;
    • rerun Strix against unchanged BandScope head c03001e3af3452057d4ec21dec402ce39fc2fa04 and require a terminal authoritative finding/no-finding report bound to that exact head before treating the gate as recovered.
  8. seonghobae commented on Aug 26, 2026

    @seonghobae
    ContributorAuthor

    Fresh ScopeWeave exact-head evidence after the protected-main GPT-5.4 reasoning repair exposes a remaining, separate Strix fallback lifecycle defect that still belongs to this central owner boundary.

    ScopeWeave PR #610 exact head fb381e86cdb0c6bcc6397a278070cb26e964a4d6 failed required Strix run 32976934806, job 98203848191, while the trusted workflow source was predecessor .github@e00bd7964f332b69cf7b430b0cb5ad486eef8258:

    1. primary nvidia_nim/nvidia/nemotron-3-super-120b-a12b exhausted bounded retries on HTTP 429;
    2. fallback nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 returned HTTP 410 and explicitly reported end-of-life at 2026-08-26T09:00:00Z;
    3. final openai-direct/gpt-5.4 then hit the function-tools/reasoning-effort HTTP 400 that has since been repaired on protected .github/main by merged fix(strix): honor GPT-5.4 tool reasoning contract #1350.

    Fresh protected .github/main is now f655a901f7ccdfef0d62694c818ad2896a2f5da1, so the GPT-5.4 reasoning failure is historical for the next rerun. However, that exact protected-main strix.yml still hard-codes the EOL nvidia/llama-3.3-nemotron-super-49b-v1.5 in the nvidia_nim fallback list. A new ScopeWeave run can therefore still waste a complete fallback slot on a deterministically unavailable provider model before reaching direct OpenAI.

    Please treat this as a RED acceptance case for the central Strix owner, not a ScopeWeave source defect. Smallest acceptable remedy: make the Strix NVIDIA fallback lifecycle-aware (or otherwise remove the retired pin through the existing trusted model-selection boundary) without broadening credentials, weakening fail-closed evidence, or accepting a provider error as a clean scan. Required GREEN evidence: central focused/full Strix contracts on the owner head plus a rerun of required Strix on the unchanged ScopeWeave #610 head under the then-current protected central workflow. The canary must show the EOL model is not attempted and the authoritative scan still fails closed on genuine provider exhaustion or missing vulnerability evidence.

  9. seonghobae commented on Aug 26, 2026

    @seonghobae
    ContributorAuthor

    Fresh OriginWeave canary confirms the remaining Strix failure is still central provider-lifecycle state, not an OriginWeave source defect.

    • consumer: ContextualWisdomLab/OriginWeave#37
    • exact consumer head: 34b825f6959b0ba08ab8f2785601a2e5033c1b3b
    • exact protected base: b05d5acca82b9d916ada2c8e82f59f92a89817e1
    • required Strix workflow/job: 32944287472 / 98281866567
    • trusted central workflow SHA from the job: 874f47b3856ca6bdb6bc71d48173b8f34ba7b9ca

    The exact attempt sequence is:

    1. primary nvidia_nim/nvidia/nemotron-3-super-120b-a12b exhausted bounded retries on HTTP 429;
    2. fallback nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 returned HTTP 410 Gone and states the model reached EOL at 2026-08-26T09:00:00Z;
    3. final openai-direct/gpt-5.4 reached the direct provider but exhausted on HTTP 429 insufficient_quota / credit_balance_exhausted.

    No authoritative vulnerability finding/no-finding report was produced, so the gate correctly remains fail-closed as STRIX_PROVIDER_UNAVAILABLE. OriginWeave native CI, SAST, and Security Scan are separate evidence and must not substitute for Strix.

    The permanently retired NVIDIA fallback is the actionable owner defect: remove/replace it through the reviewed central model-selection boundary while preserving bounded attempts, exact-head binding, credential separation, and fail-closed exhaustion. This canary should be rerun only after that central owner repair reaches protected .github/main; the unchanged consumer head must then produce terminal authoritative Strix evidence before promotion.

  10. seonghobae commented on Aug 26, 2026

    @seonghobae
    ContributorAuthor

    Fresh Orgmetra exact-head canary confirms the Strix failure is still owned by the central provider/fallback boundary, not by the leaf PR.

    Consumer tuple:

    • leaf PR: ContextualWisdomLab/Orgmetra#45
    • exact head: 90aa2284a43b5f2491ee3f20c6946c4a78520c36
    • live base observed: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • required Strix run/job: 33010253596 / 98314385714
    • Strix report artifact: 9622747073, digest sha256:45aeeae8700b0e31cd9d8adcb8b4aec451d2eed35f3882686fa92187d804ca78

    The exact job acquired a runner, materialized the target workspace, fetched the PR head, installed Strix, and reached Run Strix (quick). The fail-closed cascade was then entirely provider-side:

    1. primary nvidia_nim/nvidia/nemotron-3-super-120b-a12b repeatedly returned HTTP 429 Too Many Requests and exhausted bounded retries;
    2. fallback openrouter/free failed with OpenRouter API 502 Invalid URL (provider metadata: Stealth);
    3. final openai-direct/gpt-5.4 now reaches the direct OpenAI service rather than the old NVIDIA-base 404 path, but fails with HTTP 429 insufficient_quota / credit_balance_exhausted (You have no credits remaining).

    No authoritative vulnerability report was produced. One attempt displayed Vulnerabilities 0, but the gate correctly refused to promote it because provider-failure markers remained and the structured report was incomplete. That fail-closed behavior must be preserved.

    This is useful evidence that the API-base routing repair alone is insufficient for reliable required-check completion: the owner boundary also needs a healthy provider/key/credit fallback chain (and the OpenRouter invalid-URL route should be classified separately). No Orgmetra source workaround exists without weakening the required security gate.

    Acceptance canary after owner repair: rerun Strix on unchanged Orgmetra #45 head 90aa2284...; require a completed authoritative structured report with no provider-failure markers and terminal required-check success. Do not treat log-only 0 vulnerabilities output as passing evidence.

  11. seonghobae commented on Aug 26, 2026

    @seonghobae
    ContributorAuthor

    Fresh OriginWeave consumer evidence after central Strix 548a97560070d31b03d14bee0ac98a990bd88482 (post-#1297) shows the fallback contract is still not robust enough; no OriginWeave source workaround is appropriate.

    Exact consumer:

    • ContextualWisdomLab/OriginWeave#229
    • exact head 585a7d5545b13f18d76f79100ff4d47ac423e861
    • exact live base b05d5acca82b9d916ada2c8e82f59f92a89817e1
    • Strix run 33000778334, attempt 2; job 98341504012
    • report artifact 9625898775, SHA-256 28b8a28f664dbc7e43ebc4ef6b7122b217d8aa8f013c1da698fb5ffb291ac7bd

    The bounded provider cascade failed closed without authoritative vulnerability evidence:

    1. primary nvidia_nim/nvidia/nemotron-3-super-120b-a12b repeatedly returned HTTP 429;
    2. the live-NVIDIA fallback resolver selected nvidia_nim/nvidia/llama-3.1-nemotron-ultra-253b-v1, but NVIDIA returned HTTP 404 / Function ... Not found for account ..., indicating that catalog selection did not prove account-invocable availability;
    3. openrouter/free repeatedly returned HTTP 502 Invalid URL from provider Stealth;
    4. openai-direct/gpt-5.4 reached the direct OpenAI route but repeatedly returned HTTP 429 credit_balance_exhausted / insufficient quota.

    This is materially different from the old inherited-NVIDIA-base 404 fixed by #1326: direct OpenAI routing is now correct enough to reach OpenAI, but all authority-equivalent fallbacks exhausted. The NVIDIA 404 is also stronger than ordinary transient rate limiting: a live resolver must not promote a model as usable merely because it appears in a catalog when the configured account cannot invoke it.

    Please keep Strix fail-closed, but add/extend test-first central owner coverage so model resolution validates account-invocable capability (or rejects a 404-selected candidate from the current run), records the finite provider-unavailable class, and advances only to reviewed authority-equivalent fallbacks under existing bounded attempt/deadline/resource rules. Repeated OpenRouter Stealth 502 should remain a typed provider-unavailable outcome rather than be retried indefinitely; OpenAI billing/quota remains an external fail-closed condition, not a success fallback.

    The same job also logged inability to compute the PR merge base and used the existing direct base/head diff fallback for changed-file enumeration. That did retain one scannable changed file, so I am not attributing the provider failure to OriginWeave diff selection; central owner should nevertheless preserve exact base/head identity when validating the eventual fix.

  12. seonghobae commented on Aug 27, 2026

    @seonghobae
    ContributorAuthor

    Fresh Orgmetra post-owner-repair acceptance canary has been started without changing leaf source.

    • consumer: ContextualWisdomLab/Orgmetra#45
    • unchanged exact head: 90aa2284a43b5f2491ee3f20c6946c4a78520c36
    • live base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • prior Strix run/job: 33010253596 / 98314385714 (provider/fallback failure already documented here)
    • safe re-run attempt 2 job: 98365671632
    • current state: queued, no runner assigned yet; therefore non-passing

    All Orgmetra-local exact-head product/Foundation/Recovery/SAST/Security gates remain terminal GREEN. This rerun is the required unchanged-head canary after the central Strix owner repairs; do not treat it as GREEN unless it produces a complete authoritative structured Strix report with no provider-failure markers and a terminal successful required check. No .github source/ref/workflow/PR state was modified from the Orgmetra loop.

  13. seonghobae commented on Aug 27, 2026

    @seonghobae
    ContributorAuthor

    Fresh downstream canary from ContextualWisdomLab/OriginWeave#43 confirms the required Strix failure is still entirely in the central provider/runtime boundary, not the OriginWeave MV3 product code.

    • OriginWeave PR Refresh governance audit after clearfolio rollout #43 exact head: 14728625b180c5ffe6ba0aa1b8d61a16f9347252
    • protected live base used by the run: b05d5acca82b9d916ada2c8e82f59f92a89817e1
    • required Strix run/job: 33025899963 / 98367053503
    • trusted central Strix source: .github@548a97560070d31b03d14bee0ac98a990bd88482
    • Strix report artifact: 9628800510, digest sha256:0e81d1aafd01af8cc9f0300d237d42f1ed79038cd62028dc0d46c2a2fde8e5de

    The exact-head cascade was:

    1. primary nvidia_nim/nvidia/nemotron-3-super-120b-a12b repeatedly returned HTTP 429;
    2. NVIDIA fallback nvidia_nim/nvidia/llama-3.1-nemotron-ultra-253b-v1 returned 404 Function ... Not found for account;
    3. openrouter/free repeatedly returned 502 Invalid URL from provider Stealth;
    4. final openai-direct/gpt-5.4 repeatedly returned 429 insufficient_quota / credit_balance_exhausted.

    One NVIDIA attempt printed Vulnerabilities 0, but it also emitted a provider-failure traceback and did not yield authoritative clean report evidence; the central gate correctly kept the run fail-closed as STRIX_PROVIDER_UNAVAILABLE. No OriginWeave source workaround is appropriate. Please retain this exact-head canary for the central provider/fallback repair and rerun against the unchanged leaf head after the owner fix; only a terminal authoritative exact-head Strix result should count.

  14. 64 remaining items

  15. seonghobae commented on Aug 30, 2026

    @seonghobae
    ContributorAuthor

    Post-#1434 unchanged-head Orgmetra canary now reaches the actual Strix runtime and exposes a later, distinct sandbox/bootstrap boundary. Existing owner path only; no Orgmetra workaround is appropriate.

    • consumer: ContextualWisdomLab/Orgmetra#40
    • exact target head: 6917e41f9053fab6f7e99f8185f2137e8fc5fca5
    • exact live base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • Required Strix run: 33294991609, attempt 5
    • replacement job: 99247471558
    • trusted central workflow SHA: e36a1f716d2ba0334c0e40597d7f9a90eee9cb14
    • pinned contextual-orchestrator: 30c6d71680e659f25a0a433d4726ad0d437f9757
    • diagnostic artifact: 9731299631, 17,897 bytes, SHA-256 99d91e741cfc66787d057f54f1fcd6863dce2103b4cc421ba7c6110cdbcf6eb7

    Progress proved GREEN before the failure: exact target/base and PR-scoped workspace materialize; 4 changed scannable files are selected; 5/5 provider secrets pass; contextual-orchestrator reaches /healthz + provider-route after 43s; gateway chat/completions preflight succeeds; the free pool is resolved; hash-pinned strix-agent==1.5.3 installs and its executable SHA-256 is d2dd9753453674e0081508a08d869e7b629c15f11b70294b980033272734f073; Run Strix (quick) actually starts.

    The first scan-stage causal boundary is then local Strix/Caido sandbox bootstrap, before meaningful LLM vulnerability analysis: loginAsGuest failed after 10 attempts: curl exit 7: Failed to connect to 127.0.0.1 port 48080. The Strix invocation retries Caido bootstrap three times and every attempt gets the same connection refusal. The outer trusted gate classifies this as STRIX_PROVIDER_UNAVAILABLE, waits and retries the entire gate once; the second Strix invocation again performs three Caido attempts and fails identically. The bounded retry/time policy then fails closed. No authoritative vulnerability finding/no-finding was produced; any initial UI display such as Vulnerabilities 0 and the uploaded artifact are diagnostic-only, not a clean scan.

    This exact canary is useful because the provider/gateway setup now succeeds far enough to show that retrying model/provider discovery cannot repair the first failure. The local Caido runtime/proxy on 127.0.0.1:48080 never becomes reachable. The current provider-unavailable classification also conflates a scanner sandbox readiness defect with an LLM provider failure, causing a futile full-provider retry.

    Owner-side acceptance:

    1. RED: reproduce the unchanged ⚡ Bolt: JSON 파싱 성능 최적화 #40 head reaching Run Strix (quick) and failing Caido guest bootstrap at 127.0.0.1:48080 before any meaningful scanner/model work.
    2. Smallest remedy should be at the trusted Strix/Strix-agent sandbox integration boundary: make Caido runtime startup/readiness/port binding and guest-login lifecycle reliable and classify local sandbox bootstrap separately from provider exhaustion. Preserve fail-closed semantics and the existing bounded retry/time budget.
    3. GREEN: on the unchanged/fresh exact consumer head, Caido local API becomes reachable, loginAsGuest succeeds, Strix proceeds through actual LLM vulnerability analysis, and the required workflow emits a complete structured terminal finding/no-finding report. A pre-bootstrap Vulnerabilities 0, diagnostic artifact, skipped scanner, or provider-status surrogate must not satisfy the required gate.

    No Orgmetra source/ref/workflow/settings state and no foreign writer-owned source/ref/workflow state was mutated for this evidence.

  16. seonghobae commented on Aug 30, 2026

    @seonghobae
    ContributorAuthor

    Fresh unchanged-head Orgmetra dependency-root Strix canary against the current trusted central source now reaches the actual Strix model/tool call and exposes a later API-compatibility boundary. Existing owner path only; no Orgmetra workaround is appropriate.

    • consumer: ContextualWisdomLab/Orgmetra#40
    • exact target head: 6917e41f9053fab6f7e99f8185f2137e8fc5fca5
    • exact live base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • Required Strix run: 33294991609, attempt 6
    • replacement job: 99251706718
    • trusted central workflow SHA: 8b3235d22129035b49ac481a40a341002540e2af
    • pinned contextual-orchestrator: 30c6d71680e659f25a0a433d4726ad0d437f9757
    • diagnostic-only artifact: 9731760870, SHA-256 ba71a0681837e18dbeb903bb76c81145fa9a2e40850d3110772675abd13545cc

    This attempt passes trusted-source resolution/checkout, exact target/base materialization, required-workflow smoke, 5/5 provider-secret gate, contextual-orchestrator installation, sidecar /healthz, provider-route preflight, gateway chat/completions preflight, hash-pinned strix-agent==1.5.3 installation, and PR-scoped input preparation. The sidecar pool is orchestrator/free; two NVIDIA NIM routes are runtime-ready. Strix then actually starts.

    The first causal boundary is now the first streaming tool-capable Strix request. On each of the three bounded attempts, before any model response, the OpenAI-compatible gateway returns HTTP 400 invalid_stream_options: stream_options.include_usage=true is not supported with tools or response_format. Strix is running with openai/orchestrator/free; the wrapper retries after 90s and 180s, then correctly fails closed as STRIX_PROVIDER_UNAVAILABLE. Vulnerabilities 0 / Tokens 0 are not clean evidence because the scan never receives its first model response and no authoritative structured finding/no-finding report completes.

    This is not repairable in Orgmetra source: the leaf cannot alter the centrally pinned Strix/OpenAI Agents request semantics or the shared contextual-orchestrator OpenAI-compatible contract. Material owner remedies are limited to the compatibility boundary: (a) support/normalize stream_options.include_usage for streamed tool/response-format chat requests when consistent with the advertised gateway contract, (b) have the trusted Strix adapter omit/translate that option for this tool/response-format path, or (c) pin a compatible Strix/OpenAI-Agents combination. Preserve streaming/tool execution and fail-closed semantics; do not solve this with a leaf shim or synthetic success.

    GREEN acceptance: rerun Required Strix without changing Orgmetra head 6917e41f...; require the first tool-capable streamed request to succeed through the trusted gateway, complete the bounded real scanner, publish an authoritative structured finding/no-finding artifact bound to this exact head/base, and finish terminal GREEN. invalid_stream_options, provider/runtime failure, diagnostic-only artifacts, status-only/model-only output, or predecessor-head evidence remain non-passing.

  17. seonghobae commented on Aug 30, 2026

    @seonghobae
    ContributorAuthor

    Independent second Orgmetra consumer reproduces the same late Strix/OpenAI-compatible streaming contract failure, confirming this is not #40-specific and not repairable in Orgmetra source.

    • consumer: ContextualWisdomLab/Orgmetra#42
    • exact target head: fb03c0837b38424412fa774576a8ded0f9847896
    • exact live base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • Required Strix run: 33298604116, attempt 3
    • job: 99249799353
    • trusted central workflow SHA used by this run: 71abbd1a8a67adfce72a5a016c974b7df246f964
    • pinned contextual-orchestrator: 30c6d71680e659f25a0a433d4726ad0d437f9757
    • diagnostic-only artifact: 9731510432, SHA-256 79c88d64158c2402988bca10109745ec6aa0bbe5aba43180a75f26ca9beef32f

    This run passes exact target/base materialization, required-workflow smoke, 5/5 provider-secret gate, sidecar /healthz + provider-route, gateway chat/completions preflight, hash-pinned strix-agent==1.5.3 installation, and PR-scoped input materialization (3 scannable files). The sidecar retains two runtime-ready NVIDIA NIM routes. Strix then starts for real.

    All three bounded scanner attempts fail before the first model response with the same HTTP 400 contract error seen on #40: invalid_stream_options — stream_options.include_usage=true is not supported with tools or response_format. The wrapper backs off 90s/180s and then correctly fails closed as STRIX_PROVIDER_UNAVAILABLE. Vulnerabilities 0 and Tokens 0 are not clean evidence because no first model response or authoritative structured finding/no-finding result exists.

    This independently confirms the causal boundary is the shared Strix/OpenAI-Agents ↔ contextual-orchestrator OpenAI-compatible streamed-tool request contract, not Orgmetra application code. Preserve the existing remedies/acceptance criteria already recorded on this owner issue; do not add a leaf shim or synthetic pass.

    GREEN canary: rerun Required Strix without changing Orgmetra head fb03c083...; the first streamed tool/response-format request must be accepted by the trusted gateway, the bounded scanner must complete, and an authoritative structured finding/no-finding artifact bound to this exact head/base must terminate GREEN.

  18. seonghobae commented on Aug 30, 2026

    @seonghobae
    ContributorAuthor

    GREEN downstream acceptance canary for the existing Strix owner boundary; no Orgmetra workaround was introduced.

    • consumer: ContextualWisdomLab/Orgmetra#40
    • exact unchanged consumer head: 6917e41f9053fab6f7e99f8185f2137e8fc5fca5
    • exact live base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • Required Strix run: 33294991609, attempt 7
    • replacement job/check: 99255954448 — terminal SUCCESS
    • trusted central workflow source actually checked out: .github@702392a2fb80149d25734a4cae173ed5fecd6d83 (fix(strix): disable SDK streaming for contextual-orchestrator gateway (#1448))
    • contextual-orchestrator pin actually vendored: 30c6d71680e659f25a0a433d4726ad0d437f9757

    The trusted flow materially executed end to end: exact target/base resolution, 5/5 provider-secret gate, sidecar /healthz + provider-route, gateway chat/completions preflight, pinned strix-agent==1.5.3 install, four PR-scoped scannable-file inputs, and Run Strix (quick) all succeeded. Strix reports Penetration test completed, Vulnerabilities 0 (No exploitable vulnerabilities detected), and Strix run succeeded for model 'orchestrator/free' in 184s; model accounting recorded ~172.2K input tokens / 211 output tokens.

    The uploaded report artifact is 9732103749, 11,372 bytes, SHA-256 e38ba72f4e51285209182e4402e0f27186a75ab7fbd3ba14fde458a960ec4e63.

    This satisfies the unchanged-head GREEN acceptance previously requested here for the Orgmetra #40 canary: the prior invalid_stream_options / loopback compatibility boundary no longer prevents a material authoritative Strix scan. This comment is consumer acceptance evidence only; the dedicated owner can decide issue lifecycle/closure based on the wider cross-repository contract.

  19. seonghobae commented on Aug 30, 2026

    @seonghobae
    ContributorAuthor

    Fresh Orgmetra unchanged-head materialization canaries after the current central Strix repairs remain fail-closed at the required-workflow owner boundary; no Orgmetra source/ref/workflow/settings mutation was made.

    Two exact consumers reproduced a pre-job cancellation when the existing failed Strix job was re-run through GitHub's supported rerun path:

    • ContextualWisdomLab/Orgmetra#45@0023f13469cc0f6f9d1da08f3b1548062fa61dbe, live base develop@9e3e4847510e1e612b48474ba42b177b8ed824df: run 33295582098, attempt 2, terminal cancelled; latest jobs endpoint returns total_count: 0.
    • ContextualWisdomLab/Orgmetra#47@92bdb21bf4b9e19190ccd23409d55ed7102ee663, same live base: run 33301542487, attempt 2, terminal cancelled before any authoritative scanner result.

    These heads already have separate Orgmetra-local quality evidence and prior Strix causal canaries. This comment does not infer a deeper cause from cancellation alone. It records the narrower acceptance failure: a required Strix rerun must materialize an exact-head job, perform the bounded scanner execution, and terminate with authoritative finding/no-finding evidence. Cancelled/zero-job/status-only evidence remains non-passing. Please preserve this materialization canary alongside the successful unchanged-head #40/#42 downstream canaries; no leaf shim, no-op commit, route override, or gate weakening is appropriate.

  20. seonghobae commented on Aug 30, 2026

    @seonghobae
    ContributorAuthor

    Fresh Orgmetra consumer evidence exposes a separate Required Strix evidence-integrity false-green at the existing Strix owner/control-plane boundary. This is an owner handoff only; no .github source/ref/workflow/settings mutation was made from the Orgmetra loop.

    Consumer and exact provenance:

    • repository/PR: ContextualWisdomLab/Orgmetra#51
    • exact head: aa33f8f9f2a7c0a72c91947e3edac90334bcaabf
    • live base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • Required Strix run: 33231136865, attempt 4
    • Strix job: 99317767069, terminal SUCCESS
    • trusted central workflow SHA: 1ff8268255b061461d9d49b4cab4febf9a8e7bfa
    • vendored contextual-orchestrator SHA: 30c6d71680e659f25a0a433d4726ad0d437f9757
    • report artifact: 9738615449, 11,587 bytes, SHA-256 4dc2d2c102a5ca0686fb3a4069e4539a32f59df92a85029befdcde1462121a6b

    The execution path now materially works: trusted target/head materialization succeeds; the sidecar reaches health/provider preflight and chat/completions; the PR scope retains 2 scannable changed files; Run Strix (quick) executes and exits 0; artifact collection/upload succeeds. Console output says Vulnerabilities 0 (No exploitable vulnerabilities detected) and the uploaded findings.sarif contains zero results.

    However the SAME terminal-success artifact is internally contradictory and not defensibly bound to this target. penetration_test_report.md says: An external assessment of the Acme Customer Portal identified multiple weaknesses that could lead to unauthorized access to customer data. The same unrelated sentence is persisted as run.json -> scan_results.executive_summary, while that run.json simultaneously marks the scan completed/successful. The report does not identify Orgmetra or the exact PR head, and its prose claims multiple weaknesses while the canonical SARIF/result count is zero. Under the current Required Strix contract, this contradictory untrusted LLM prose still permits a terminal GREEN required check.

    This is not an Orgmetra source defect: a branch-local shim cannot authenticate or reconcile a foreign required-workflow scanner artifact without weakening/bypassing the owner control. The first causal boundary is the central Strix evidence validator/gate accepting an internally inconsistent, target-mismatched report as authoritative passing evidence.

    RED/acceptance canary for the existing owner lane:

    1. Bind machine-readable Strix evidence to exact consumer repository, PR number, exact head SHA and base SHA independently of model prose.
    2. Treat LLM-generated narrative as untrusted explanatory draft evidence only; never let it establish pass/fail by itself.
    3. Fail closed when the final report/run metadata names an unrelated target/product, lacks required exact-target provenance, or conflicts materially with the canonical machine-readable findings/result count.
    4. Specifically, results=[] / Vulnerabilities 0 plus prose claiming multiple weaknesses ... unauthorized access to customer data must be classified as evidence-integrity conflict, not GREEN.
    5. Preserve genuine findings: any canonical qualifying finding remains hard-failing and must never be downgraded because the prose disagrees.
    6. Re-run the unchanged Orgmetra#51 head after the owner repair. GREEN requires exact-target provenance plus mutually consistent structured result and report metadata, then a terminal Required Strix result.

    The attempt-4 runtime/provider recovery itself is useful progress; this handoff is specifically about the newly exposed artifact-integrity boundary after execution finally completed.

  21. seonghobae commented on Aug 31, 2026

    @seonghobae
    ContributorAuthor

    Fresh independent Orgmetra canary confirms the remaining Strix evidence-integrity defect is central, not an Orgmetra source finding.

    • consumer: ContextualWisdomLab/Orgmetra#71
    • exact PR head: 1d7ed5bbfb18b35d589c69a6e8d93754ba299589
    • exact protected base observed: 9e3e4847510e1e612b48474ba42b177b8ed824df
    • required Strix run/attempt/job: 32843324802 / attempt 2 / 99329024386
    • trusted central Strix workflow source: 1ff8268255b061461d9d49b4cab4febf9a8e7bfa
    • uploaded artifact: 9739849083, digest sha256:5159373118a005175ff7e2e512bfc07b4ae6bc186abbc77eab6d7051ccc1726c

    The job materially fetched the exact base and exact head, retained one scannable changed file, ran the pinned Strix path, and concluded success. The scanner emitted Vulnerabilities 0 / No exploitable vulnerabilities detected and the wrapper therefore returned GREEN. However, in the same terminal scan summary the model reported an unrelated target and contradictory findings: An external assessment of the Acme Customer Portal identified multiple weaknesses that could lead to unauthorized access to customer data, followed by an SSRF remediation recommendation. That narrative is neither Orgmetra nor compatible with the structured zero-vulnerability result.

    First failing boundary: scanner/wrapper evidence validation does not fail closed when the generated narrative is target-mismatched or contradicts the structured result. This means a mechanical Strix success cannot currently be acquisition-grade evidence for this exact consumer even though the target checkout itself was correct.

    RED acceptance canary for the existing owner repair:

    1. Re-run the unchanged Orgmetra Centralize PR governance scheduler contract #71 exact head/base.
    2. Artifact/report metadata must bind repository, PR number, exact head and exact base.
    3. Generated narrative must be attributable to that target and logically consistent with structured findings/SARIF/result counts.
    4. Any target mismatch, unrelated named system, or structured↔narrative contradiction must fail closed rather than publish GREEN.
    5. Only a target-bound, internally consistent terminal artifact may qualify as Strix evidence.

    Smallest remedy belongs in the central Strix report-validation/wrapper boundary; no Orgmetra-local workaround or foreign-repository mutation is appropriate. Required GREEN proof is an unchanged-head rerun of #71 producing provenance-bound, internally consistent evidence or a deliberate fail-closed result if consistency cannot be established.

  22. seonghobae commented on Aug 31, 2026

    @seonghobae
    ContributorAuthor

    Fresh Orgmetra exact-head canary confirms the same trusted Strix local-transport contract mismatch on the active product-gap baseline lane; no Orgmetra source workaround is correct.

    • consumer: ContextualWisdomLab/Orgmetra#100
    • exact head: 0d46d1878cde4baa27934e28f25687a803124473
    • independently refetched protected base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • Required Strix run/job: 33296405370 / 99216732085
    • trusted central workflow SHA: 6c8ee24046d743b3981c566c6e29f99f09137f6a
    • pinned contextual-orchestrator: b21645116b352967e50fc497b87eb745b9cc8c61
    • pinned scanner: strix-agent==1.5.3
    • diagnostic artifact: 9727625024, SHA-256 96bc1aece35bbdb5fb77cb4de93fc96ccaadd2479c5dd28edd6e2640b31010f2

    Passed boundaries before failure: exact PR/base materialization, trusted required-workflow smoke, provider credential gate, contextual-orchestrator /healthz, provider-route preflight, gateway chat/completions preflight, pinned Strix install/trust checks, and PR scoping to one scannable changed file.

    First causal boundary is central and occurs before vulnerability analysis: the trusted workflow intentionally writes LLM_API_BASE_FILE=http://127.0.0.1:18080/v1, then its own trusted Strix quick gate rejects that configured local sidecar with ERROR: LLM_API_BASE must be an https URL when configured. and exits 2. No authoritative finding/no-finding report exists. The artifact is diagnostic-only and remains non-passing.

    Smallest owner-boundary remedy remains narrow: allow HTTP only for the exact trusted workflow-launched loopback contextual-orchestrator endpoint (127.0.0.1:18080[/v1]) under the existing authenticated local-sidecar/token/health/preflight controls, while preserving HTTPS-only rejection for every non-loopback configured API base and all existing fail-closed provider/finding semantics.

    RED/GREEN acceptance for this consumer: without changing Orgmetra head 0d46d187…, a repaired trusted workflow must pass the transport validation, actually enter bounded Strix analysis against the exact head/base, and publish a terminal authoritative finding/no-finding result bound to repo/PR/head/base. Setup-only success, exit-2 configuration failure, diagnostics-only artifacts, skipped/neutral/model-only results, or predecessor evidence remain non-passing.

  23. seonghobae commented on Aug 31, 2026

    @seonghobae
    ContributorAuthor

    Fresh Orgmetra unchanged-head retry canary exposes a distinct required-workflow materialization failure before Strix can execute.

    • consumer: ContextualWisdomLab/Orgmetra#141
    • exact unchanged head: 8d12ee7cd05549a69c6debf5e0de0c5fc8668f33
    • live protected base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • required Strix run: 33200837826
    • original failed Strix job: 98949661352
    • supported failed-job rerun was accepted on 2026-08-31 and created run attempt 2
    • attempt 2 terminal state: completed/cancelled
    • attempt 2 job inventory: total_count=0, jobs=[]

    The original attempt had already materialized the exact target and reached Strix before failing at the central provider/routing boundary. The supported unchanged-head retry now fails even earlier: GitHub creates attempt 2, but the required workflow terminates cancelled without materializing any job. Therefore no checkout, trusted-source validation, scanner execution, structured finding/no-finding result, or report artifact exists for the retry. This cannot be repaired correctly in Orgmetra source, and a no-op leaf commit would only manufacture a new head rather than repair the owner boundary.

    RED acceptance for the existing central owner lane: retrying an eligible open, non-draft required-Strix consumer on an unchanged exact head must either materialize a job bound to the exact repository/PR/head/base, or fail closed with an explicit typed cause that preserves actionable owner evidence. cancelled with zero jobs is non-passing and insufficiently attributable.

    GREEN proof: rerun this same unchanged #141 head after the owner repair; require a materialized Strix job under the then-current trusted central source, exact repo/PR/head/base binding, substantive scanner execution, and an authoritative internally consistent structured report with terminal source-backed pass/fail. Do not weaken fail-closed semantics or reinterpret zero-job cancellation as scan evidence.

  24. seonghobae commented on Aug 31, 2026

    @seonghobae
    ContributorAuthor

    Fresh follow-up on the existing Orgmetra #141 unchanged-head canary proves the current zero-job retry attempt is stranded under GitHub's supported retry paths, not merely non-materialized.

    Consumer: ContextualWisdomLab/Orgmetra#141

    • exact unchanged head: 8d12ee7cd05549a69c6debf5e0de0c5fc8668f33
    • live protected base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • required Strix run: 33200837826, current run_attempt=2, terminal cancelled, current attempt has zero jobs
    • original materialized attempt job: 98949661352

    After re-fetching the unchanged run/head/base, two supported recovery paths were exercised without changing Orgmetra source or refs:

    1. re-run failed jobs for run 33200837826 -> HTTP 403: This workflow run cannot be retried
    2. re-run original materialized job 98949661352 -> HTTP 403: Only jobs from the current attempt can be re-run

    Because attempt 2 contains no jobs, there is also no current-attempt job that can be individually retried. The required-workflow consumer is therefore trapped: GitHub has replaced the useful materialized attempt with a zero-job terminal attempt while rejecting both normal retry mechanisms. A no-op Orgmetra commit would create a different head and manufacture a new trigger rather than repair this owner boundary.

    Smallest owner-boundary remedy: ensure a required-Strix retry cannot replace a materialized attempt with a terminal zero-job attempt without a recoverable/typed dispatch path, or provide the existing required-workflow owner path a supported fresh dispatch/retry mechanism that remains bound to the same unchanged consumer head after transient cancellation.

    GREEN acceptance: without Orgmetra head churn, routine admin bypass, or gate weakening, the then-current exact #141 head must receive a newly materialized required Strix job that proves checkout of the exact repo/PR/head/base, validates trusted central source, executes substantive scanning, and publishes internally consistent provenance-bound structured evidence with a terminal source-backed verdict. A zero-job cancellation or an unretriable workflow shell remains non-passing.

  25. seonghobae commented on Aug 31, 2026

    @seonghobae
    ContributorAuthor

    Fresh downstream Strix retry/materialization canary from ContextualWisdomLab/Orgmetra#56.

    • exact unchanged leaf head: 68af42cb80807b6638d745d1687fd3c6a814d64f
    • protected live leaf base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • required Strix run: 33262099906
    • current run state: terminal CANCELLED
    • current jobs endpoint: total_count=0, jobs=[]; there is no scanner/materialized finding-or-no-finding job to retry
    • a fresh supported rerun-failed-jobs request on the unchanged head now returns GitHub 403 This workflow run cannot be retried

    This is therefore a central required-workflow retry/materialization boundary, not an Orgmetra vulnerability finding or leaf source defect. No no-op leaf commit or scanner shim is appropriate.

    RED acceptance: an unchanged protected consumer head can become stranded with a required Strix run that is cancelled, exposes zero current-attempt jobs, and is rejected by the supported run-level retry API.

    GREEN acceptance: after the owner repair, the same unchanged consumer head must have a supported recovery path that materializes a real Strix scanner job and reaches an exact-repository/PR/head/base-bound structured finding-or-no-finding verdict. Cancelled/status-only/predecessor evidence must remain non-passing.

  26. seonghobae commented on Sep 1, 2026

    @seonghobae
    ContributorAuthor

    Second independent Orgmetra canary now reproduces the stranded zero-job Strix retry state previously seen on Orgmetra #141.

    Consumer: ContextualWisdomLab/Orgmetra#59

    • exact unchanged head: f361012212d4c90e20acb9edaa5302850a340573
    • live protected base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
    • required Strix run: 33279409151
    • supported failed-job rerun created run_attempt=2
    • attempt 2 is terminal completed/cancelled
    • current attempt materializes zero jobs (jobs=[])
    • after refetching the unchanged head/run, GitHub's supported re-run failed jobs path was exercised and returned 403 This workflow run cannot be retried

    This is before any scanner/provider finding can exist, so an Orgmetra source workaround cannot repair the first failing boundary. It independently confirms the #141 materialization/retry dead-end rather than a leaf security defect.

    GREEN acceptance: after the central owner repair, a fresh supported event for unchanged Orgmetra #59 head must materialize a real Strix job, bind it to the exact head/current central source, and reach an authoritative terminal finding/no-finding result. A cancelled zero-job attempt, status-only record, or predecessor scan must remain non-passing.

  27. seven7763 commented on Sep 1, 2026

    @seven7763

    The three root causes in the Ask list are each independently fatal to the fallback chain, and the order matters for whoever picks this up — fixing (2) alone still dies on (1), and fixing (1)+(2) still dies on (3):

    1. Slug mapping (openai-direct → openai_direct): the mapping layer at strix.yml:784-788 is the only place the provider slug gets normalized, so an unknown-slug fallback silently routes to the default base URL instead of failing loudly. A set -euo pipefail-style assertion on the slug against a known-provider list (or a literal case whitelist) would convert this from "404 at runtime, 4s before the gate fails" to "config error at workflow parse time."

    2. Missing /v1 segment when the base URL comes from STRIX_OPENAI_API_BASE_FILE: this is the same assembled-vs-configured URL divergence that bites most custom-endpoint setups. Cheapest guard: log the finally-assembled URL once per job (echo "resolved base: ${BASE}"), because a 404 with the resolved URL visible is a 10-second diagnosis instead of a stranded canary.

    3. Model-id tier mismatch (gpt-5.4 vs gpt-5.4-turbo-style alias): worth confirming with a direct probe before the next canary spend, since a contract that names an id the key tier can't see will fail identically after (1) and (2) are fixed:

    curl -s "$BASE/v1/models" -H "Authorization: Bearer $KEY" | jq -r '.data[].id' | grep -F 'gpt-5.4'
    

    Suggested sequencing: fix (1) → probe (3) directly → fix (2) → re-run the scan. That way the retry canary isn't burned on a chain that still has an unknown failure mode downstream. Happy to help verify (3) if you can share what the key tier expects for the direct-OpenAI alias shape.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority: highHigh-priority or P1 work

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions