[justindressler]

Problem and result

Problem. Strata can dump logprobs for verification, but there is no per-request way to
read the logits of arbitrary label tokens at the final position. Without that, the engine
cannot serve a System One decision contract — a caller that wants a typed answer with a
calibrated probability (noul yes/no, choice pick-one, score ordinal) has to make the
model generate prose and then parse it, which is slow, unparsable in the failure case, and
gives a confidence the model chose to write rather than a number the caller can threshold.

Result. An opt-in engine wire (--jev-brow) that returns the final-row logits for
exactly the label tokens asked, plus a serve-layer POST /v1/systemone endpoint that applies
the Jev readout math and per-tuple post-hoc calibration (Platt for noul, temperature for
choice/score). One decode step produces a typed answer with a probability and a margin;
the same model keeps serving chat, can answer decisions inside a chat call, and can decide
then narrate. Everything is gated by STRATA_JEV_MODES; with it unset, the process behaves
exactly like stock Strata (a decision surface that is not enabled returns 404, so probing
learns nothing).

Related discussion

• No issue or scope agreement exists for this one: it is a direct port of the jevcpp
  reference implementation's wire and math onto the Strata engine. Opening it as a PR rather
  than asking first is deliberate, and a maintainer can redirect it to an issue if that is
  the preferred shape.
• @TwoAnts commented in this thread that reading jev to check the stop token was something he
  had considered — that is the same readout, used for a different purpose.
• Prior close: 2026-10-19 tracker reset for 0.2.0-beta, not a content rejection. Reopening
  through /reopen.

Approach

• Engine (additive, 5 files, +149 lines): Verifier::brow_row in verify.hpp/.cpp plus
  the --jev-brow flag; final-row logits for a supplied token list, fail-closed on any
  mismatch. No change to existing decode paths.
• Serve: /v1/systemone with the donor's error contract (422 +
  {"error": {"field": ...}}), the frozen D3 prompt block, and readout math ported from
  jevcpp. Calibration is post-hoc and argmax-preserving, per tuple (mode|qtype|set), hot-
  swappable through POST /admin/calib, with donor-schema log rows for the offline fit.py.
• Mode gating: STRATA_JEV_MODES=chat,decision (default for the jevport presets),
  decision alone, or unset for stock behaviour; unknown tokens refuse to boot.
• Compatibility: no wire change to /v1/chat/completions unless "jev" is present; no
  new default; model format unchanged. Divergences from the donor plan are listed in §4d of
  the detail below, each with the reason.

Validation

Revision: branch jev-port head fbffc99a, base 296b59d2 (v0.1.43 lineage).
Environment: RTX 3090 24 GB, QFN = Qwen3.8-Flash-Next IQ3_XXS MoE, 32k ctx, calibration
tuple p-jevcpp-qwen38-27b|q4_k_xl|timber|v1|V|all.

Passed:

• Jev unit suite (mock engine) 73/73; full serve/discovery run 858 executed, 0 failed,
  11 skipped.
• Concurrency: 7 clients × 3 questions FIFO-serialized, zero cross-talk observed.
• Calibration A/B on held-out items, offline replay of identical prompts: per-tuple accuracy,
  NLL and ECE15 pre→post — table in §4a below.
• Head-to-head against the donor's own benchmark (same three tasks, different model and
  quant): §4b below.
• Warm single-client performance, n=10 per cell (p50 / p95 / rps): §4c below.
• Live responses captured for text, image-direct, decide-then-narrate, multi-question and
  inline-chat shapes: §2 below.

Failed:

• none observed on this revision.

Not run:

• Re-validation on current main (v0.2.1.4, 9de04cd4). This branch is 42 commits behind
  and mergeable_state is dirty, so a rebase is required before the numbers above can be
  re-claimed on the current tree. Queued next.
• HIP and SYCL builds — only CUDA was built and executed.
• Any GPU other than the RTX 3090, and any model other than QFN.

Limits

• Calibration is per model and per quantization; the shipped tuples are for
  QFN/IQ3_XXS/32k. Another model or quant needs its own refit before the probabilities mean
  anything — the PR ships the fitting path, not universal numbers.
• Multiple questions against one state cost roughly one readout each until prefix reuse
  lands (that is the separate suffix-reuse PR, [Engine] suffix-reuse: conversation KV prefix mount, quant-safe splices, state checkpoint ladder #2351).
• image_mode: direct reads raw image embeddings; it was validated on the QFN vision path
  only.
• Deliberate gaps against the donor plan are enumerated in §4d, not silently dropped.

---

Background and design detail

What a System One decision model is. Most uses of a language model are
"System Two": the model writes text and the caller reads, parses and trusts it. A
System One decision model does the opposite. It receives a state (text, JSON, or an
image) and a set of typed questions, and returns typed answers with calibrated
probabilities: a yes/no (noul), a pick-one (choice), or an ordinal rating (score).
It never generates prose to decide. Nothing has to be parsed, and the confidence is a number
the caller can threshold, route on, or log, rather than a sentence the model chose to write.
That makes it suited to fast, repeatable, auditable decisions inside a loop: routing,
guardrails, triage, scoring, and game or agent control.

Jev. "Jev" is TypeSafe's commercial System One decision API: POST /v1/systemone with
{state, questions}, answers read from token probabilities rather than generated text. Its
request/response shape is the wire this work follows. Open implementations of the same idea
already exist (see References); the goals here are correctness, speed and honest
probabilities, not novelty.

Laya. Laya (convaiinnovations/laya, Apache-2.0) approaches the same contract with a
different mechanism: a small purpose-trained encoder (ModernBERT-large, 421M parameters;
non-autoregressive) that answers all questions in one forward pass (~33 ms reported),
trained with reinforcement learning against strictly proper scoring rules (RLCD) so that
honest probabilities are what maximises reward. Two things in its documentation shaped this PR: its probabilities ship
over-confident until a temperature is refit per question type and option count
(reported mean ECE 0.466 → 0.081, with the instruction to do this on your own data before
trusting them), and its comparison against Jev uses third-party published Jev figures
that it did not measure itself. We kept the first idea (per-tuple calibration, refit on
local data) and measured our own numbers on our own harness. The mechanism differs: a
general LLM read at its label logits, with no task-specific training.

jevcpp. The reference implementation this port follows: a llama.cpp-based server that
serves the Jev wire from a generative model by reading the label logits at the final
prompt position (Qwen3.8-27B, with a shipped calibration file and a metamorphic/golden test
set). Its plan and amendments define the request contract, the frozen prompt template, the
readout math and the test strategy that this PR ports.

What this PR does. It brings that System One surface to Strata. Strata's engine could
already dump logprobs for verification, but had no per-request way to read the logits of
arbitrary label tokens. This PR adds an opt-in --jev-brow engine wire (BROW) that returns
the final-row logits for exactly the label tokens asked, and a serve-layer
/v1/systemone endpoint that applies the Jev readout math and per-tuple post-hoc
calibration (Platt scaling for yes/no, temperature scaling for choice/score) on top. The
same model keeps chat, and can answer decisions inside a chat call, from an image directly,
or decide-then-narrate. It is opt-in per process (STRATA_JEV_MODES).

Donor reference: jevcpp on Qwen3.8-27B (q4_k_xl), shipped calibration tuple
p-jevcpp-qwen38-27b|q4_k_xl|timber|v1|V|all (T_choice 1.5861). This port runs on
QFN = Qwen3.8-Flash-Next IQ3_XXS MoE (RTX 3090, Strata engine v0.1.43-lineage, 32k ctx).

1. Features (the full jevcpp surface)

Feature	Request	Status
Core decision endpoint	POST /v1/systemone {state, questions} — noul (yes/no), choice (letters), score (0-10 levels)	✅ live
Frozen D3 prompt	byte-frozen question block (donor render, enable_thinking=false, verbatim string states)	✅ golden-tested
BROW readout	--jev-brow engine flag; final-row logits, fail-closed	✅ engine commit
Calibration (M3.2)	post-hoc, argmax-preserving: Platt (a,c) for noul, temperature for choice/score; per-tuple (L/V/D/S) files calib-<mode>.json; POST /admin/calib hot-swap; donor-schema log rows for fit.py	✅ live
V-mode (M3.1)	header X-Jev-Mode: V — per-option yes/no branches, p = softmax(s_yes − s_no), name-keyed	✅ live
Image → decision (M3.8)	image_b64 + image_mode: summary (vision → 160-tok summary → decide) or direct (raw image embeddings + question, ONE readout)	✅ live
Decide-then-generate (M3.9)	"then": {"generate": {...}} — answers folded into the last user turn, one wire request	✅ live
Chat surface (M3.3)	POST /v1/chat/completions with `"jev": {state	messages, questions}`
Concurrency	FIFO-serialized engine turns; 7 clients × 3 questions, zero cross-talk (tested)	✅ tested

Error contract: donor's 422 + {"error": {"field": ...}}. Streaming refused with the
jev extension, state XOR messages, image_mode validated, D+V refused with 422.

2. Usage

# text decision (one decode step)
curl -s $URL/v1/systemone -d '{
  "state": {"engine": "hot", "load": 0.9},
  "questions": {"q1": {"type": "noul", "instructions": "Throttle?"}}}'

# image → decision, direct mode (raw image tokens decide)
curl -s $URL/v1/systemone -d '{
  "image_b64": "'"$(<screen.b64)"'", "image_mode": "direct",
  "questions": {"threat": {"type": "noul", "instructions": "Enemy visible?"}}}'

# decide, then narrate
curl -s $URL/v1/systemone -d '{
  "state": {"hp": 12}, "questions": {...},
  "then": {"generate": {"messages": [{"role": "user", "content": "Narrate."}],
                        "max_tokens": 128}}}'

# MULTIPLE questions, mixed types, ONE state (each question = its own readout;
# cost scales ~linearly until prefix reuse lands)
curl -s $URL/v1/systemone -d '{
  "state": "Reactor at 84%. Coolant loop A nominal, loop B pressure rising slowly.",
  "questions": {
    "threat":  {"type": "noul",   "instructions": "Is loop B a current threat?"},
    "action":  {"type": "choice", "instructions": "Which action first?",
                "criteria": {"inspect": null, "throttle": null, "shutdown": null}},
    "urgency": {"type": "score",  "instructions": "How urgent is intervention?",
                "criteria": ["none", "low", "medium", "high", "critical"]}}}'
# -> {"answers": {"threat":  {"type": "noul", "noul": 0.5429, "calib": "platt_v1"},
#                 {"action":  {"type": "choice", "choice": 0,
#                              "probabilities": [0.830, 0.160, 0.010],
#                              "confidence": 0.7453, "calib": "T_choice_v1"},
#                 {"urgency": {"type": "score", "score": 1.5928,
#                              "probabilities": [0.103, 0.374, 0.376, 0.121, 0.026],
#                              "confidence": 0.2201, "calib": "T_score_v1"}},
#      "margin": top1−top2 present on every answer (cascade-routing number)}
#     "usage": {"prompt_tokens": 303, "readout_tokens": 3}}
# (live response, QFN; note each answer carries its OWN calibrator id)

# INLINE jev on the chat surface — decisions inside an OpenAI chat call; the
# conversation itself is the state (v1-chat template), no "state" field needed
curl -s $URL/v1/chat/completions -d '{
  "model": "jev",
  "messages": [{"role": "system", "content": "You are the plant supervisor."},
               {"role": "user", "content": "Loop B pressure is rising. Inspect now?"}],
  "jev": {"questions": {"inspect":
      {"type": "noul", "instructions": "Should loop B be inspected now?"}}}}'
# -> {"id": "jev-1792055290765", "object": "jev.response",
#     "model": "qwen3.8-flash-next-iq3_xxs-jevport",
#     "jev": {"answers": {"inspect": {"type": "noul", "noul": 0.9230,
#                                      "calib": "platt_v1"}},
#             "usage": {"prompt_tokens": 82, "readout_tokens": 1}}}
# (live response; jev answers ride alongside the normal chat fields, so any
#  OpenAI client gets decisions without a second endpoint)

# MODE GATING — decisions are OPT-IN (donor --modes; the default differs, see section 4d)
# STRATA_JEV_MODES unset            -> "chat" ONLY: a process that never mentions
#                                      jev behaves exactly like stock Strata
# STRATA_JEV_MODES="chat,decision"  -> both (what the jevport presets run)
# STRATA_JEV_MODES="decision"       -> decision-only: no generation path at all
# Unknown tokens fail closed at load (serve refuses to start).
# "decision" gates /v1/systemone + the chat jev extension;
# "chat"     gates plain generation + then.generate narration.
# Gated surfaces return 404 not_found_error (probing learns nothing).

Full surface + examples: docs/JEV-SYSTEMONE.md. Refit calibration:
calib/README.md (the suite IS the procedure). Mode gating:
STRATA_JEV_MODES env ("decision" = decision-only — no generation path
exists on the instance at all; outputs are probabilities over the caller's own
labels. Unset = "chat" only: decisions are opt-in, and the jevport presets set
"chat,decision" explicitly. Fail-closed on unknown tokens and on gated
routes).

Why logit readout instead of asking the model for confidence: models asked to state their
own confidence are mostly overconfident with high calibration error (Groot &
Valdenegro-Toro, TrustNLP 2024, see References). Reading label probabilities from the
logits and calibrating them post-hoc on held-out data (Platt / temperature scaling) is the
standard alternative, and it is what this PR does; its measured effect is in section 4a.
Laya reaches the same conclusion from the training side: its probabilities also need a
temperature refit on local data before they can be trusted.

3. Test strategy

Unit (mock engine) — 73/73 green (jev modules; full serve/ discovery 858 run, 0 fail, 11 skipped): readout math invariants, prompt rendering
(byte-pinned noul/choice blocks — the Yes/No-swap regression is now a golden test),
per-tuple calibration isolation, V-mode softmax, image two-call/direct shapes,
decide-then-generate, chat-surface validation arms, 7×3 concurrency + FIFO
serialization.

Live suites (calib/bench_suite.py): dev-split fit → held-out test eval,
standard top-label ECE15 + mean NLL, per-item JSONL logs. Guards: Platt slope
a≤0 refused (label/prompt inversion detector — it caught a real bug), temperature
at search boundaries flagged.

Adversarial audit protocol (Justin's rule, 2026-10-15): 1-in-10 of all test
items re-verified by hand — text golds re-read from source passages, image items
regenerated from committed seeds with sha256 match against the logged hash, sampled
error images independently re-counted with vision. Result across text+image+real
suites: 0 scoring hallucinations, 0 gold errors, 1 documented CV-Bench ambiguity.
Artifacts: audit-*.json, audit-imgs/, *-item-log.jsonl (920 rows).

Real-benchmark converters (calib/convert_benchmarks.py): POPE adversarial,
CV-Bench, CountBenchQA via HF datasets-server; banned-field self-test (answer-bearing
fields never enter requests), sha256 image manifest, dev/test image-disjoint, POPE
sampled by image at exactly 50/50.

Vision strategy (externally reviewed before implementation):
6-arm design (D/S/S-QC/S-V/blind/shuffled — the last two are contamination
detectors), circular rotation for choice position bias, phased screen→full→frozen
PR gate.

4. Benchmarking results (all live, QFN IQ3_XXS, RTX 3090)

4a. Pre/post calibration (held-out test, offline replay of identical items)

tuple (mode|qtype|set)	n	acc pre→post	NLL pre→post	ECE15 pre→post
L|noul|BoolQ	100	.890 → .910	.298 → .262	.084 → .033
L|choice|ARC	100	1.000 → 1.000	.005 → .007	.005 → .006
L|score|SST-5	100	.520 → .520	1.471 → 1.049	.301 → .154
D|noul|POPE-adv	60	.867 → .883	.204 → .195	.122 → .081
D|choice|CV-Bench	20	.650 → .650	.944 → .863	.145 → .260*
D|score|CountBench	20	1.000 → 1.000	.102 → .036	.091 → .033

*CVB n=20, bin-noise territory; NLL improves, acc flat. Fitted: L a=.6127 c=.6539
Tc=1.0557 Ts=2.263 · D a=1.0571 c=−.172 Tc=1.5456 Ts=.6555.
Also shown in calib-prepost-ab.json: the OLD single global calibrator made image
tuples WORSE than raw (CountBench ECE .091→.451!) — the per-tuple fix is the story.

4b. Benchmark vs jevcpp's own benchmark (donor m2_baseline, same 3 tasks)

Donor: Qwen3.8-27B q4_k_xl, dense, its own server. Port: QFN IQ3_XXS MoE, Strata.
Same task sets (BoolQ / ARC / SST-5 slices from jevcpp's eval data), same metrics
code, post-calibration numbers:

task	jevcpp 27B (donor)	jev-port QFN (this PR)	port, donor-ECE defn*
BoolQ noul acc	0.905 raw / 0.890 cal	0.910 cal	—
BoolQ ECE15	0.036 raw / 0.049 cal	0.033 cal	0.035 cal
ARC choice acc	0.990	1.000	—
SST-5 score acc	0.569	0.520 (−4.9pp ≈ 1 SE at n=100)	—
SST-5 ECE15	0.110 raw / 0.030 cal	0.154 cal	0.156 cal
SST-5 NLL	1.103 raw / 1.031 cal	1.049 cal	—

*Both ECE variants are top-label, 15 bins; the donor scores each bin at its CENTER
((b+0.5)/15), this PR's harness scores the bin MEAN confidence. The donor-definition
column recomputes our logged raw_p under the donor's exact b13_eval formula so the
comparison is apples-to-apples. Under either definition the direction is the same.

The 26B-smaller IQ3 MoE matches the donor within ~1 standard error on noul and choice
(at n=100, SE≈3-5pp); score is −4.9pp (≈1 SE; the confounds are model size, MoE vs
dense, and quant — not separable at this n). Donor noul calibration HURT it
(0.905→0.890 acc); ours helps (0.890→0.910). SST-5 calibrated ECE remains ~5× the
donor's under both definitions — T_score=2.263 is doing damage control on a weak
5-way readout, not achieving donor-grade confidence. Stated plainly.

4c. Performance (warm, single client, n=10 per cell)

workload	p50	p95	rps
text 1 question	272 ms	368 ms	3.7
text 3 questions	1002 ms	1016 ms	1.0
image direct, 1 question	275 ms	362 ms	3.6
image summary, 1 question	1625 ms	2731 ms	0.6

These cells are the warm-repeat regime (identical or near-identical prompts;
the engine's prefix mount fires). Instrumented per-turn measurements
(temporary build, since reverted) showed the two regimes explicitly, at a
67-token prompt: fresh question ≈ 1.0 s end-to-end (~0.97–1.14 s prefill +
decode + readout); identical repeat ≈ 0.35–0.45 s (mount skips prefill).
Multi-question requests and benchmark items are mostly fresh-prompt, so
expect the fresh column when prompts vary; the p50s above reflect settled
repeat traffic. Prefill dominates in both — that is the case for the
state-KV-reuse follow-up.

vs donor (27B dense, its m1_perf matrix, 1q-1c-warm): 71 ms p50 / 14.0 rps.
Measured budget of the port's ~350 ms (proxy, fresh connection), decomposed live:

component	cost	note
serve HTTP + JSON + calibration	~1 ms	GET /health and /v1/models both p50 1 ms
engine turn (prefill + 1 decode + BROW readout)	~260 ms	prefill-dominated on fresh prompts
fresh TCP + auth handshake	~47 ms	keep-alive removes it (255 ms direct p50)
llama-swap proxy hop	~44 ms	direct-to-serve-port removes it (302 ms)

Streaming is not a factor: systemone responses are single JSON bodies with
Content-Length (donor contract refuses stream: true). The gap vs the donor is
the un-prefixed engine turn, not protocol overhead — the donor amortized prefill
across forks (prefix-once + seq_cp); this port re-prefills per request. State-KV
reuse / prefix-LRU (the separate suffix-reuse work, out of scope here) collapses
the prefill share and projects decisions to ~60–80 ms. Repeat-identical requests
already hit the engine's prefix mount: 278–300 ms via proxy, and long states
(~470 tok) repeat at the same cost, confirming the mount fires when the prompt is
a full prefix. Image direct ≈ text latency (encoder cached); summary costs ~6×
direct (full 160-token generation).

Determinism / noise floor (measured, mechanism identified): repeat-decision
probability drift on this engine is p99 |Δp| ≈ 0.04–0.07 with zero argmax
flips in 720 repeated decisions across idle, 8-decision and mixed 4-chat+4-
decision load (blockers-run/noise-floor.json). The donor's 5e-3 gate was a
DENSE-model gate and never applied here. Mechanism (verified in engine
source): MoE expert placement uses atomicAdd slot assignment
(src/prefill/moe_fused.cu place_kernel), so row order inside each expert is
warp-dispatch order — nondeterministic per run; float addition is non-
associative, so the expert GEMM sums differ in the last ulp and the divergence
compounds across layers; IQ3_XXS keeps the wobble above dequant noise and the
readout softmax amplifies it near p≈0.5–0.9 (Δp ≈ p(1−p)·Δd). It is a property
of the model+engine pair, not of load — the three load scenarios measured the
same floor. Decision margins (>0.9 typical) exceed the floor by an order of
magnitude; fitted calibrators average it out over the dev split. A
deterministic-placement option (sort rows by source index after the atomic
fill) would lower it, as an upstream engine follow-up — out of scope here.
Single-item probability deltas smaller than this floor (e.g. POPE +1.6pp at
n=60) must not be read as signal.

4d. Not ported / diverged (deliberate, vs the donor plan)

Donor item	Why not
--modes chat,decision gating	Ported as STRATA_JEV_MODES env (A0.1) with a deliberate divergence: unset = chat only (donor defaulted both-on). Opt-in keeps an unconfigured Strata process behaviorally identical to stock, per the repo rule that a new feature is opt-in and the default path stays unchanged. Values: "chat" / "decision" / "chat,decision"; fail-closed 404 on gated routes, refuse-to-start on unknown tokens. /admin/modes runtime flip not ported — restart is the policy boundary here
Prefix-fork + batched decode + naive differential (A2.7/A2.10)	Strata's engine exposes BROW final-row logits, not seq forks; correctness is anchored by the golden battery + the ported jevref math, not a naive twin
V2 image-in-branch (crashed ggml)	Superseded by image_mode: direct — same single-readout goal, no fork, D-tuple-calibrated, benched on POPE/CVB/CountBench
Chat-KV reuse for decisions (A3.5, PARTIAL in donor)	Strata's conversation cache provides prefix reuse engine-side (272 ms warm L-mode)
B2 LoRA decision adapter	Donor's own gate: only if B1 left gaps; the score tuple is the candidate, deferred
Batched/naive/fallback golden HTTP variants	One serve path exists; the golden battery + 7×3 concurrency arm cover it

All 10 pre-PR blockers from the Opus review
(pre-PR external review) are closed: strict per-tuple
fallback, per-tuple log files, B1.4 tuple refusal, fail-closed /admin/calib,
token audit (PASS), a211 metamorphic battery, noise floor (see 4c), soak
(2,200 req, 0 jev errors, VRAM drift 0.026%), S-tuple fit (live), bootstrap
CIs + Brier (blockers-run/cis-brier.json).

5. Artifacts

• calib-prepost-ab.json — 4a's numbers + method (offline replay, identical items)
• perf-matrix.json — 4c (live measured)
• text-suite.md, image-suite.md, real-suite.md + *-item-log.jsonl — suite reports
• convert-manifest.json — real-benchmark provenance (sha256 per image)
• audit-text-sample.json, audit-image-sample.json, audit-real-sample.json — 1-in-10 audits
• Donor reference files: jevcpp/reports/m2_baseline.json, jevcpp/reports/m1_perf.json, jevcpp/calib/calib.json (all in the donor tree, not this repo)

6. Scope of change, compatibility, and testing status

Engine diff (5 files, +149 lines, additive): Verifier::brow_row (verify.hpp/.cpp),
the --jev-brow flag and BROW serve-loop handler (generate.cpp), a CMake test target
(verify_brow_test, CUDA builds only), and its test. Everything is opt-in: the handler
runs only when --jev-brow is passed and the server sends BROW; no existing request
path changes. One line runs on every verify window regardless of the flag
(last_row_T_ = T; in Verifier::run), an int store with no behavior change.

Default behavior: STRATA_JEV_MODES unset = chat only. Verified end to end with the
env var absent: /v1/systemone and the chat jev extension return 404, plain chat
returns 200. A stock Strata process is behaviorally unchanged.

Tested: CUDA build and GPU runs on an RTX 3090 (QFN IQ3_XXS, Qwen3.8-Flash-Next); the
full serve/ test discovery (858 tests) passes with the jev modules opted in.

Not tested in this PR: HIP, SYCL and Metal builds. brow_row is written for those
backends (the HIP compat headers map cudaMemcpy* to hipMemcpy*; a Metal branch is
included), but none of them was built or run as part of this PR.

7. References and prior work

Decision-model products and implementations

• TypeSafe Jev: the commercial System One decision API whose /v1/systemone wire this
  follows.
• Laya: Nandha Kishor M., convaiinnovations/laya (Apache-2.0).
  https://github.com/NandhaKishorM/laya, model card convaiinnovations/laya. Source of the
  RLCD training description, the one-forward-pass encoder design, and the per-(question
  type, option count) temperature-refit finding. Its README cites third-party Jev
  measurements of 236–276 ms p50 (AbdelStark/jev-benchmarks, nibzard/decision-model-benchmark).
• jevcpp: the llama.cpp-based reference implementation and its PLAN.md,
  PLAN.amendments.md, DECISIONS.md, m2_baseline.json, m1_perf.json, calib.json
  (donor tree, not part of this repo). Its plan also lists open Jev-compatible projects as
  prior art: DIY-Jev (Rust), llm2jev, verdict, jevper, jev-compatible-server.

Calibration

• J. Platt, Probabilistic Outputs for Support Vector Machines and Comparisons to
  Regularized Likelihood Methods, 1999: the Platt scaling used for yes/no.
• C. Guo, G. Pleiss, Y. Sun, K. Q. Weinberger, On Calibration of Modern Neural Networks,
  ICML 2017: temperature scaling (choice/score) and the Expected Calibration Error metric.
• T. Groot, M. Valdenegro-Toro, Overconfidence is Key: Verbalized Uncertainty Evaluation
  in Large Language and Vision-Language Models, TrustNLP 2024 (arXiv:2405.02917): LLMs and
  VLMs asked to state their own confidence show high calibration error and are mostly
  overconfident, the reason decisions are read from logits and calibrated externally.

Readout design (surveyed in docs/ research notes, 2026-10-15)

• EleutherAI lm-evaluation-harness and HELM: loglikelihood multiple-choice scoring,
  single-token letter labels, acc / acc_norm, and the smoke-test invariants adopted in
  the golden battery.
• OpenAI Cookbook, Using logprobs, and single-token classifier write-ups: first-token
  top_logprobs classification and top-1 − top-2 margin as the routing signal
  (returned on every answer here).
• Constrained-decoding systems (Outlines, SGLang, XGrammar): the generate-then-parse
  approach this design avoids, and its documented whitespace/prefix pitfalls.
• Real-time agent context: A Survey on LLM-Based Game Agents (arXiv:2404.02039) and
  The Latent Bridge (arXiv:2606.24470): why a single-pass fast path matters for
  frame-rate decision loops.

Benchmarks used for calibration and evaluation: BoolQ, ARC, SST-5 (text); POPE
(adversarial), CV-Bench, CountBenchQA (image).

[TwoAnts]

Wow, the other day I was actually thinking about using jev to detect whether the stop token in the answer is as expected, so as to prevent the task from ending illegally. Turns out it's already there now, haha.

[Niko1221]

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.

[justindressler]

Rebased onto main at 011c9e60 and re-validated on 0.2.0-beta: #3643 supersedes this one.

What changed in the move: 42 commits replayed with no conflicts, and the feature code is unchanged
(the 38 added files are byte-identical to this branch, and the 5 modified files carry the same added
lines). What was run on the new base: CUDA build (nvcc 13.0.88, arch 86) exits 0 with no new warnings;
ctest has 156 tests and the 71 GPU-class ones were re-run on an RTX 3090, where 70 pass and the one
exception needs libstrata_kq.so, which this CMake configuration does not build; the full serve/
discovery is green on CI (1002 tests, 14 skipped); and a live POST /v1/systemone on the 3090 answers
a noul/choice/score state with platt_v1 / T_choice_v1 / T_score_v1 and three readout tokens.

Not run: HIP, SYCL, Metal, Windows, macOS, and the app's UI tests. Those are listed in #3643 as not run
rather than claimed.