You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
/metrics: expose the speculative draft counts (offered / accepted) for llm-visuals #457
Hi, I run Strata 0.1.34 with Qwen3.8-Flash-Next IQ3_S at 262K on my RTX 5090 behind llama-swap (119-135 tok/s decode, ~5,000 tok/s prefill, 0.8 s to resume a 71K conversation), thank you for it.
I monitor it live with llm-visuals, a terminal dashboard that polls Strata's /metrics every second (its Strata adapter is on the branch feature/strata). Throughput, context, layers and requests work; the speculative decoding panel stays empty: "Strata does not report draft acceptance".
The server already has the MTP draft counts: serve/server.py reads them from the engine's DONE line (self.last.update(drafts_accepted=..., drafts_offered=...)), returns them in the response timings as draft_n / draft_n_accepted (#83), and the server log prints them ("drafts accepted 601 of 1181").
They are missing from GET /metrics: not in live, not in the requests history, not in totals (checked on 0.1.34). From the response timings I only get the figure once a request has finished; what I would like to see in llm-visuals is the acceptance while the model generates.
Proposal (additive, no change to existing fields):
in live, while generating: "drafts_offered" and "drafts_accepted" so far for the current request (if the engine can report them before DONE);
in each requests entry: the same two counts for the finished request (null when the engine did not report them, as draft_n is omitted today);
Added in 0.1.35: each /metrics request entry has drafts_offered and drafts_accepted (null when the engine didn't report them), and totals keeps their running sums. Live counts during a request need an engine protocol change, so they're for later. Thanks for the idea!
Activity
/metrics: each request's speculative draft counts, and their sums in …
Engine 0.1.35: Windows AMD uses its bundled HIP runtime (#468 #461), …
Niko1221 commentedon Oct 2, 2026
Added in 0.1.35: each
/metricsrequest entry hasdrafts_offeredanddrafts_accepted(null when the engine didn't report them), andtotalskeeps their running sums. Live counts during a request need an engine protocol change, so they're for later. Thanks for the idea!rebase fixes: multi-session QSA ordinals in the park snapshot, new st…
rebase fixes: multi-session QSA ordinals in the park snapshot, new st…
/metrics: each request's speculative draft counts, and their sums in …
Engine 0.1.35: Windows AMD uses its bundled HIP runtime (#468 #461), …
/metrics: each request's speculative draft counts, and their sums in …
Engine 0.1.35: Windows AMD uses its bundled HIP runtime (Niko1221#468 N…
/metrics: each request's speculative draft counts, and their sums in …
Engine 0.1.35: Windows AMD uses its bundled HIP runtime (Niko1221#468 N…