[street-tripy]

Title

Issue: Resolves #2857

Summary

With "parallel": N the server runs several requests at once, but the Monitor still shows one
request's numbers: the header pill reads the newest request's rate and the Model state card
shows one request's progress. There is no way to see on the page whether the requests are
decoding together or one after the other.

This adds a Workers card to the Monitor — one widget per batch slot with its own state, live
tok/s, tokens written and context fill — and makes the header pill name every decoding stream
(2 decoding · 80.4 + 76.1 tok/s). It is opt-in by construction: the card is in the markup
hidden and is shown only when /metrics reports live.parallel, which the server sets only for
a run config with "parallel": N.

What changed

• serve/web/index.html (+7): the workers-card markup after Model state, hidden by
  default, with workers-sum, workers and workers-note.
• serve/web/app.js (+63 / -1): workerWidgetsFor() builds one widget per slot the first time
  and reuses it, so state and progress transitions animate instead of being replaced every
  second; renderWorkers() fills it from the existing live.slots and is called from
  renderMonitor(). The pill line gained a branch: when two or more slots are decoding it lists
  their rates; otherwise it is the old text.
• serve/web/app.css (+17): the card's styles, from existing design tokens only (no new
  colors, fonts or selectors outside .worker*).
• serve/test_monitor.py (+11): test_the_workers_card_is_on_the_page_and_opt_in — the same
  page assertion as test_the_card_is_on_the_page, plus that the card ships hidden and is
  gated on live.parallel.

Extra Notes

• Nothing outside serve/web and this test. No server, setup or engine file changed, so
  there is nothing to build: no CUDA, HIP or SYCL file is touched.
• The default path is unchanged. Checked on this branch with a /metrics payload without
  parallel: the card stays hidden, no widget is built, and the pill reads
  Generating · 79.8 tok/s as before. With parallel: 2 both slots render with their own rate.
  No console errors in either state.
• Tests: python -m unittest serve.test_monitor serve.test_runconfig → 20 tests, all pass
  (19 on main + the new one). node --check serve/web/app.js is clean.
• No new API. The widget reads live.slots as slots_view() in serve/server.py already
  publishes it; the server and its tests are untouched.
• The widgets are filled one second at a time from the existing poll(), the way the request
  table already is; each widget keeps its own context bar against engine.max_context.
• Left alone on purpose: the standalone API Monitor page (serve/web/monitor.html), the Chat
  and About tabs, and every hardware card.
• Possibly touching the same markup as Monitor: per-GPU view, more cards, and a layout you can rearrange #1114 (rearrangeable Monitor layout) and Monitor: one line per card on a layer split (load, VRAM, temperature, power, PCIe) #2317 (one line
  per card on a layer split); no functional overlap — happy to rebase if either lands first.

[street-tripy]

Found a bug in this card while running it on my own box (RTX 4090, Swift 1.5 IQ3_S, "parallel": 2): the
per-slot number slid. Worker 1 opened at 300 t/s and took a couple of minutes to walk down to the 35 it is
really writing at, so the card was only right once the request was nearly over.

live.slots[].tok_s is generated / (now - first_token) (server.py:1921) — the mean since the first token.
The header pill does not have this problem: it goes through _tok_s() / _rate_of(), the 2 s window
(RATE_WINDOW_S), and _tok_s_mean() is documented as "the whole-request mean since the first token (the old
formula), kept so the two can be compared". The per-slot field was never converted, and the card was faithfully
drawing the old formula: the first tokens over the sliver of a second since the last one.

So the card and the pill now take that window rule from consecutive samples — 2.0 s window, 0.25 s minimum
span, the mean as the fallback for a slot's first sample, all as _rate_of has it — computed once per poll so
the two agree. A slot that goes idle, or takes a new request whose counter starts over, starts its ring over.
live.slots[].tok_s is left as it is: no server, payload or Prometheus change.

Measured against a /metrics payload with the two slots writing at 35 and 30 t/s: the widgets read 35.0 and
30.0 while the published means drift at 35.6 and 30.5, and the pill reads 2 decoding · 35.0 + 30.0 tok/s.
The new test pins that case — 30 tokens in the first 0.1 s, then 35/s, giving 300.0, 35.0, 35.0, 35.0 where the
mean over the same samples is 300.0, 59.1, 47.6, 43.5. python -m unittest serve.test_monitor serve.test_runconfig serve.test_prometheus → 27 tests, all pass; node --check serve/web/app.js clean; no
console errors. Without live.parallel the page is unchanged: card hidden, 0 widgets, 8 tiles, pill
Generating · 65.0 tok/s.

The mean is still what other readers of live.slots[].tok_s get (Prometheus, the API Monitor page). Fixing it
there means a per-slot rate ring in slots_view(), which is a server change — left out of this pull request on
purpose.

[street-tripy]

fb38287 — the card reads as rows, and a request outside a slot gets a row of its own.

Rows, not boxes. Each slot stretches the card, stacked in slot order: name, state, rate, what it has written, context fill, bar. Measured 895 px in a 937 px card, one column, 100 px tall (the boxes were 440 px wide, 157 px tall).

A slot that is reading now shows a rate. It had a dash, on the reasoning that the batch slot reader handles BADM/BDONE/BGEN/BT only (server.py:1683-1830) and so never sees a prompt-progress line. True of its own loop — but the reader delegates to _control() (server.py:1704), and _control parses the engine's PP lines and sets engine.prefill_tok_s_mean (server.py:1283-1288). So the reading row shows that figure, the same one the Speed card's right-hand number reads. It is engine-wide, so it belongs to the slot that is reading, which is the only one: prompts are admitted one at a time. The cached prefix is not read, so it is the rate over the new tail tokens.

prompt_tokens / elapsed_s is not that rate, and I had it in here: prompt_tokens is the whole prompt, known at admission, so the ratio opens at 50,000 t/s and only lands on the truth as the read ends. Removed.

The queue row. Requests in flight that are not in a slot (live.outside_slots, server.py:3775) get a row labelled Solo, first in the card, always present, so a queued request has a place to appear. The aggregate live block is the newest parallel request's status, so its numbers belong to the request outside a slot only when that is the one and only request in flight — the common case, and what the box showed while writing this: both slots idle holding their conversations, one request running on its own. Then the row shows its rate, tokens, seconds and context fill; with a slotted request alongside it shows the count and the queue state and no rate, rather than one request's speed on another's row.

Checked in a browser against a /metrics payload: reading — Solo first "Waiting · 1 request in flight outside a slot · 1 waiting", workers 1,320 t/s "reading 8,128 prompt tokens · engine prefill"; one request outside a slot, both slots idle — Solo "Decoding 69.6 t/s · 1,200 tokens · 17.4 s outside a slot · 9.3k / 256K"; both decoding, queue empty — Solo "Free · nothing outside a slot", workers 35.0 and 30.0 t/s. On a server running with "parallel": 2, two agents decoding at once: Worker 1 28.2 t/s, Worker 2 32.5 t/s, rows 695 px wide in one column. No console errors. python -m unittest serve.test_monitor serve.test_runconfig serve.test_prometheus → 28 tests, all pass; node --check serve/web/app.js clean. Without live.parallel the card stays away, so the default path is untouched.

[Niko1221]

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.