Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions docs-site/src/content/docs/guides/web-dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,11 +156,18 @@ fallback until a valid sample is available. Speed uses output tokens per second
full request duration: below 15, 15 to below 50, or at least 50. Unavailable speed values are
excluded when a speed filter is active. Success means 2xx; errors mean 4xx or 5xx.

The request detail also shows **Decode rate (est.)**. When the proxy observed both ends, it is
Logs labels each decode estimate **Generation window** or **After visible output** in the list
and attempt table, and uses **Output rate during generation (est.)** or
**Output rate after first visible output (est.)** in request details. Older cached responses
without a timing method show **Timing unknown** and **Output rate (est.; timing method unknown)**
in details. A visible caption below the detail rate explains the timing method; an unavailable
rate has no caption. When the proxy observed both ends, it is
output tokens over the generation window: from the first output item or block, reasoning included,
to the last output delta. Older rows without that window use the time after the first visible
token instead. Both are proxy-side observations, not the provider's internal token timing, so the
value is always an estimate, and a window under one second shows as unavailable. The end-to-end
token until the end of the request instead. Both are proxy-side observations, not the provider's internal token timing, so the
value is always an estimate, and a window under one second shows as unavailable. Both numerators
include reported reasoning output tokens, so the two timing bases are not directly comparable.
Historical logs are not rewritten and missing generation timestamps are not invented. The end-to-end
tok/s column and speed filter above are not affected.

Active filters show the matching count out of the loaded total. Reset filters restores all
Expand Down
11 changes: 11 additions & 0 deletions docs-site/src/content/docs/reference/management-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,17 @@ the client window after edits, eviction, query changes or restart. Invalid curso
for older servers. This reduces response bytes for stable windows; server projection remains bounded
by the current window size.

Successful `/api/logs` `displayMetrics.decodeTokPerSecond` values carry
`timingBasis: "generation-window" | "legacy-post-visible-output"` alongside `kind`, `value`
and `estimated: true`; attempt values identify their own window. Unavailable results keep their
existing `reason` and no timing basis. The field is derived only at read time: stored JSONL,
end-to-end `tokPerSecond`, request-history DTOs and aggregate throughput are unchanged.
Older DTOs can omit the field; clients must treat the timing basis as unknown in that case.
The dashboard names these methods **Generation window** and **After visible output**, with
**Timing unknown** for a missing or unfamiliar basis. Request details show **Output rate during
generation (est.)**, **Output rate after first visible output (est.)**, or **Output rate (est.;
timing method unknown)** and a visible explanation of the timing method when a rate is available.

| Method and path | Purpose | Notable errors |
| --- | --- | --- |
| `GET /api/logs` | Query filtered in-memory request logs | — |
Expand Down
10 changes: 10 additions & 0 deletions gui/src/i18n/de.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1019,6 +1019,16 @@ export const de: Record<TKey, string> = {
"logs.detail.section.usage": "Roh-Nutzung",
"logs.detail.ttft": "TTFT",
"logs.detail.decodeTokPerSec": "Dekodierrate (geschätzt)",
"logs.decodeBasis.generation": "Generierungsfenster",
"logs.decodeBasis.legacy": "Nach sichtbarer Ausgabe",
"logs.decodeBasis.unknown": "Zeitmessung unbekannt",
"logs.detail.decodeGeneration": "Ausgaberate während der Generierung (geschätzt)",
"logs.detail.decodeLegacy": "Ausgaberate nach der ersten sichtbaren Ausgabe (geschätzt)",
"logs.detail.decodeUnknown": "Ausgaberate (geschätzt; Zeitmessmethode unbekannt)",
"logs.detail.decodeBasisHint.generation": "Geschätzt anhand der Zeit von der ersten vom Proxy beobachteten Ausgabe, einschließlich Reasoning, bis zum letzten Ausgabe-Delta.",
"logs.detail.decodeBasisHint.legacy": "Geschätzt anhand der Zeit von der ersten sichtbaren Ausgabe bis zum Ende der Anfrage; dabei werden alle gemeldeten Ausgabetokens einschließlich Reasoning gezählt.",
"logs.detail.decodeBasisHint.unknown": "Der Server hat nicht angegeben, welche Zeitmessmethode für diese Schätzung verwendet wurde.",
"logs.detail.endToEndTokPerSec": "End-to-End-Ausgabe tok/s",
"logs.detail.reason.ttft_missing": "Für diese Anfrage wurde keine Zeit bis zum ersten Token erfasst, es gibt also kein Dekodierfenster zum Messen.",
"logs.detail.reason.decode_window_too_short": "Das gemessene Ausgabefenster lag unter einer Sekunde und ist zu kurz für eine Schätzung der Dekodierrate.",
"logs.detail.costTotal": "Listenpreis-Äquivalent",
Expand Down
10 changes: 10 additions & 0 deletions gui/src/i18n/en.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1077,6 +1077,16 @@ export const en = {
"logs.detail.section.usage": "Raw usage",
"logs.detail.ttft": "TTFT",
"logs.detail.decodeTokPerSec": "Decode rate (est.)",
"logs.decodeBasis.generation": "Generation window",
"logs.decodeBasis.legacy": "After visible output",
"logs.decodeBasis.unknown": "Timing unknown",
"logs.detail.decodeGeneration": "Output rate during generation (est.)",
"logs.detail.decodeLegacy": "Output rate after first visible output (est.)",
"logs.detail.decodeUnknown": "Output rate (est.; timing method unknown)",
"logs.detail.decodeBasisHint.generation": "Estimated using the time from the first output observed by the proxy, including reasoning, to the last output delta.",
"logs.detail.decodeBasisHint.legacy": "Estimated using the time from the first visible output to the end of the request, while counting all reported output tokens, including reasoning.",
"logs.detail.decodeBasisHint.unknown": "The server did not report which timing method was used for this estimate.",
"logs.detail.endToEndTokPerSec": "End-to-end output tok/s",
"logs.detail.reason.ttft_missing": "No time-to-first-token was recorded for this request, so there is no decode window to measure.",
"logs.detail.reason.decode_window_too_short": "The measured output window was under a second, which is too short to estimate a decode rate from.",
"logs.detail.costTotal": "List-price equivalent",
Expand Down
10 changes: 10 additions & 0 deletions gui/src/i18n/fr.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1058,6 +1058,16 @@ export const fr: Record<TKey, string> = {
"logs.detail.section.usage": "Utilisation brute",
"logs.detail.ttft": "TTFT",
"logs.detail.decodeTokPerSec": "Débit de décodage (est.)",
"logs.decodeBasis.generation": "Fenêtre de génération",
"logs.decodeBasis.legacy": "Après la sortie visible",
"logs.decodeBasis.unknown": "Mesure du temps inconnue",
"logs.detail.decodeGeneration": "Débit de sortie pendant la génération (est.)",
"logs.detail.decodeLegacy": "Débit de sortie après la première sortie visible (est.)",
"logs.detail.decodeUnknown": "Débit de sortie (est. ; méthode de mesure du temps inconnue)",
"logs.detail.decodeBasisHint.generation": "Estimation fondée sur le temps entre la première sortie observée par le proxy, raisonnement compris, et le dernier delta de sortie.",
"logs.detail.decodeBasisHint.legacy": "Estimation fondée sur le temps entre la première sortie visible et la fin de la requête, en comptant tous les jetons de sortie signalés, raisonnement compris.",
"logs.detail.decodeBasisHint.unknown": "Le serveur n’a pas indiqué quelle méthode de mesure du temps a été utilisée pour cette estimation.",
"logs.detail.endToEndTokPerSec": "Débit de sortie de bout en bout (jetons/s)",
"logs.detail.reason.ttft_missing": "Aucun délai jusqu’au premier token n’a été enregistré pour cette requête, il n’y a donc pas de fenêtre de décodage à mesurer.",
"logs.detail.reason.decode_window_too_short": "La fenêtre de sortie mesurée durait moins d’une seconde, ce qui est trop court pour estimer un débit de décodage.",
"logs.detail.costTotal": "Équivalent au tarif catalogue",
Expand Down
10 changes: 10 additions & 0 deletions gui/src/i18n/ja.ts
Original file line number Diff line number Diff line change
Expand Up @@ -960,6 +960,16 @@ export const ja: Record<TKey, string> = {
"logs.detail.section.usage": "生の使用量",
"logs.detail.ttft": "TTFT",
"logs.detail.decodeTokPerSec": "デコード速度(推定)",
"logs.decodeBasis.generation": "生成区間",
"logs.decodeBasis.legacy": "表示された出力の後",
"logs.decodeBasis.unknown": "計測方法不明",
"logs.detail.decodeGeneration": "生成中の出力速度(推定)",
"logs.detail.decodeLegacy": "最初に表示された出力以降の出力速度(推定)",
"logs.detail.decodeUnknown": "出力速度(推定・時間の計測方法不明)",
"logs.detail.decodeBasisHint.generation": "推論を含め、プロキシが最初の出力を観測してから最後の出力差分を観測するまでの時間で推定しています。",
"logs.detail.decodeBasisHint.legacy": "最初に出力が表示されてからリクエストが終了するまでの時間で、推論を含む報告されたすべての出力トークンを数えて推定しています。",
"logs.detail.decodeBasisHint.unknown": "この推定に使われた時間の計測方法は、サーバーから報告されていません。",
"logs.detail.endToEndTokPerSec": "リクエスト全体の出力 tok/s",
"logs.detail.reason.ttft_missing": "このリクエストでは最初のトークンまでの時間が記録されていないため、測定できるデコード区間がありません。",
"logs.detail.reason.decode_window_too_short": "計測された出力区間が1秒未満で、デコード速度を推定するには短すぎます。",
"logs.detail.costTotal": "定価相当額",
Expand Down
10 changes: 10 additions & 0 deletions gui/src/i18n/ko.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1058,6 +1058,16 @@ export const ko: Record<TKey, string> = {
"logs.detail.section.usage": "원본 usage",
"logs.detail.ttft": "TTFT",
"logs.detail.decodeTokPerSec": "디코드 속도(추정)",
"logs.decodeBasis.generation": "생성 구간",
"logs.decodeBasis.legacy": "화면에 출력된 뒤",
"logs.decodeBasis.unknown": "측정 방식 미상",
"logs.detail.decodeGeneration": "생성 중 출력 속도(추정)",
"logs.detail.decodeLegacy": "화면에 첫 출력이 나온 뒤의 출력 속도(추정)",
"logs.detail.decodeUnknown": "출력 속도(추정; 시간 측정 방식 미상)",
"logs.detail.decodeBasisHint.generation": "추론을 포함해 프록시가 첫 출력을 관측한 시점부터 마지막 출력 조각을 관측한 시점까지 걸린 시간으로 추정해요.",
"logs.detail.decodeBasisHint.legacy": "화면에 첫 출력이 나온 시점부터 요청이 끝날 때까지 걸린 시간으로 추정해요. 추론 토큰을 포함해 보고된 모든 출력 토큰을 계산에 넣어요.",
"logs.detail.decodeBasisHint.unknown": "서버가 이 추정값에 어떤 시간 측정 방식을 썼는지 알려주지 않았어요.",
"logs.detail.endToEndTokPerSec": "전체 요청 출력 tok/s",
"logs.detail.reason.ttft_missing": "이 요청은 첫 토큰까지 걸린 시간이 기록되지 않아서 측정할 디코드 구간이 없어요.",
"logs.detail.reason.decode_window_too_short": "측정된 출력 구간이 1초도 안 돼서 디코드 속도를 추정하기엔 너무 짧아요.",
"logs.detail.costTotal": "정가 환산치",
Expand Down
10 changes: 10 additions & 0 deletions gui/src/i18n/pt.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1072,6 +1072,16 @@ export const pt: Record<TKey, string> = {
"logs.detail.section.usage": "Uso bruto",
"logs.detail.ttft": "TTFT",
"logs.detail.decodeTokPerSec": "Taxa de decodificação (est.)",
"logs.decodeBasis.generation": "Janela de geração",
"logs.decodeBasis.legacy": "Após a saída visível",
"logs.decodeBasis.unknown": "Medição de tempo desconhecida",
"logs.detail.decodeGeneration": "Taxa de saída durante a geração (est.)",
"logs.detail.decodeLegacy": "Taxa de saída após a primeira saída visível (est.)",
"logs.detail.decodeUnknown": "Taxa de saída (est.; método de medição de tempo desconhecido)",
"logs.detail.decodeBasisHint.generation": "Estimada usando o tempo entre a primeira saída observada pelo proxy, incluindo raciocínio, e o último delta de saída.",
"logs.detail.decodeBasisHint.legacy": "Estimada usando o tempo entre a primeira saída visível e o fim da requisição, contando todos os tokens de saída informados, incluindo raciocínio.",
"logs.detail.decodeBasisHint.unknown": "O servidor não informou qual método de medição de tempo foi usado nesta estimativa.",
"logs.detail.endToEndTokPerSec": "Saída de ponta a ponta (tok/s)",
"logs.detail.reason.ttft_missing": "Nenhum tempo até o primeiro token foi registrado para esta requisição, portanto não há janela de decodificação para medir.",
"logs.detail.reason.decode_window_too_short": "A janela de saída medida foi inferior a um segundo, curta demais para estimar a taxa de decodificação.",
"logs.detail.costTotal": "Equivalente ao preço de tabela",
Expand Down
10 changes: 10 additions & 0 deletions gui/src/i18n/ru.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1036,6 +1036,16 @@ export const ru: Record<TKey, string> = {
"logs.detail.section.usage": "Сырые данные использования",
"logs.detail.ttft": "TTFT",
"logs.detail.decodeTokPerSec": "Скорость декодирования (оценка)",
"logs.decodeBasis.generation": "Окно генерации",
"logs.decodeBasis.legacy": "После видимого вывода",
"logs.decodeBasis.unknown": "Метод замера времени неизвестен",
"logs.detail.decodeGeneration": "Скорость вывода во время генерации (оценка)",
"logs.detail.decodeLegacy": "Скорость вывода после первого видимого вывода (оценка)",
"logs.detail.decodeUnknown": "Скорость вывода (оценка; метод замера времени неизвестен)",
"logs.detail.decodeBasisHint.generation": "Оценка по времени от первого вывода, наблюдаемого прокси, включая рассуждения, до последнего приращения вывода.",
"logs.detail.decodeBasisHint.legacy": "Оценка по времени от первого видимого вывода до конца запроса с подсчётом всех сообщённых токенов вывода, включая рассуждения.",
"logs.detail.decodeBasisHint.unknown": "Сервер не сообщил, какой метод замера времени использован для этой оценки.",
"logs.detail.endToEndTokPerSec": "Сквозной вывод (токенов/с)",
"logs.detail.reason.ttft_missing": "Для этого запроса не записано время до первого токена, поэтому измерять нечего.",
"logs.detail.reason.decode_window_too_short": "Измеренное окно вывода длилось меньше секунды — слишком мало для оценки скорости декодирования.",
"logs.detail.costTotal": "Эквивалент по прайс-листу",
Expand Down
10 changes: 10 additions & 0 deletions gui/src/i18n/tr.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1063,6 +1063,16 @@ export const tr: Record<TKey, string> = {
"logs.detail.section.usage": "Ham kullanım",
"logs.detail.ttft": "TTFT",
"logs.detail.decodeTokPerSec": "Çözme hızı (tahmini)",
"logs.decodeBasis.generation": "Üretim aralığı",
"logs.decodeBasis.legacy": "Görünür çıktı sonrası",
"logs.decodeBasis.unknown": "Zaman ölçümü bilinmiyor",
"logs.detail.decodeGeneration": "Üretim sırasındaki çıktı hızı (tahmini)",
"logs.detail.decodeLegacy": "İlk görünür çıktıdan sonraki çıktı hızı (tahmini)",
"logs.detail.decodeUnknown": "Çıktı hızı (tahmini; zaman ölçüm yöntemi bilinmiyor)",
"logs.detail.decodeBasisHint.generation": "Akıl yürütme dahil, proxy’nin gözlemlediği ilk çıktıdan son çıktı parçasına kadar geçen süre kullanılarak tahmin edilir.",
"logs.detail.decodeBasisHint.legacy": "İlk görünür çıktıdan isteğin sonuna kadar geçen süre kullanılarak tahmin edilir; akıl yürütme dahil bildirilen tüm çıktı tokenları sayılır.",
"logs.detail.decodeBasisHint.unknown": "Sunucu bu tahmin için hangi zaman ölçüm yönteminin kullanıldığını bildirmedi.",
"logs.detail.endToEndTokPerSec": "Uçtan uca çıktı jeton/sn",
"logs.detail.reason.ttft_missing": "Bu istek için ilk token süresi kaydedilmediğinden ölçülecek bir çözme aralığı yok.",
"logs.detail.reason.decode_window_too_short": "Ölçülen çıktı aralığı bir saniyenin altındaydı; çözme hızını tahmin etmek için fazla kısa.",
"logs.detail.costTotal": "Liste fiyatı eşdeğeri",
Expand Down
10 changes: 10 additions & 0 deletions gui/src/i18n/vi.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1057,6 +1057,16 @@ export const vi: Record<TKey, string> = {
"logs.detail.section.usage": "Mức sử dụng thô (Raw usage)",
"logs.detail.ttft": "TTFT",
"logs.detail.decodeTokPerSec": "Tốc độ decode (ước tính)",
"logs.decodeBasis.generation": "Khoảng tạo đầu ra",
"logs.decodeBasis.legacy": "Sau đầu ra hiển thị",
"logs.decodeBasis.unknown": "Chưa rõ cách đo thời gian",
"logs.detail.decodeGeneration": "Tốc độ đầu ra trong khi tạo (ước tính)",
"logs.detail.decodeLegacy": "Tốc độ đầu ra sau lần hiển thị đầu tiên (ước tính)",
"logs.detail.decodeUnknown": "Tốc độ đầu ra (ước tính; chưa rõ cách đo thời gian)",
"logs.detail.decodeBasisHint.generation": "Ước tính theo thời gian từ đầu ra đầu tiên mà proxy quan sát được, bao gồm suy luận, đến phần đầu ra bổ sung cuối cùng.",
"logs.detail.decodeBasisHint.legacy": "Ước tính theo thời gian từ đầu ra hiển thị đầu tiên đến khi yêu cầu kết thúc, đồng thời đếm tất cả token đầu ra được báo cáo, bao gồm suy luận.",
"logs.detail.decodeBasisHint.unknown": "Máy chủ không báo cáo cách đo thời gian được dùng cho ước tính này.",
"logs.detail.endToEndTokPerSec": "Đầu ra toàn yêu cầu (tok/s)",
"logs.detail.reason.ttft_missing": "Không ghi nhận thời gian đến token đầu tiên (time-to-first-token) cho request này, nên không có cửa sổ decode nào để đo lường.",
"logs.detail.reason.decode_window_too_short": "Cửa sổ đầu ra đo được nhỏ hơn một giây, quá ngắn để ước tính tốc độ decode.",
"logs.detail.costTotal": "Tương đương giá niêm yết",
Expand Down
10 changes: 10 additions & 0 deletions gui/src/i18n/zh-TW.ts
Original file line number Diff line number Diff line change
Expand Up @@ -837,6 +837,16 @@ export const zhTW: Record<TKey, string> = {
"logs.detail.section.usage": "原始 usage",
"logs.detail.ttft": "TTFT",
"logs.detail.decodeTokPerSec": "解碼速率(估算)",
"logs.decodeBasis.generation": "生成區間",
"logs.decodeBasis.legacy": "可見輸出之後",
"logs.decodeBasis.unknown": "計時方式未知",
"logs.detail.decodeGeneration": "生成期間的輸出速率(估算)",
"logs.detail.decodeLegacy": "首次可見輸出後的輸出速率(估算)",
"logs.detail.decodeUnknown": "輸出速率(估算;計時方式未知)",
"logs.detail.decodeBasisHint.generation": "依代理觀測到首次輸出(包括推理)至最後一個輸出增量之間的時間估算。",
"logs.detail.decodeBasisHint.legacy": "依首次可見輸出至請求結束之間的時間估算,同時計入所有已回報的輸出 token,包括推理 token。",
"logs.detail.decodeBasisHint.unknown": "伺服器未回報此估算使用的計時方式。",
"logs.detail.endToEndTokPerSec": "端到端輸出 tok/s",
"logs.detail.reason.ttft_missing": "此請求沒有記錄第一個 token 的時間,因此沒有可測量的解碼區間。",
"logs.detail.reason.decode_window_too_short": "測得的輸出區間不到一秒,太短,無法估算解碼速率。",
"logs.detail.costTotal": "標價折算",
Expand Down
Loading
Loading