Repository navigation
Description
Upstream source
- QuantumNous/new-api PR: feat: record perf metrics when async tasks reach terminal status QuantumNous/new-api#7481
- Upstream head:
a886cfb7230e393321e6d1677d6eb5de298f4654 - Relevant upstream files:
pkg/perf_metrics/metrics.gopkg/perf_metrics/metrics_test.goservice/task_polling.go
Upstream adds performance samples when asynchronous video/music tasks reach SUCCESS or FAILURE, so task-based models contribute success-rate/latency data instead of remaining permanently without performance data.
Current LMM gap
LMM already has apps/api-go/pkg/perf_metrics and records synchronous relay samples through RecordRelaySample, but the async task polling paths do not currently emit performance samples.
The upstream patch is conceptually applicable, but should not be copied verbatim because LMM's task lifecycle has diverged around durable refunds and custom settlement. In particular:
apps/api-go/service/task_polling.gohas LMM-specific terminal paths such assweepTimedOutTasksandfailTaskForPolling.- LMM's
settleTaskBillingOnCompleteand refund flow are different from upstream and must remain unchanged. - terminal writes are protected by
UpdateWithStatusCAS; metrics must only be emitted by the CAS winner to avoid duplicate samples under overlapping pollers / multiple instances.
Relevant current files:
apps/api-go/pkg/perf_metrics/metrics.goapps/api-go/model/task.goapps/api-go/service/task_polling.go
model.Task already contains Group, SubmitTime, StartTime, FinishTime, Properties.OriginModelName and PrivateData.BillingContext.OriginModelName, so no schema migration is needed.
Suggested implementation
- Add a small
RecordTaskResult(task, taskResult)helper toapps/api-go/pkg/perf_metrics.- model name: prefer
BillingContext.OriginModelNamewhen present, otherwiseProperties.OriginModelName; - group:
task.Group; - success: terminal status is
SUCCESS; - latency:
SubmitTime -> FinishTime; - optional throughput: only for successful tasks when the provider reports
TotalTokensorCompletionTokens; - no user/task IDs or response payloads in metric keys.
- model name: prefer
- Emit the sample only after a successful terminal CAS transition. Do not emit on non-terminal updates or CAS loss.
- Cover LMM-specific terminal paths instead of introducing upstream's billing finalizer wholesale:
- normal video/task terminal transition;
- Suno terminal transition;
failTaskForPolling/ missing upstream task id;- channel lookup failure that terminally fails a task;
- timeout sweep (including an explicit decision whether legacy timeout rows should be sampled; if sampled, they must remain failure-only and must not alter legacy refund behavior).
- Keep all existing settlement/refund calls and their ordering/semantics intact. Metrics must be observational only.
Acceptance criteria
- An async task contributes exactly one performance sample when it wins the durable transition to
SUCCESSorFAILURE. - A competing poller that loses
UpdateWithStatusdoes not emit another sample. - Non-terminal
SUBMITTED/QUEUED/IN_PROGRESSupdates emit no sample. - Successful token-reporting tasks populate throughput; tasks without token usage still populate success rate and latency.
- Model-name fallback works for both current billing-context tasks and older tasks that only have
Properties.OriginModelName. - Timeout / missing-task-id / channel lookup failure paths are covered deliberately rather than silently omitted.
- Existing quota, subscription settlement, durable refund reconciliation, pricing, group routing, model price lock and
/fastbehavior are unchanged. - Focused tests cover success, failure, token/no-token, CAS-loser/no-duplicate, and at least one LMM-specific failure/timeout path.
-
go test ./pkg/perf_metrics ./service -count=1(or the LMM-equivalent focused packages) passes; rungo veton touched packages.
Why issue first instead of a direct cherry-pick
The useful part is small, but the upstream patch also refactors terminal settlement into a shared finalizer. LMM's billing/refund lifecycle is fork-specific, so copying that refactor would create unnecessary financial-risk surface. This should be implemented as a metrics-only adaptation on top of the current LMM terminal/CAS paths.
Activity
LIghtJUNction commentedon Sep 20, 2026
Upstream #7481 has now been merged to
QuantumNous/new-apimain as QuantumNous/new-api@3abbb81 (2026-09-20). Final upstream scope remains the same three files (pkg/perf_metrics/metrics.go,pkg/perf_metrics/metrics_test.go,service/task_polling.go) and still centralizes terminal sampling inside a billing finalizer. Keep this issue's LMM-specific adaptation plan unchanged: emit metrics only after the durable terminal CAS winner, but do not import upstream's settlement/refund refactor over LMM's custom billing lifecycle.feat(perf): sample async task terminal results after durable CAS (#583)