Repository navigation
Ouro LoopCD preemption #98
Description
Tracking the preemption part of #85, based on #86 (d700ecc).
Initial scope: single-worker synchronous eager, default two_head.
- Save live LoopCD references with the KV/hidden-state snapshot; recapture references the resumed stage will replace.
- Restore the correct request, token position and loop depth.
- Handle blocked KV allocation, cancellation, request-ID reuse and cleanup.
- Fix refill scheduling for requests restored into PRELUDE or CODA.
The eager patch passes CPU tests and short official Ouro-1.4B FP32/BF16 checks against uninterrupted guided runs.
Next I'd like to cover multi-request KV pressure and buffer reuse, then Graph suspend/resume once the Ouro Graph baseline is qualified. I'll coordinate that with the Graph work. Graph preemption stays disabled for now; async and P/D are outside this issue.
Could you add this under #85?
Activity
0z5a commentedon Oct 9, 2026
The Task 0 dependency #86 now has a direct-descendant update containing the mixed-depth continuation oracle and scale harness. Its tested
e8aa1eesource leaves production recurrence/readout unchanged; the later publication commit only adds audited results. The originald700eccpin is preserved as the historical preemption base.Full FP32/Triton mixed-depth correctness and complete native C32 cost have passed their original audits. A focused continuation of this issue could reuse those helpers for repeated suspend/resume under multi-request KV pressure, delayed restore and identifier reuse, then compare restoration overhead against uninterrupted guided generation. Graph suspend/resume still needs its own reference/buffer lifetime qualification. The retained BF16 independent-oracle failure, uncertain quality confirmation and unexecuted async/P-D scope remain explicit; this dependency update does not close them.
Update after review cleanup: PR #86 is now at
57abd17, based on current main. Benchmark/recipe/result files have been removed from the PR diff; core guidance and continuation/lifecycle checks remain. Readout projects only real request rows and reuses bounded coda graphs. 297 selected CPU checks passed; new CUDA parity and performance work remains pending. Earlier H20/H100 figures refer to the archived source/results, and the failed full-model BF16 gate remains failed. A possible downstream follow-up is to reconcile #101 with this base and validate preemption plus the new coda path together before claiming combined GPU behavior.