Skip to content

[Doc][M9] PD and KV transfer walkthrough - #64

Closed
YouHengfei wants to merge 6 commits into
InferMatrixLab:mainfrom
YouHengfei:docs/m9-pd-refactoring
Closed

YouHengfei wants to merge 6 commits into
InferMatrixLab:mainfrom
YouHengfei:docs/m9-pd-refactoring

Conversation

@YouHengfei

@YouHengfei YouHengfei commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Required human contributor answers

1. What outcome are you trying to achieve?

Re-pin the M9 walkthrough to current main so it serves as accurate evidence for the RFC #111 discussion. (User: "主要是固定新的walkthrough"; maintainer review below recommends exactly this.)

2. What is the highest-leverage variable for that outcome?

Accuracy of the evidence document against the reviewed source: reviewers and RFC participants must reason about the current StageEngine/StageWorker/PDStageConnector architecture, not stale line-level detail.

3. How will this work move that variable rather than create busywork?

Rewrites the walkthrough against the pinned baseline 21f59c9 and deletes all expired worker/ownership detail, per the maintainer self-review on this PR: "keep the draft, re-pin to current main, and rewrite the worker/ownership section against StageEngine/StageWorker ... it is the line-level detail that expired, not the document's purpose."

4. Who is the customer or user, and what do they actually need?

Maintainers reviewing this PR and participants in RFC #111 / module M9 (#32): they need a current-state description of ownership, the handoff state machine, and remaining review/validation work to make contract decisions.

PR necessity and bugfix evidence

Yes — this PR is the agreed evidence document for RFC #111 (see prior discussion moving the proposal to #111). Documentation update, not a bugfix; no runtime failure is claimed. One code-level discrepancy found during the re-pin (coordinator rebuilds all worker configs as OuroConfig, rejecting Nanbeige's num_loops; reproduced at the CPU metadata boundary) is documented as remaining work, not fixed here.

Purpose

Refs #32 and #111. Evidence document for the M9 RFC; interface decisions belong to #111.

Re-pins docs/pd_walkthrough.md to main 21f59c9 (post unified-stage refactor #118). The document now describes only the current tree:

  • Ownership map across PDEngine, StageClient/StageWorker, StageEngine, PDStageConnector/RequestTransfer, NixlConnector, Scheduler and KVCacheManager.
  • Shared transfer operations (reserve_transfer, can_schedule, prefill_submitted, accept_import, release_transfer_compute, release_transfer, suspend_transfer) and their contracts.
  • Coordinator reservation credits (reserved_blocks/slots) versus physical allocator usage; message scopes including generic control/reply profiling.
  • Handoff sequence with the received → ack ordering (independent of compute admission) and the eight distinct completion milestones.
  • Cancellation, fail-closed transport error handling, sender-before-receiver termination, and lease/preemption interaction.
  • Remaining work: model-neutral coordinator metadata (Nanbeige mismatch), acknowledgment wording, borrowing-boundary review, and the open [REFACT][M9][1/N]: Type PD control-plane protocol and transfer phases #131/[REFACT][M9][2/N] Clarify PD coordinator reservation credits #132 decisions.
  • Test inventory and remaining validation, including the scope and limits of draft Add a real-checkpoint P->D agreement test for the RFC #111 handoff contract #119.

Test Plan

vllm-rlt Version / Commit: 22951a5 on docs/m9-pd-refactoring, merging main 21f59c9.

Documentation only; no runtime change.

  • git diff upstream/main --check: clean.
  • Relative link targets in the walkthrough resolve.
  • Targeted CPU suites against the merged tree: pytest -q tests/test_transfer_connector.py tests/test_stage_engine.py tests/test_pd.py tests/test_profiling.py → 89 passed, 34 skipped (GPU cases skipped, CUDA hidden).
  • Nanbeige metadata mismatch reproduced at the CPU config boundary (OuroConfig(**asdict(NanbeigeConfig())) → TypeError: num_loops); documented in §6.

Test Result

As above. No GPU, real-checkpoint or performance runs; no runtime behavior is claimed or changed.

@bjf-frz

bjf-frz commented Oct 9, 2026

Copy link
Copy Markdown
Collaborator

I suggest moving this proposal to a dedicated M9 RFC linked to #32, so we can agree on ownership and cross-module contracts before implementation.

@YouHengfei

Copy link
Copy Markdown
Contributor Author

I suggest moving this proposal to a dedicated M9 RFC linked to #32, so we can agree on ownership and cross-module contracts before implementation.

Thanks ! I’ll move the proposal into a dedicated M9 RFC linked to #32 and this PR, focusing on state ownership, handoff contracts, and dependencies on M3/M4/M5 and M1 before implementation.

@YouHengfei
YouHengfei force-pushed the docs/m9-pd-refactoring branch from b56f588 to 07d196e Compare October 16, 2026 10:47
@YouHengfei YouHengfei changed the title [REFACT][M9] Propose M9 PD and KV transfer refactoring draft [Doc][M9] PD and KV transfer walkthrough Oct 16, 2026
@YouHengfei

Copy link
Copy Markdown
Contributor Author

The design proposal has moved to the dedicated M9 RFC: #111. This PR now retains the baseline walkthrough (docs/pd_walkthrough.md, pinned at 9d601ce) as the evidence document; contract and ownership decisions are discussed in the RFC issue.

@hsliuustc0106

Copy link
Copy Markdown
Collaborator

Maintainer self-review: this walkthrough is materially stale against current main, so it should stay a draft until re-pinned.

Concrete drift since the re-pin at 9d601ce:

  • c3ce155 ("complete unified stage execution") removed class PDWorker. Current main has StageEngine (vllm_rlt/engine/stage_engine.py:24) and StageWorker (vllm_rlt/engine/worker.py:22); vllm_rlt/pd/engine.py now contains only Peer, Pending, PDEngine.
  • The document references PDWorker 16 times and covers line-by-line call paths, so most of the worker section no longer matches the tree.
  • main has also moved to 21f59c9; 73ea302 changed serving/speculative graph behaviour after this pin.

Recommendation: keep the draft, re-pin to current main, and rewrite the worker/ownership section against StageEngine/StageWorker before requesting review. The call-chain and ownership framing is useful — it is the line-level detail that expired, not the document's purpose.

Signed-off-by: youhengfei <474029121@qq.com>
Re-pin the M9 evidence document to 21f59c9 and describe only the current
architecture: StageClient/StageWorker/StageEngine with PDStageConnector,
the received -> ack handoff, reservation credits, cancellation/lease
ordering, and remaining review and validation work.

Signed-off-by: youhengfei <474029121@qq.com>
@YouHengfei

Copy link
Copy Markdown
Contributor Author

Re-pinned to current main (21f59c9) at 22951a5. The walkthrough now describes only the current tree: StageClient/StageWorker/StageEngine with PDStageConnector, the received → ack handoff, reservation credits, cancellation/lease ordering, and remaining review/validation work. All expired worker-level detail and old-baseline comparisons are removed. One discrepancy found during the re-pin (coordinator rebuilds worker configs as OuroConfig, rejecting Nanbeige's num_loops) is documented in §6 as remaining work, not fixed here. Validation: docs-only; git diff --check clean; targeted CPU suites 89 passed / 34 skipped on the merged tree.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants