Skip to content

[REFACT][M9][2/N] Clarify PD coordinator reservation credits - #132

Closed
YouHengfei wants to merge 3 commits into
InferMatrixLab:mainfrom
YouHengfei:refactor/m9-pd-credits
Closed

YouHengfei wants to merge 3 commits into
InferMatrixLab:mainfrom
YouHengfei:refactor/m9-pd-credits

Conversation

@YouHengfei

Copy link
Copy Markdown
Contributor

Required human contributor answers

1. What outcome are you trying to achieve?

Per RFC #111 (Proposal, resource ownership): "Coordinator reservation credits must be named and exposed separately from physical allocator usage." This PR implements the naming part of that agreed requirement inside the M9 Coordinator.

2. What is the highest-leverage variable for that outcome?

Per RFC #111 decision item 6: "Confirm how coordinator reservation credits and physical allocator usage should be named and exposed." Making the reservation-vs-physical distinction explicit in the coordinator's own state names is the variable this PR moves.

3. How will this work move that variable rather than create busywork?

Today Peer.blocks/slots/compute_slots and the facade shim cache_manager.num_used_blocks present conservative admission budgets as if they were physical allocator usage. Renaming the counters to reserved_*, naming the aggregate, and documenting the distinct release points removes that ambiguity where the confusion actually lives, without touching admission math, the wire protocol, or cancellation/release ordering.

4. Who is the customer or user, and what do they actually need?

M9 implementers and reviewers, and the M1/M2 owners who will later replace the frontend shim with a real resource-statistics interface (RFC #111 ownership table). They need coordinator state whose names state what the numbers mean, so admission reservations are never again read as physical pressure.

PR necessity and bugfix evidence

A code PR is needed: this is a source rename plus characterization coverage that cannot be delivered as documentation or configuration. It is an M9-internal, behavior-preserving step consistent with RFC #111 migration step 2 (internal extraction before dependent module contracts land). Not a bugfix.

Purpose

Refs #111. Depends on #131 (its commits appear in the diff until #131 merges).

Make coordinator-side reservation accounting explicit without changing PD behavior. These counters represent admission budgets, not physical KV allocator usage.

Changes:

  • Rename Peer.blocks/slots/compute_slots to reserved_*.
  • Rename Pending.p_blocks/d_blocks to reserved_p_blocks/reserved_d_blocks.
  • Name the reservation aggregate (_reserved_blocks_total) and document the distinct release points: compute capacity returns at prefill completion; blocks and request slots remain reserved until worker release.
  • Retain the frontend cache_manager.num_used_blocks compatibility shim, including its existing refresh timing.

No changes to admission policy, budget calculations, wire messages, transfer execution, or cancellation/release ordering. Frontend interface replacement remains outside this PR.

Test Plan

vllm-rlt Version / Commit: branch refactor/m9-pd-credits, stacked on #131 (c6801d0).

  • Preserve existing protocol tests, updating renamed field references.
  • Add 13 parameterized cases covering block rounding, KV-plane counts, decode budgets, P/D slot and block limits, and watermark behavior.
  • Strengthen lifecycle checks for duplicate completion, early-finish reservations, and cancellation cleanup.
  • Run PD/profiling tests, the full CPU suite, and changed-file Ruff checks.
  • Compare the production AST against the baseline after reversing the approved renames and inlining the aggregate helper.

Test Result

  • PD/profiling: 51 passed, 15 skipped (GPU).
  • Full CPU suite: 640 passed, 159 skipped, 1 failed; baseline on the same environment: 627 passed, 159 skipped, 1 failed.
  • The single failure is identical before and after: test_run_refuses_the_stored_baseline_at_another_depth — lm_eval==0.4.9.2 references AutoModelForVision2Seq, unavailable in installed transformers==5.18.0. No unrelated dependency fix is included.
  • Ruff lint/format, whitespace checks, and the normalized AST comparison passed.
  • GPU validation for this diff is pending a two-device reservation; earlier [REFACT][M9][1/N]: Type PD control-plane protocol and transfer phases #131 GPU results are not counted as validation of A4.

Kept as draft pending author review and remaining validation.

Rename Peer.blocks/slots/compute_slots and Pending.p_blocks/d_blocks to
reserved_* names so coordinator admission budgets are no longer confused
with physical allocator usage. Name the reservation aggregate and document
the distinct release points; keep the frontend num_used_blocks shim and its
refresh timing unchanged.

Refs InferMatrixLab#111 decision item 6. Behavior-preserving: admission math, wire
messages, transfer execution, and cancel/release ordering are untouched,
with 13 new characterization cases pinning reservation accounting.
@YouHengfei YouHengfei changed the title [Core][M9] Clarify PD coordinator reservation credits [REFECTOR][M9][2/N] Clarify PD coordinator reservation credits Oct 18, 2026
@YouHengfei YouHengfei changed the title [REFECTOR][M9][2/N] Clarify PD coordinator reservation credits [REFECT][M9][2/N] Clarify PD coordinator reservation credits Oct 18, 2026
@YouHengfei YouHengfei changed the title [REFECT][M9][2/N] Clarify PD coordinator reservation credits [REFACT][M9][2/N] Clarify PD coordinator reservation credits Oct 19, 2026
@YouHengfei YouHengfei closed this Oct 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant