Repository navigation
[REFACT][M9][2/N] Clarify PD coordinator reservation credits - #132
Closed
YouHengfei wants to merge 3 commits into
Closed
YouHengfei wants to merge 3 commits into
YouHengfei wants to merge 3 commits into
Conversation
Rename Peer.blocks/slots/compute_slots and Pending.p_blocks/d_blocks to reserved_* names so coordinator admission budgets are no longer confused with physical allocator usage. Name the reservation aggregate and document the distinct release points; keep the frontend num_used_blocks shim and its refresh timing unchanged. Refs InferMatrixLab#111 decision item 6. Behavior-preserving: admission math, wire messages, transfer execution, and cancel/release ordering are untouched, with 13 new characterization cases pinning reservation accounting.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Required human contributor answers
1. What outcome are you trying to achieve?
Per RFC #111 (Proposal, resource ownership): "Coordinator reservation credits must be named and exposed separately from physical allocator usage." This PR implements the naming part of that agreed requirement inside the M9 Coordinator.
2. What is the highest-leverage variable for that outcome?
Per RFC #111 decision item 6: "Confirm how coordinator reservation credits and physical allocator usage should be named and exposed." Making the reservation-vs-physical distinction explicit in the coordinator's own state names is the variable this PR moves.
3. How will this work move that variable rather than create busywork?
Today
Peer.blocks/slots/compute_slotsand the facade shimcache_manager.num_used_blockspresent conservative admission budgets as if they were physical allocator usage. Renaming the counters toreserved_*, naming the aggregate, and documenting the distinct release points removes that ambiguity where the confusion actually lives, without touching admission math, the wire protocol, or cancellation/release ordering.4. Who is the customer or user, and what do they actually need?
M9 implementers and reviewers, and the M1/M2 owners who will later replace the frontend shim with a real resource-statistics interface (RFC #111 ownership table). They need coordinator state whose names state what the numbers mean, so admission reservations are never again read as physical pressure.
PR necessity and bugfix evidence
A code PR is needed: this is a source rename plus characterization coverage that cannot be delivered as documentation or configuration. It is an M9-internal, behavior-preserving step consistent with RFC #111 migration step 2 (internal extraction before dependent module contracts land). Not a bugfix.
Purpose
Refs #111. Depends on #131 (its commits appear in the diff until #131 merges).
Make coordinator-side reservation accounting explicit without changing PD behavior. These counters represent admission budgets, not physical KV allocator usage.
Changes:
Peer.blocks/slots/compute_slotstoreserved_*.Pending.p_blocks/d_blockstoreserved_p_blocks/reserved_d_blocks._reserved_blocks_total) and document the distinct release points: compute capacity returns at prefill completion; blocks and request slots remain reserved until worker release.cache_manager.num_used_blockscompatibility shim, including its existing refresh timing.No changes to admission policy, budget calculations, wire messages, transfer execution, or cancellation/release ordering. Frontend interface replacement remains outside this PR.
Test Plan
vllm-rlt Version / Commit: branch
refactor/m9-pd-credits, stacked on #131 (c6801d0).Test Result
test_run_refuses_the_stored_baseline_at_another_depth—lm_eval==0.4.9.2referencesAutoModelForVision2Seq, unavailable in installedtransformers==5.18.0. No unrelated dependency fix is included.Kept as draft pending author review and remaining validation.