Skip to content

build(deps): bump vLLM to 0.26.0 and make the pin real - #176

Merged
rominf merged 3 commits into
mainfrom
build/vllm-0-26-0
Aug 14, 2026
Merged

rominf merged 3 commits into
mainfrom
build/vllm-0-26-0

Conversation

@rominf

@rominf rominf commented Aug 4, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Two changes to the vLLM install path in engines/vllm:

  • Bump the pinned vLLM release from 0.23.0 to 0.26.0 (published 2026-07-27). 0.26.0 brings support for additional model architectures over 0.23.0.
  • Pass an exact requirement to uv pip install instead of a bare package name, on every path including the index override. This is the more important half of the change.

The install was not actually pinned

The install pushed a bare vllm argument into the uv pip install invocation. The only thing constraining the release was the version segment of the ROCm wheel index URL (https://wheels.vllm.ai/rocm/<version>/<abi>). That is not a pin:

  • if the index ever serves more than one version under that path, the resolver is free to pick any of them;
  • if no wheel under that path matches the interpreter, the resolver falls back to PyPI (--extra-index-url keeps PyPI in play by design, because vLLM's dependencies need it) and installs a generic build.

The second case is not hypothetical — resolving bare vllm against the ROCm index on a CPython 3.13 host (the ROCm wheels are cp312) yields plain vllm==0.26.0 from PyPI rather than the ROCm build.

The install now requests vllm==0.26.0+rocm723 explicitly, so a mismatch fails loudly instead of installing something else.

ABI tag

The ROCm ABI tag stays rocm723. The variant has shifted between releases in the past, so this was the main risk in the bump. Resolving against the live index confirms rocm723 is published for 0.26.0, while neighbouring variants such as rocm724 and rocm730 are not:

$ uv pip compile r724.txt --python-version 3.12 \
    --extra-index-url https://wheels.vllm.ai/rocm/0.26.0/rocm724 \
    --index-strategy unsafe-best-match --no-deps
  x No solution found when resolving dependencies:
  `-> Because there is no version of vllm==0.26.0+rocm724 [...]

The override stays pinned

ROCM_CLI_VLLM_ROCM_INDEX_URL selects a different vLLM/ROCm combination without rebuilding, and a hardcoded vllm==0.26.0+rocm723 cannot resolve against an index serving some other release. The fix is to keep the requirement in lockstep with the index rather than dropping it — an unpinned override is exactly the PyPI-fallback hazard this PR exists to remove.

Version and ABI are treated as the single source of truth from which both the URL and the requirement derive:

  • Not set — the built-in index and vllm==0.26.0+rocm723.
  • Set to a published release index (https://wheels.vllm.ai/rocm/<version>/<abi>) — the version and ABI are parsed back out of the URL and the requirement becomes vllm==<version>+<abi>. Pointing at a different release of the same index therefore works and stays pinned, with no extra configuration.
  • Set to anything else — the build cannot be determined, so the install fails with an error quoting the URL and the expected shape, rather than silently unpinning. A clear failure beats quietly installing the wrong artifact.

Non-goals

  • Deriving the ABI tag from the detected ROCm runtime rather than hardcoding it is a separate change that rewrites the same line; it is deliberately not folded in here.
  • An explicit version/build override variable, for indexes that do not follow the published path shape, is deliberately left out: a repo-wide change is moving every engine version pin into one config file with a generated override variable per dependency, and vLLM will adopt that convention rather than introduce a one-off variable now.
  • Opting into the rolling top-level index (https://wheels.vllm.ai/rocm/, which is latest-only) is out of scope — it is rejected here precisely because it is not a pin.

Test plan

Verified against the live index with uv:

$ uv pip compile req.txt --python-version 3.12 \
    --extra-index-url https://wheels.vllm.ai/rocm/0.26.0/rocm723 \
    --index-strategy unsafe-best-match --no-deps
Resolved 1 package in 12ms
vllm==0.26.0+rocm723

Both the bare vllm requirement and the exact vllm==0.26.0+rocm723 spec resolve to the same wheel, confirming the new index path and ABI tag exist and are self-consistent.

Unit tests cover the resolution rules as pure functions taking the environment value as a parameter (nothing touches process env): nothing set, empty/whitespace values, an override of the published shape (with and without a trailing slash), and overrides that must error rather than unpin — a foreign host, the rolling top-level index, a missing ABI segment, an empty version segment, and a malformed segment.

Local checks: cargo test --workspace --all-targets, cargo clippy --workspace --all-targets -- -D warnings, cargo clippy --locked -p e2e-cucumber --test e2e -- -D warnings, and cargo fmt --all --check all pass. (Two rocm-core proc_lifecycle tests fail on my machine both with and without this change — a local process-group sandboxing artifact, unrelated to this diff.)

Verification gap — please read

This change is weakly covered by automated testing, and honesty about that is more useful than a green checkmark:

  • No test asserts that either literal is correct — the tests compare against the constants, so they stay green regardless of what the constants say. vllm_pinned_spec_matches_extra_index_url does assert the two literals agree with each other (it parses the version and ABI tag back out of the spec and requires the index path to end with them), so a half-finished bump fails in CI; it cannot tell whether the pair is published upstream.
  • Nothing in default CI fetches the index.

A typo in either string therefore stays green all the way through CI and only surfaces on a real install against a GPU runner. The uv pip compile runs above are the actual verification for this PR.

Risk

Low for the pin itself (a bad string now fails the install loudly rather than installing the wrong build). The override path can now fail where it previously proceeded — that is intentional: it only fails for a URL whose build cannot be determined, where the alternative was installing an unpinned, possibly non-ROCm build. Medium for the version bump, in the ordinary sense that a new vLLM release can change runtime behavior — that surface is not exercised by this repo's default CI.

@rominf
rominf force-pushed the build/vllm-0-26-0 branch 2 times, most recently from a1cc1ab to 850f703 Compare August 11, 2026 16:15
rominf added 3 commits August 12, 2026 10:05
The vLLM install passed a bare `vllm` requirement to `uv pip install`, so
the only thing constraining the release was the version segment of the
ROCm wheel index URL. That is not a pin: if the index ever serves more
than one version under that path, or no wheel there matches the
interpreter (in which case the resolver falls back to PyPI), the install
silently drifts.

Pass the exact `vllm==0.26.0+rocm723` requirement instead, and move the
index URL to the matching 0.26.0 path. The ROCm ABI tag stays `rocm723`;
it is the only variant published for this release.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The pin only covered the built-in wheel index. Setting
ROCM_CLI_VLLM_ROCM_INDEX_URL dropped the requirement back to a bare `vllm`,
which is the same hazard the pin exists to remove: the install passes
`--extra-index-url`, so PyPI stays in play by design for vLLM's
dependencies, and a bare requirement can silently resolve to PyPI's
non-ROCm build. On a CPython 3.13 host, where the ROCm wheels are cp312,
bare `vllm` resolves to plain `vllm==0.26.0`.

Keep the requirement in lockstep with the index instead of dropping it.
Published release indexes are `.../rocm/<version>/<abi>`, so an overridden
URL of that shape has its version and ABI parsed back out and the
requirement becomes `vllm==<version>+<abi>`. Pointing at a different release
of the same index — the case the override documents — therefore works and
stays pinned, with no extra configuration.

A URL of any other shape cannot be pinned automatically, and unpinning is
not an acceptable fallback, so it is rejected with an error quoting the URL
and the expected shape. Failing clearly beats quietly installing the wrong
artifact.

Also add a test parsing the version and ABI back out of the pinned spec and
asserting the index path ends with them, so a half-finished bump fails in CI
rather than on a GPU runner, and report the requirement actually passed to
`uv` in the install failure message rather than a hardcoded `vllm`.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
The doc block predates the split into `vllm_install_target` (reads the
environment) and `resolve_vllm_rocm_extra_index_url` (decides from a
supplied value). It stayed on the latter, still telling the reader to
"override with ROCM_CLI_VLLM_ROCM_INDEX_URL" -- a variable that function
no longer looks at.

Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
@rominf
rominf force-pushed the build/vllm-0-26-0 branch from 850f703 to 7bdc689 Compare August 12, 2026 10:19
@rominf
rominf marked this pull request as ready for review August 12, 2026 11:00
@rominf
rominf requested a review from a team as a code owner August 12, 2026 11:00
@juhovainio

Copy link
Copy Markdown
Collaborator

I reviewed this against the live engines/vllm/src/lib.rs code, not just the diff.

The core claim checks out: before this change, install_vllm_with_uv passed a bare vllm string to uv pip install, so only the wheel-index URL constrained the version. Since the install also passes --extra-index-url (leaving PyPI in play for vLLM's dependencies), a bare requirement really could resolve to PyPI's non-ROCm build on an ABI mismatch, exactly as described.

The fix and its edge cases hold up:

  • resolve_vllm_install_target returns the hardcoded VLLM_PINNED_SPEC when the index URL matches the built-in default, and otherwise calls vllm_rocm_build_from_index_url to recover <version>+<abi> from a same-shape override, so the requirement always tracks whichever index is actually queried.
  • vllm_rocm_build_from_index_url's parsing is correct on the cases that matter: the rolling top-level index (.../rocm/), a URL missing the ABI segment, an empty version/ABI segment (double slash), and a segment containing a space are all rejected via the valid()" character check or the split_once/strip_prefixchain returningNone` — I traced each by hand against the code and they match the corresponding tests exactly.
  • Rejection is a hard error (naming the URL and expected shape) rather than falling back to unpinned, which is the right call given --extra-index-url keeps PyPI reachable.
  • I checked for any other caller of the old vllm_rocm_extra_index_url(); it's fully replaced by vllm_install_target() with a single call site in install_vllm_with_uv, and all the touched functions are private to this module, so there's no risk of a stale caller elsewhere.

The PR itself is upfront about what isn't covered: no test or CI step confirms 0.26.0/rocm723 actually exists on the live wheel index (that was verified manually via uv pip compile per the PR body, not automated), and the version bump itself isn't exercised by default CI. That's a reasonable scope boundary for a pin-mechanism fix, not something I'd block on.

CI is green across the board. This looks safe to approve pending final validation of the version bump itself (i.e., that 0.26.0 actually runs cleanly), which is outside what a code review can confirm.

@fredespi

Copy link
Copy Markdown
Collaborator

Reviewed this at 7bdc689 (diff against 076cdff, one file, +185/-12). I don't see anything blocking. Recording what I actually checked, including one thing I was fairly sure was a bug and turned out not to be.

The claim that the old code wasn't pinned holds up. --extra-index-url keeps PyPI in the resolution, and PyPI publishes a plain vllm 0.26.0 with no local version segment, so a bare vllm argument genuinely could resolve to the non-ROCm build. Requesting vllm==0.26.0+rocm723 can't: a local version specifier only matches a distribution carrying that exact local segment, which only the ROCm index serves. The fix does what the description says it does.

The thing I thought was broken, and wasn't. Your test plan resolves with --index-strategy unsafe-best-match, but uv_pip_install_base doesn't pass that flag — the shipped command is uv pip install vllm==0.26.0+rocm723 --extra-index-url ... with uv's default first-index. That looked like a real gap, since first-index exists precisely to stop a resolver crossing indexes. My first attempts to confirm it did fail, with uv even printing the "use --index-strategy unsafe-best-match" hint — but that was my probe's fault, not the code's: I was resolving for macOS-arm64 and then for x86_64-unknown-linux-gnu, which uv treats as a manylinux_2_17 baseline, and the ROCm wheel is manylinux_2_34. Once I resolved for a platform the wheel actually targets, it works with the shipped flags:

$ uv pip compile req.txt --python-version 3.12 --python-platform x86_64-manylinux_2_34 \
    --extra-index-url https://wheels.vllm.ai/rocm/0.26.0/rocm723 --no-deps
Resolved 1 package in 18ms
vllm==0.26.0+rocm723

So no --index-strategy is needed. Worth knowing that the failure mode, if it ever did occur, would be loud rather than silent.

The ABI tag and index path are real. https://wheels.vllm.ai/rocm/0.26.0/rocm723 serves exactly one wheel, vllm-0.26.0+rocm723-cp312-cp312-manylinux_2_34_x86_64.whl, which matches the pin and confirms the rocm723 tag carried over to 0.26.0. The drift guard (vllm_pinned_spec_matches_extra_index_url) means the constant and the spec can't silently diverge later, and vllm_default_index_has_the_published_release_shape means the built-in URL stays parseable by the same code path overrides use. No stale 0.23.0 pin left anywhere — the remaining hits in the tree are the cucumber/darling crates and illustrative strings in e2e fixtures.

One consequence worth being aware of, not a change request. That single wheel is cp312 and manylinux_2_34, so the pin is only satisfiable on Python 3.12 with glibc ≥ 2.34. On a host outside that — RHEL 8 (glibc 2.28) or SLES 15.7 (2.31), both of which rocm install driver supports, or any 3.13 interpreter — the install now fails instead of quietly installing PyPI's generic build. That's the intended trade and I think it's the right one. The rough edge is what the user sees: install_vllm_with_uv surfaces uv's raw resolver output, which in this situation leads with dependency-confusion wording and suggests --index-strategy unsafe-best-match — advice that would not help here and, if followed via UV_INDEX_STRATEGY, would reintroduce exactly the PyPI fallback this PR removes. A short hint on that error path ("no ROCm vLLM wheel for this interpreter/glibc") would save someone a confusing detour, but it's separable from this change.

This is a diff review plus live resolution against the index; I haven't run an actual install on a ROCm host, so it isn't an approval. On that basis it looks safe to approve after whatever final validation you want.

@fredespi

Copy link
Copy Markdown
Collaborator

To make my recommendation explicit rather than leave it implied: I think this is ready to approve, and I found nothing that needs fixing first. The pin is correct, the index path and rocm723 tag are verified against the live index, drift between the constant and the spec is guarded by a test, and the override path either stays pinned or fails loudly instead of quietly unpinning.

Two things I'd want the approver to hold consciously, neither of which is a change request.

This is the only review on the PR. Mine is the entire review history here, so a second pair of eyes is worth more than it would be on something that had already been round-tripped a few times.

The other is a product call rather than a code call, and it's the reason I'm not treating "no findings" as the whole answer. The pin narrows where vLLM can install to Python 3.12 on glibc ≥ 2.34, because that is the only wheel published for this build. On RHEL 8 (glibc 2.28), SLES 15.7 (2.31), or any 3.13 interpreter, rocm will now fail to install vLLM where it previously succeeded by fetching PyPI's non-ROCm build. Failing is the right behaviour and this PR argues that case well — but "those hosts can no longer install vLLM through rocm-cli" is a decision worth making knowingly rather than inheriting from a version bump. If that's already understood and accepted, there's nothing blocking here from my side.

@juhovainio

Copy link
Copy Markdown
Collaborator

Reviewed the full diff against current main and the live branch (head 7bdc689). This is a well-scoped, well-tested change and I didn't find anything to flag.

The core fix is real: a bare vllm requirement combined with --extra-index-url is not a pin, since --extra-index-url keeps PyPI in play by design, and the PR correctly identifies the CPython 3.13 host case where that silently falls through to a non-ROCm PyPI build. Pinning to vllm==0.26.0+rocm723 closes that.

The override-path handling (patch 2/3) is the more interesting part and it's done right: vllm_rocm_build_from_index_url parses <version>/<abi> back out of a published-shape URL so a same-shape override stays pinned automatically, and anything that doesn't fit that shape fails loudly with an error naming the URL and the expected format, rather than silently installing unpinned. I checked the parser against multi-segment and malformed inputs (extra path segments, embedded slashes, spaces) and it rejects all of them correctly, since split_once('/') only accepts exactly two segments after the prefix and the valid() charset check catches the rest.

Confirmed there are no other callers of the removed vllm_rocm_extra_index_url() function or the old bare-vllm install path elsewhere in the tree, so this is fully self-contained to engines/vllm/src/lib.rs. Test coverage matches what the PR description claims: default pin, blank override, same-shape override (with and without trailing slash), and every rejection case, plus a test tying VLLM_PINNED_SPEC to VLLM_ROCM_EXTRA_INDEX_URL so a half-finished version bump fails CI instead of drifting silently. All checks are green.

The PR description is upfront about the real gap (nothing in default CI fetches the index, so a typo in either literal only surfaces on a real install), which matches what a diff-only review can and can't verify.

Looks safe to approve.

@rominf

rominf commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks both — this is the second pair of eyes the change needed, and the independent checks against the live index are worth more than a diff read here.

On --index-strategy (@fredespi): agreed, and thanks for chasing that to ground rather than filing it as a finding. first-index is sufficient, and your x86_64-manylinux_2_34 resolution matches what I saw. The detail that the failure mode would be loud rather than silent is the part worth remembering.

On the error-path hint: you are right, and I have opened #237 for it. uv's raw output leads with dependency-confusion wording and points at --index-strategy unsafe-best-match, which does not help on a host the wheel was never built for and, if followed through UV_INDEX_STRATEGY, reintroduces exactly the PyPI fallback this change removes. I agree it is separable, so it is tracked rather than bolted onto an approved PR.

On the interpreter/glibc narrowing: accepted knowingly, and thank you for insisting it be a decision rather than something inherited from a version bump. The pin restricts vLLM installs to CPython 3.12 on glibc >= 2.34 because that is the only wheel published for this build, so RHEL 8, SLES 15.7, and any 3.13 interpreter now fail where they previously succeeded by fetching a non-ROCm build. That is the trade I want: this project's whole no-fallback stance is that a loud failure beats an install that looks fine and is silently not ROCm. If the published wheel matrix widens, the pin follows it. #237 is what makes that failure legible when it happens.

On the untested version bump (@juhovainio): correct, and deliberately out of scope. Nothing here demonstrates that 0.26.0 runs — only that what gets installed is the ROCm build rather than PyPI's. Validating the release itself is a separate step on real hardware.

@rominf
rominf requested a review from a team August 13, 2026 06:44
@rominf
rominf added this pull request to the merge queue Aug 14, 2026
Merged via the queue into main with commit b9d732d Aug 14, 2026
23 checks passed
@rominf
rominf deleted the build/vllm-0-26-0 branch August 14, 2026 12:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants