Skip to content

vLLM fails at startup on ROCm 10.1.0: tensorizer KeyError under Python 3.14 (cp314 is the only published build) #1083

Open

Description

@juhovainio

Current behavior

rocm serve qwen --engine vllm on ROCm 10.1.0 installs vLLM and then fails at startup: importing vLLM's tensorizer dependency raises a KeyError under Python 3.14, so the engine core never starts.

tensorizer/_crypt/_encryption.py", line 367, in __new__
    type, NamedTuple(name, **{k: annotations[k] for k in entries})
KeyError: 'crypto_stream_salsa20_NONCEBYTES'
...
RuntimeError: Engine core initialization failed. See root cause above.

Python 3.12 is not a way out. After rocm install sdk --version 10.1.0 --python-version 3.12, rocm engines install vllm refuses: "vLLM for ROCm 10.1.0 is published for cp314 only". So there is no working Python for vLLM on ROCm 10.1.0.

I did not confirm the cause. The traceback is in a metaclass __new__ that reads class annotations, which changed in Python 3.14, so that is my guess.

Expected behavior

rocm serve <model> --engine vllm on a fresh ROCm 10.1.0 install either serves the model or is refused up front with a reason, not a startup crash after the install reports dependency_check: satisfied and device_check: usable.

Steps to reproduce

  1. rocm install sdk --channel release --version 10.1.0 --family gfx1151 --yes (managed Python 3.14)
  2. rocm engines install vllm --runtime-id <that runtime>: succeeds.
  3. rocm serve qwen --engine vllm --runtime-id <that runtime>: service ends failed with the traceback above in rocm services logs <id>.

Possible solution

Optional: the install-time dependency check could import the vLLM plugins (or tensorizer) so this surfaces at install, not at serve.

Your environment

  • rocm-cli channel: nightly
  • rocm-cli version (rocm --version): rocm-cli 0.0.0-dev (main, 691834a)
  • Platform: WSL2
  • OS / Distro: Ubuntu 26.04.1 LTS
  • GPU/APU: AMD RYZEN AI MAX+ PRO 395 w/ Radeon 8060S (gfx1151)

Relevant log output

vLLM 0.29.1.dev0+rocm10.1.0, torch 2.13.0+rocm10.1.0; the traceback is the one above (the same KeyError appears in the APIServer and EngineCore processes).

Additional context

Related but separate: re-running rocm install sdk --version 10.1.0 --python-version 3.12 reuses the same runtime key as the Python 3.14 install and replaces that environment in place, removing the installed vLLM, while reporting success.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions