Add AMD ROCm/HIP backend for SmolVLA and π0.5 on gfx1151

CHANGELOG.md

@@ -6,6 +6,15 @@ Notable changes to vla.cpp. Format loosely follows [Keep a Changelog](https://ke
 
 ### Added
 
+- **ROCm/HIP backend for VLA inference.** `-DGGML_HIP=ON` selects ggml's HIP
+  backend without compiling vla.cpp's CUDA-only kernels. On Linux `gfx1200`
+  (Radeon RX 9060 XT) with ROCm 7.14 and llama.cpp `b11223`, every arch except
+  BitVLA (int2, CUDA only) runs and is benchmarked in
+  `docs/benchmark/rx-9060-xt.md`, and all but OpenVLA-OFT were checked against
+  the CPU build on fixed inputs; SmolVLA and π0.5 were also checked on
+  `gfx1151`. SmolVLA takes the same F32 LM weights and F16
+  cross-attention K/V on HIP as on CUDA. See `docs/backend/rocm.md` for the
+  measured numerical differences, build commands and current limits.
 - **New model: ACT** (LeRobot `policies/act`). `scripts/convert_act_to_gguf.py`
   converts a LeRobot `pretrained_model` directory, folding the ResNet's frozen
   batch norms and dropping the training-only VAE encoder. No language input, so

CMakeLists.txt

@@ -19,7 +19,7 @@ elseif(UNIX)
 endif()
 
 set(_vla_accel "")
-foreach(_flag GGML_CUDA GGML_SYCL GGML_METAL GGML_OPENVINO GGML_HEXAGON GGML_OPENCL)
+foreach(_flag GGML_CUDA GGML_HIP GGML_VULKAN GGML_SYCL GGML_METAL GGML_OPENVINO GGML_HEXAGON GGML_OPENCL)
     if(${_flag})
         list(APPEND _vla_accel ${_flag})
     endif()
@@ -39,7 +39,7 @@ if(_vla_accel_n GREATER 0 AND GGML_BACKEND_DL)
         "GGML_BACKEND_DL=ON is not supported with ${_vla_accel}: the backend is "
         "built as a module and src/backend.h links its init directly.")
 endif()
-foreach(_flag GGML_VULKAN GGML_HIP GGML_MUSA GGML_CANN GGML_WEBGPU)
+foreach(_flag GGML_VULKAN GGML_MUSA GGML_CANN GGML_WEBGPU)
     if(${_flag})
         message(WARNING "${_flag}=ON reaches vlm-server through llama only; src/backend.h has no ${_flag} path, so the VLA archs run on CPU.")
     endif()
@@ -375,6 +375,9 @@ if(GGML_CUDA)
     )
     target_link_libraries(vla_cuda_fused PUBLIC CUDA::cublas CUDA::cublasLt CUDA::cudart ggml-cuda vla_cuda_ops)
     target_link_libraries(vla_core PRIVATE vla_cuda_fused)
+elseif(GGML_HIP)
+    # ggml-hip also exports GGML_USE_CUDA; distinguish HIP from CUDA-only code.
+    target_compile_definitions(vla_core PUBLIC GGML_USE_HIP)
 endif()
 
 if(GGML_SYCL AND NOT GGML_CUDA)

README.md

@@ -12,8 +12,9 @@ A C++ inference engine for **Vision-Language-Action (VLA) models**, built on [`l
 It runs the open VLA policies - SmolVLA, π0, BitVLA, Evo-1, GR00T N1.5/1.6/1.7 and more -
 under one runtime, each packaged as a single self-contained GGUF that needs no Python or
 PyTorch at inference time. The binaries drive robots on **CPU**, **Apple Silicon**, **CUDA** -
-from consumer GPUs down to Jetson-class boards - **Intel GPUs and NPUs** via
-SYCL and OpenVINO, or **Qualcomm Snapdragon CPUs, Adreno GPUs and Hexagon NPUs**
+from consumer GPUs down to Jetson-class boards - **AMD Radeon GPUs** via ROCm/HIP,
+**Intel GPUs and NPUs** via SYCL and OpenVINO, or **Qualcomm Snapdragon CPUs,
+Adreno GPUs and Hexagon NPUs**
 via OpenCL and the Hexagon backend.
 
 [**Learn vla.cpp**](https://fai-modelopt-tech.github.io/learn-vla-cpp/) walks through the engine design and how each policy is implemented on ggml.
@@ -57,6 +58,8 @@ tarball). Details, and the macOS and Windows notes, are in
 - CMake ≥ 3.22
 - A C++17 compiler (GCC 11+ or Clang 14+)
 - CUDA 12.x or 13.x (optional - required only for CUDA GPU builds)
+- ROCm with HIP, hipBLAS, rocBLAS, hipBLASLt and their development files
+  (optional - see [the tested AMD configurations](docs/backend/rocm.md))
 - Intel oneAPI 2025.x + GPU compute runtime (optional - only for Intel GPU
   builds, see [docs/backend/sycl.md](docs/backend/sycl.md))
 - OpenVINO 2026.x runtime (optional - only for Intel CPU/GPU/NPU builds via
@@ -97,6 +100,20 @@ cmake -B build \
 cmake --build build -j$(nproc)
 ```
 
+For ROCm/HIP, use a separate build directory and the target reported by
+`rocminfo` (`gfx1200` for the tested Radeon RX 9060 XT, `gfx1151` for the
+Radeon 8060S):
+
+```bash
+HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
+cmake -B build-rocm -G Ninja -DGGML_HIP=ON -DGPU_TARGETS=gfx1200 \
+    -DCMAKE_BUILD_TYPE=Release
+cmake --build build-rocm -j$(nproc)
+```
+
+The [ROCm backend notes](docs/backend/rocm.md) cover the runtime library path,
+device selection, validation and limits.
+
 If CMake cannot find CUDA, point the environment at it explicitly:
 
 ```bash
@@ -114,7 +131,7 @@ tree. `pip install ./bindings/python` builds the Python bindings, see
 [bindings/python/README.md](bindings/python/README.md).
 
 Check [docs/backend](docs/backend) for compiling `vla.cpp` on other platforms.
-WSL2, Apple Silicon, and Intel GPU are all tested.
+ROCm on Linux (`gfx1200`, `gfx1151`), WSL2, Apple Silicon, and Intel GPU are all tested.
 To build and run in containers instead, see [docs/DOCKER.md](docs/DOCKER.md).
 
 ---
@@ -148,22 +165,29 @@ variables are in [docs/USAGE.md](docs/USAGE.md).
 Models (rows) against platforms (columns). Legend: `Y` =
 supported (released and benchmarked), `~` = in progress, `-` = planned.
 
-| Model | CPU (x86-64 / ARM) | CUDA | [SYCL (Intel)](docs/backend/sycl.md) | [Metal](docs/backend/metal.md) | [OpenVINO](docs/backend/ov.md) | [Hexagon](docs/backend/hexagon-windows.md) |
-|---|:--:|:--:|:--:|:--:|:--:|:--:|
-| [SmolVLA](https://hf.co/vrfai/smolvla-libero-gguf)             | Y | Y | Y | Y | Y | Y | 
-| [π0](https://hf.co/vrfai/pi0-libero-finetuned-v044-gguf)       | Y | Y | - | Y | Y | ~ | 
-| [π0.5](https://hf.co/vrfai/pi05-libero-gguf)                   | Y | Y | - | Y | Y | Y | 
-| [GR00T N1.5](https://hf.co/vrfai/gr00tn1d5-libero-object-gguf) | Y | Y | - | Y | Y | Y | 
-| [GR00T N1.6](https://hf.co/vrfai/gr00tn1d6-libero-gguf)        | Y | Y | - | Y | Y | Y | 
-| [GR00T N1.7](https://hf.co/vrfai/gr00tn1d7-libero-gguf)        | Y | Y | - | Y | Y | Y | 
-| [BitVLA](https://hf.co/vrfai/bitvla-libero-gguf)               | Y | Y | - | ~ | - | - | 
-| [Evo-1](https://hf.co/vrfai/evo1-libero-gguf)                  | Y | Y | Y | Y | Y | Y | 
-| [VLA-Adapter](https://hf.co/vrfai/vla-adapter-libero-gguf)     | Y | Y | ~ | Y | Y | Y | 
-| [OpenVLA-OFT](https://hf.co/vrfai/openvla-oft-libero-gguf)     | Y | Y | - | Y | Y | - | 
-| [VLA-JEPA](https://hf.co/vrfai/vla-jepa-libero)                | Y | Y | - | Y | Y | ~ | 
-| [Octo-Small](https://hf.co/vrfai/octo-small-libero-gguf)       | Y | Y | Y | Y | - | Y | 
-| [TurboVLA](https://hf.co/vrfai/turbovla-libero-gguf)           | Y | Y | Y | Y | Y | Y | 
-| [ACT](https://hf.co/ravediamond/act-aloha-sim-transfer-cube-gguf) | Y | Y | - | Y | - | - | 
+| Model | CPU (x86-64 / ARM) | CUDA | [ROCm (AMD)](docs/backend/rocm.md) | [SYCL (Intel)](docs/backend/sycl.md) | [Metal](docs/backend/metal.md) | [OpenVINO](docs/backend/ov.md) | [Hexagon](docs/backend/hexagon-windows.md) |
+|---|:--:|:--:|:--:|:--:|:--:|:--:|:--:|
+| [SmolVLA](https://hf.co/vrfai/smolvla-libero-gguf)             | Y | Y | Y | Y | Y | Y | Y |
+| [π0](https://hf.co/vrfai/pi0-libero-finetuned-v044-gguf)       | Y | Y | Y | - | Y | Y | ~ |
+| [π0.5](https://hf.co/vrfai/pi05-libero-gguf)                   | Y | Y | Y | - | Y | Y | Y |
+| [GR00T N1.5](https://hf.co/vrfai/gr00tn1d5-libero-object-gguf) | Y | Y | Y | - | Y | Y | Y |
+| [GR00T N1.6](https://hf.co/vrfai/gr00tn1d6-libero-gguf)        | Y | Y | Y | - | Y | Y | Y |
+| [GR00T N1.7](https://hf.co/vrfai/gr00tn1d7-libero-gguf)        | Y | Y | Y | - | Y | Y | Y |
+| [BitVLA](https://hf.co/vrfai/bitvla-libero-gguf)               | Y | Y | - | - | ~ | - | - |
+| [Evo-1](https://hf.co/vrfai/evo1-libero-gguf)                  | Y | Y | Y | Y | Y | Y | Y |
+| [VLA-Adapter](https://hf.co/vrfai/vla-adapter-libero-gguf)     | Y | Y | Y | ~ | Y | Y | Y |
+| [OpenVLA-OFT](https://hf.co/vrfai/openvla-oft-libero-gguf)     | Y | Y | Y | - | Y | Y | - |
+| [VLA-JEPA](https://hf.co/vrfai/vla-jepa-libero)                | Y | Y | Y | - | Y | Y | ~ |
+| [Octo-Small](https://hf.co/vrfai/octo-small-libero-gguf)       | Y | Y | Y | Y | Y | - | Y |
+| [TurboVLA](https://hf.co/vrfai/turbovla-libero-gguf)           | Y | Y | Y | Y | Y | Y | Y |
+| [ACT](https://hf.co/ravediamond/act-aloha-sim-transfer-cube-gguf) | Y | Y | Y | - | Y | - | - |
+
+ROCm `Y` means fixed-input agreement with the CPU build and a synthetic-input
+benchmark on Linux `gfx1200` (Radeon RX 9060 XT) with ROCm 7.14 and llama.cpp
+`b11223`; SmolVLA and π0.5 were also checked on `gfx1151`. OpenVLA-OFT, too
+large for the CPU reference host, was checked for finite, repeatable actions
+only. BitVLA's published int2 GGUF needs vla.cpp's CUDA kernels. These engine
+checks do not establish robot-task success.
 
 Adding a model? Use [ggml-compiler](https://github.com/khanhnd61-vr/ggml-compiler)
 to generate and verify ggml C++ from its PyTorch implementation, then integrate
@@ -234,7 +258,7 @@ Wiring, recording, training and queue sizing are in the
 | [docs/QUANTIZATION.md](docs/QUANTIZATION.md) | FoldQuant W8A8 / W4A4: the GGUF contract, the integer arithmetic, converting a FoldQuantVLA quantized model |
 | [docs/DOCKER.md](docs/DOCKER.md) | Building and running the eval in containers |
 | [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | Engine design: layers, the prediction path, backends, adding an architecture |
-| [docs/backend/](docs/backend) | Per-backend build and run notes: [SYCL](docs/backend/sycl.md), [OpenVINO](docs/backend/ov.md), [Metal](docs/backend/metal.md), [Hexagon](docs/backend/hexagon.md), [Hexagon on Windows](docs/backend/hexagon-windows.md), [WSL2](docs/backend/wsl.md), [CUDA bit-exact path](docs/backend/cuda-exact.md) |
+| [docs/backend/](docs/backend) | Per-backend build and run notes: [ROCm](docs/backend/rocm.md), [SYCL](docs/backend/sycl.md), [OpenVINO](docs/backend/ov.md), [Metal](docs/backend/metal.md), [Hexagon](docs/backend/hexagon.md), [Hexagon on Windows](docs/backend/hexagon-windows.md), [WSL2](docs/backend/wsl.md), [CUDA bit-exact path](docs/backend/cuda-exact.md) |
 | [docs/benchmark/](docs/benchmark) | Per-device latency and memory for every model, and the fastest flags per device |
 | [docs/KNOWN_ISSUES.md](docs/KNOWN_ISSUES.md) | Known issues and their resolutions |
 | [docs/ADOPTION.md](docs/ADOPTION.md) | C ABI, Python bindings, release packaging, and what is left |

docs/ARCHITECTURE.md

@@ -2,8 +2,8 @@
 
 vla.cpp runs Vision-Language-Action (VLA) policies on the ggml/llama.cpp runtime.
 Every model is a self-contained GGUF that the engine loads, detects, and drives on
-CPU, CUDA, Metal, SYCL, OpenVINO, OpenCL or Hexagon. This page is the map; the
-source is the detail.
+CPU, CUDA, ROCm/HIP, Metal, SYCL, OpenVINO, OpenCL or Hexagon. This page is the
+map; the source is the detail.
 
 ## Layers