Repository navigation
Closed
Description
Problem Description
When setting up rocm, I ran into the following issue - when installing the lemonade llama.cpp:rocm backend, the installation timed out in lemonade while extracting therock binary (from a downloaded tarball).
However lemonade does not need to install this at all since everything has already been installed by the rocm wheels. Other than timing out on windows, this installation takes up 24GB extra space. Admittedly, it could be some misconfiguration on my end, but I couldn't find info about it online...
Below is an AI generated summary of the changes made to make it work:
## 1. Environment Variable Overrides (Registry Level)
The following environment variables were permanently injected into your Windows User registry to patch the behavior of the Lemonade server and the ROCm math libraries.
### Lemonade Server Patches:
* `LEMONADE_NO_FETCH_EXECUTABLES=true`
* **Default:** `false` (implicit)
* **Purpose:** The official kill-switch that forces the Lemonade server to completely skip the automated downloading of backend binaries (like the massive 30GB `therock` tarball) during installation and startup.
### ROCm SDK Pointers:
To resolve a version mismatch with your global Python environment (`v10.0.0` vs `v7.13.0`), these variables were set to point directly to the fully extracted SDK embedded inside the Python wheels on your disk:
* `ROCM_PATH = C:\Users\k\.rocm\runtimes\wheel\release-wheel-gfx110x-all-7-13-0\Lib\site-packages\_rocm_sdk_core`
* `HIP_PATH = C:\Users\k\.rocm\runtimes\wheel\release-wheel-gfx110x-all-7-13-0\Lib\site-packages\_rocm_sdk_core`
### Windows PATH Additions:
In order for the `llama-server.exe` backend to dynamically load the GPU math `.dll` files (`ggml-hip.dll`, `rocblas.dll`, etc.) at runtime, the following folders were appended to the very end of your Windows `PATH` environment variable:
1. `C:\Users\k\.rocm\runtimes\wheel\release-wheel-gfx110x-all-7-13-0\Lib\site-packages\_rocm_sdk_core\bin`
2. `C:\Users\k\.rocm\runtimes\wheel\release-wheel-gfx110x-all-7-13-0\Lib\site-packages\_rocm_sdk_libraries_gfx110X_all\bin`
## 2. Configuration File Modifications (`config.json`)
The core configuration file for the embeddable Lemonade engine was modified to treat the backend as an external executable.
**File Location:** `C:\Users\k\.rocm\runtimes\wheel\release-wheel-gfx110x-all-7-13-0\engines\lemonade\runtime\config.json`
### Changes Made:
* **`llamacpp.rocm_bin`**
* **Default:** `"builtin"`
* **New Value:** `"C:\\Users\\k\\.rocm\\runtimes\\wheel\\release-wheel-gfx110x-all-7-13-0\\engines\\lemonade\\runtime\\bin\\llamacpp\\rocm-stable\\llama-server.exe"`
* **Purpose:** By changing this from `builtin` to an absolute path, Lemonade treats the backend as an externally provided, manually managed binary. This completely bypasses all internal validation, version checking, and redownloading logic that was causing the server to panic on startup.
Operating System
Windows 10.0.19045
CPU
AMD Ryzen 5 5600X 6-Core Processor
GPU
AMD Radeon RX 7900 XTX
ROCm Version
rocm 0.1.0
Installation Method
Python wheels
Installed ROCm Packages / Versions
Installed ROCm packages / versions
Paste output here
ROCm Component
No response
Steps to Reproduce
No response
(Optional for Linux users) Output of rocminfo --support
rocminfo --support output
Paste output here
Additional Information
No response
Metadata
Metadata
Assignees
Labels
No labels
Activity
fix(lemonade): enable backend version alignment on Windows
fix(lemonade): enable backend version alignment on Windows
fix(lemonade): enable backend version alignment on Windows (ROCm#576)
fix(lemonade): enable backend version alignment on Windows (#576)