Skip to content

[Issue]: rocm-cli managed lemonade rocm backend hangs on install / uses extra space on windows #392

Closed

Description

@kgskgs

Problem Description

When setting up rocm, I ran into the following issue - when installing the lemonade llama.cpp:rocm backend, the installation timed out in lemonade while extracting therock binary (from a downloaded tarball).
However lemonade does not need to install this at all since everything has already been installed by the rocm wheels. Other than timing out on windows, this installation takes up 24GB extra space. Admittedly, it could be some misconfiguration on my end, but I couldn't find info about it online...

Below is an AI generated summary of the changes made to make it work:

## 1. Environment Variable Overrides (Registry Level)
The following environment variables were permanently injected into your Windows User registry to patch the behavior of the Lemonade server and the ROCm math libraries.

### Lemonade Server Patches:
*   `LEMONADE_NO_FETCH_EXECUTABLES=true`
    *   **Default:** `false` (implicit)
    *   **Purpose:** The official kill-switch that forces the Lemonade server to completely skip the automated downloading of backend binaries (like the massive 30GB `therock` tarball) during installation and startup.

### ROCm SDK Pointers:
To resolve a version mismatch with your global Python environment (`v10.0.0` vs `v7.13.0`), these variables were set to point directly to the fully extracted SDK embedded inside the Python wheels on your disk:
*   `ROCM_PATH = C:\Users\k\.rocm\runtimes\wheel\release-wheel-gfx110x-all-7-13-0\Lib\site-packages\_rocm_sdk_core`
*   `HIP_PATH = C:\Users\k\.rocm\runtimes\wheel\release-wheel-gfx110x-all-7-13-0\Lib\site-packages\_rocm_sdk_core`

### Windows PATH Additions:
In order for the `llama-server.exe` backend to dynamically load the GPU math `.dll` files (`ggml-hip.dll`, `rocblas.dll`, etc.) at runtime, the following folders were appended to the very end of your Windows `PATH` environment variable:
1. `C:\Users\k\.rocm\runtimes\wheel\release-wheel-gfx110x-all-7-13-0\Lib\site-packages\_rocm_sdk_core\bin`
2. `C:\Users\k\.rocm\runtimes\wheel\release-wheel-gfx110x-all-7-13-0\Lib\site-packages\_rocm_sdk_libraries_gfx110X_all\bin`

## 2. Configuration File Modifications (`config.json`)
The core configuration file for the embeddable Lemonade engine was modified to treat the backend as an external executable.
**File Location:** `C:\Users\k\.rocm\runtimes\wheel\release-wheel-gfx110x-all-7-13-0\engines\lemonade\runtime\config.json`

### Changes Made:
*   **`llamacpp.rocm_bin`**
    *   **Default:** `"builtin"`
    *   **New Value:** `"C:\\Users\\k\\.rocm\\runtimes\\wheel\\release-wheel-gfx110x-all-7-13-0\\engines\\lemonade\\runtime\\bin\\llamacpp\\rocm-stable\\llama-server.exe"`
    *   **Purpose:** By changing this from `builtin` to an absolute path, Lemonade treats the backend as an externally provided, manually managed binary. This completely bypasses all internal validation, version checking, and redownloading logic that was causing the server to panic on startup.

Operating System

Windows 10.0.19045

CPU

AMD Ryzen 5 5600X 6-Core Processor

GPU

AMD Radeon RX 7900 XTX

ROCm Version

rocm 0.1.0

Installation Method

Python wheels

Installed ROCm Packages / Versions

Installed ROCm packages / versions
Paste output here

ROCm Component

No response

Steps to Reproduce

No response

(Optional for Linux users) Output of rocminfo --support

rocminfo --support output
Paste output here

Additional Information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions