[D4ve0s]

Title

Windows AMD: ADL2 telemetry, one Monitor tile per GPU, images on Windows, and a setup that keeps a two-card split

Issue: Refs #2614, #1380, #112, #2442

Summary

A Windows AMD card had no telemetry: the Monitor's GPU tiles were empty and a PC with two cards had no way to
see them apart. This reads every AMD card through ADL2 PMLog (atiadlxx.dll over ctypes — the same interface
HWInfo and LibreHardwareMonitor use, no new dependency), gives the Monitor one tile per card, and carries the
same numbers into /metrics.

Alongside it, three things that stopped a Windows AMD install working: SETUP renumbered or dropped cards when
HIP_VISIBLE_DEVICES was already in the environment, re-running SETUP collapsed a two-card split back to one
card (the engine then died at start, reported as "the embedding buffer failed"), and Windows had no image
encoder at all.

One commit, d0dcf489 plus the review fixes listed under Telemetry and Tests: 35 files, +1501/−145 against
main (v0.1.44.1). Everything is opt-in — the CUDA path is untouched, --yes never asks the new questions,
AMD images stay off by default, and the default install path writes the same config as v0.1.44.1. No engine
(C++/CUDA/HIP kernel) changes.

What changed

Telemetry (serve/telemetry.py, +366)

• _Adl — ctypes bindings for ADL2 PMLog: adapter enumeration, PmLogStart with the sensor set (edge/mem
  temp, board power, gfx/mem activity, bus lanes, power limit), and the malloc callback through
  ucrtbase.dll (the heap atiadlxx.dll frees on).
• _AmdWin — util = max(gfx, mem activity), temp = edge else mem, power = BOARD_POWER (already watts, no
  scaling), VRAM used = DedicatedVRAMUsage, total = MemoryInfoX4. Enumerates present discrete cards,
  deduped by the UDID's DEV_xxxx code (each card repeats once per display), integrated GPU skipped.
• Every discrete card is sampled (amd_card_count()), not just the display-driving one — Adapter_Get hid a
  second idle card. HIP indices and ADL indices are different namespaces and are never mixed.
• PCIe per card — PMLog BUS_SPEED is the GPU's internal bus and power-saves to Gen4 at idle, so the slot's
  generation comes from Device Manager (DEVPKEY_PciDevice_MaxLinkSpeed, one cached PowerShell query keyed by
  DEV code) and the negotiated width from PMLog BUS_LANES. Matches HWInfo on this PC: 9070 XT Gen5 x8,
  R9700 Gen5 x4.
• A reader gives back what the driver lent it: close() releases the PMLog device handle and the ADL context,
  a finaliser covers readers that go away with the server, and the one-off call sites (free_vram_mib(),
  server.gpu_vram_mib()) release theirs. Numbers under Measured.
• sample() always emits gpus[] with name, pcie_gen_max and pcie_width per card, even for one card.
• Small NVML fix: power_limit returns float(lim) instead of a raw value that could be 0.

SETUP behaviour (setup.py, +200/−33)

• AMD asks "Do you want images?" everywhere, Windows included (default off; --yes never asks; the
  Unsloth Linux default-on is unchanged). hip_vision() is now platform-neutral: yes/gpu → gpu,
  cpu → cpu, else none.
• previous_amd_split() — a re-run with no --gpu/--gpus keeps the installed config's split (accepts
  "gpu" as a list or a "0,2" string, newest config wins, keeps the split only while every card is still
  present and usable).
• hip_devices() clears HIP_VISIBLE_DEVICES for the probe. A user-set value hid cards and renumbered
  the rest, so a saved split "0,2" named a card that was no longer at 2.
• Windows --gpus works — the old "multi-GPU is Linux only" refusal is gone. engine_step() renumbers
  the split's cards to HIP numbers after the probe (an integrated Radeon takes HIP 0) and prints what it
  settled on.
• hip_vision_windows() — for the ready-made zip: run the zip's encoder when BUILD.json says
  vision/vision_src/vision_archs match; gpu asked without an encoder built for this arch warns and
  falls back to cpu; a stale or missing encoder is recompiled as CPU with MSVC; no compiler offers the winget
  Build Tools (~2 GB), and declining leaves images off.
• install_msvc_build_tools() — winget Microsoft.VisualStudio.2022.BuildTools with the VCTools workload,
  asked first. cmake_build() pip-installs cmake and ninja when they are missing (old installs predate
  requirements.txt).
• VRAM reserve rule — Windows AMD with the CPU encoder writes no --vram-reserve-mib at all, so the engine
  keeps its 2560 MiB WDDM floor (bench: add RX 7900 XT results for IQ3_S and IQ3_XXS #1376, [Windows 10][AMD RX 9070 XT][HIP] Strata 0.1.41 loads IQ3_XXS but crashes on every generation request #1709); a written 700 would have defeated it. With a GPU encoder the
  reserve is max(reserve, 3260) = the 2560 floor + the encoder's 700. The 12 GB tip only shows where it
  applies.

Multi-GPU enumeration and JSON config

• The config is re-read and gpu normalised whether it is a list or a "0,2" string, so an old config survives.
• tools/hip/package_windows.py records vision: "cpu", vision_src (source hash) and vision_archs in
  BUILD.json, so SETUP can tell a matching encoder from a stale one.
• .gitignore adds /strata-*.bak (SETUP's config backups) and build-vision-cpu.bat.

Vision build and packaging

• tools/hip/build_windows.bat builds a portable CPU strata-vision.exe (AVX2, static CRT, llama.cpp pinned
  by setup.get_llama_cpp()) into the zip; package_windows.py --vision ships it.
• tools/vision/CMakeLists.txt gains STRATA_VISION_HIP (GGML_HIP) for the Linux HIP encoder path in
  setup.build_vision_hip() — not yet run on hardware, flagged in TEST_REQUESTS.
• tools/vision/strata_vision.cpp also sets HIP_VISIBLE_DEVICES=-1 in CPU mode — the AMD runtime ignores
  CUDA_VISIBLE_DEVICES.
• tools/strata_mcp.py drops the "hip on Windows" and vision refusals; the MCP strata_install accepts the
  same vision answers as SETUP (gpu is downgraded to cpu by setup itself when the arch has no encoder).

Monitor UI (serve/web/)

• Image encoder card: encoder cache bar, encode-time sparkline, facts (device + threads, last picture with
  tokens/grid/cache-hit, since-start totals, mmproj, picture cap), and a "Reading N pictures" pill and status
  line while an encode runs.
• Per-GPU tiles (renderGpuCards) when the PC has ≥2 cards; summary Power and PCIe tiles get a GPU:0 … /
  GPU:1 … line each; the PCIe tile shows each card's own link ("Gen5 x8", "Gen5 x4").
• app.css/app.js are cache-busted (?v=3) so an open tab can't keep a stale script.

Metrics (serve/server.py, serve/prometheus.py)

• Vision records starts/stops, per-encode times, cache hits, failures and image tokens; /metrics gains a
  vision block, encoding_images, and per-request history rows (images / image tokens / encode ms).
• Prometheus gains 7 series: strata:vision_images_total, image_tokens_total, encode_seconds_total,
  encode_failures_total, cache_images, encoder_up, encode_ms_last (absent on a cache hit). Nothing is
  emitted when the model has no vision.
• engine_args() adds --vram-reserve-mib 3260 for hip on Windows with a GPU encoder (2560 WDDM floor + 700
  for the encoder). gpu_vram_mib() releases the reader it builds per card.

Docs

• AMD_HIP.md: the ADL2 telemetry paragraph with the B650E measurements, the two-cards-one-model Windows
  section, and the Images-on-Windows section (zip encoder, 2.7–2.9 s per 512×512 picture on a 7800X3D, Build
  Tools ~2 GB, the 3260 reserve).
• TROUBLESHOOTING.md: new entry "SETUP.bat forgets a two-card split" with the HIP_VISIBLE_DEVICES root
  cause, how to check it, and the measured before/after probe results.
• DETAILS.md: Monitor sources (ADL2/NVML/sysfs, per-slot PCIe), the strata:vision_* series, the AMD
  image-encoder table row, and the Image encoder card paragraph.
• MULTI_GPU.md: Windows AMD --gpus line. AI_SETUP.md: the AMD images question. TEST_REQUESTS.md: two
  rows — Windows AMD --vision cpu reports wanted, --vision gpu never run on hardware.
• README ×7: "AMD cards read pictures through the processor, on Linux and on Windows alike."

Tests

• New serve/test_telemetry_adl.py (+118): the ADL reader never raises when ADL is missing, readings are sane
  (temp 0–120 °C, watts), every card index below card_count reads, per-card PCIe present, and
  AdlReaderCleanup — close() is safe and idempotent, and 40 one-off free_vram_mib() reads do not grow
  the process handle count (Windows only, skipped where there is no atiadlxx.dll).
• tools/test_setup_amd.py: probe ignores a set HIP_VISIBLE_DEVICES; the vision question is
  platform-neutral.
• tools/test_setup_choices.py: 5 new hip_vision_windows cases — zip encoder used, gpu from zip, gpu falls
  back to cpu when the arch is missing, stale vision_src recompiles, no compiler asks then declines.
• tools/test_setup_unsloth.py: HipImages (Windows asks, yes → cpu encoder with no --vram-reserve-mib in
  the config args, default off, --yes never asks, Linux Unsloth default stays on) and HipSplit (Windows
  --gpus 0,2 renumbers to [1,2] when the iGPU takes 0, setup-again keeps a list or a "0,2" string, a fresh
  PC stays one card, Linux keeps its split).
• serve/test_server.py: the vision instrumentation, and the three AmdTelemetry tests pinned to the Linux
  sysfs reader — they fake an amdgpu tree, so building a real ADL reader here made them depend on the machine
  (No data shown for 9070xt on dashboard. #1380). WebApp.test_page_and_files now matches web/app.js with its ?v=3 cache-bust.
• serve/test_monitor.py, test_prometheus.py, test_vision_elastic.py: the Monitor card's DOM and the
  Prometheus series. tools/test_strata_mcp.py: hip vision cpu/yes/gpu/no accepted on both platforms.

Extra Notes

Test machine

• Windows 11, Ryzen 7 7800X3D, B650E motherboard, Adrenalin 26.9.2
• AMD Radeon AI PRO R9700 32 GB — HIP 0, gfx1201, Gen5 x4 (Device Manager and HWInfo agree: "PCIe v5.0 x16 @ x4")
• AMD Radeon RX 9070 XT 16 GB — HIP 2, gfx1201, Gen5 x8
• 7800X3D integrated graphics (2 CU RDNA2, gfx1036; rocminfo's marketing name is "AMD Ryzen 7 7800X3D 8-Core
  Processor") — HIP 1, skipped by the split and by the ADL2 enumeration
• Model split across the two cards: R9700 layers 0–30, 9070 XT layers 31–47 + head, 131K context
• CPU image encoder running on the 7800X3D; a hand-built HIP encoder also ran on the R9700 (below)

Measured on that machine

• Handle growth (the finding reported on telemetry: GPU load, VRAM, temperature and power for AMD cards on Windows (#1380) #1434). 40 one-off free_vram_mib(0, amd=True) calls hold the
  process handle count at 197 → 197 now that the reader is released. Keeping the readers gave 197 → 278, about
  +2 per call — the same signature crkva measured on a 7900 XTX in telemetry: GPU load, VRAM, temperature and power for AMD cards on Windows (#1380) #1434 (267 → 347). The server's sampler
  keeps one reader per card, so the growth showed up on the one-off calls: free_vram_mib() in the startup
  retry loop and gpu_vram_mib() once per card.
• PCIe per card: 9070 XT Gen5 x8, R9700 Gen5 x4 — Device Manager and HWInfo agree, and the B650E board
  really does bifurcate, so one aggregate link number would be wrong.
• Images: 2.7–2.9 s per 512×512 picture (256 tokens) with the CPU encoder; the screenshot run measured
  3,647 ms for a 23×12 (276-token) picture, cache hit on repeat.
• Prometheus scrape after 4 image requests: strata:vision_images_total 4, vision_image_tokens_total 1073, vision_encode_seconds_total 12.767, vision_encode_failures_total 0, vision_cache_images 4,
  vision_encoder_up 1. /metrics also shows both gpus[] rows with correct ADL2 values.
• SETUP probe with HIP_VISIBLE_DEVICES="0,2" set: the old probe returned 2 cards numbered 0,1; the fixed
  probe returns all 3 (R9700 = HIP 0, 7800X3D iGPU = HIP 1, 9070 XT = HIP 2).

Test suite

• python -m unittest discover -s serve -p "test_*.py": 805 tests in 303 s, OK (15 skipped) on this machine.
• The modules this branch touches, run on their own: test_telemetry_adl 8, test_server 17 (WebApp 14 +
  AmdTelemetry 3), test_monitor + test_prometheus + test_vision_elastic 29 — all passing.
• Setup's own tests, no GPU or downloads: test_setup_amd.py 37, test_setup_choices.py 52,
  test_setup_unsloth.py 68, test_setup_update.py 8, test_setup_sycl.py 18 — all OK.
• The three AmdTelemetry tests that already failed on main on this two-card PC now pass; nothing is left
  failing on this machine.
• The Windows HIP engine and the CPU strata-vision.exe were built and then run end to end on this hardware
  (throwaway build/run scripts, not added to the repo).

The GPU image encoder on Windows (built by hand, not shipped)

The zip ships the CPU encoder and package_windows.py stamps BUILD.json vision: "cpu", vision_archs: [] —
no published zip can select the GPU encoder yet. A HIP strata-vision.exe can still be built on Windows and run
beside the ready-made engine; verified on this PC (gfx1201):

• Build: vcvars64 + the .rocm-win venv's ROCm (rocm-sdk.exe path --root), then tools/vision with
  -DSTRATA_VISION_HIP=ON -DSTRATA_VISION_CUDA=OFF, ROCm's clang/clang++ as the C/C++ compilers,
  -DCMAKE_PREFIX_PATH=<rocm> and GPU_TARGETS=gfx1201 (AMDGPU_TARGETS is deprecated). The ROCm flags must
  ride in CMAKE_CXX_FLAGS (--rocm-path=… --rocm-device-lib-path=…): on Windows ggml-hip forces
  CXX_IS_HIPCC and cmake ignores CMAKE_HIP_FLAGS, so without them the compile fails with "cannot find ROCm
  device library".
• Selection: engine\BUILD.json must say vision: "gpu" and vision_archs must name the card, or
  hip_vision_windows() warns and falls back to cpu. The config's vision section then carries gpu: true,
  max_tokens: 1024 and --vram-reserve-mib 3260 — SETUP writes all three on a fresh install; re-running over
  an installed model keeps the old config as-is (the reuse path starts it directly), so changing an installed
  model to the GPU encoder means editing the config or re-running with --setup.
• Run: the server puts engine\rocm\bin on the encoder's PATH (the config's lib_dirs), where hipBLAS and
  rocBLAS live; hip_runtime_beside_exe copies only the HIP runtime beside the exes by design. Run from a bare
  terminal the exe dies in the loader with 0xC0000135 (DLL not found) and prints nothing — prepend
  engine\rocm\bin to PATH for a standalone test.
• Measured on the R9700: model files loaded in 3.0 s, warm-up at 4,096 image tokens in 5.3 s, READY 2560;
  the Monitor's image card reads "running on the GPU", cap 1,024. Per-picture encode times: not measured yet.

Shipping it would mean building the HIP encoder in build_windows.bat and stamping vision_archs in
package_windows.py — a separate change, not in this PR.

Screenshots

Live from the machine above, model loaded, after real image requests. The PNGs are not committed.

Where this overlaps

Several open threads cover part of this, and they disagree in places. Flagging it rather than adding a sixth
silently:

• AMD Windows telemetry: Windows + AMD: dashboard GPU tiles stay blank (telemetry is Linux-sysfs only) — fixed locally via ADL, incl. a PCIe link-speed pitfall #2614 (open issue, the place to discuss it), telemetry: GPU load, VRAM, temperature and power for AMD cards on Windows (#1380) #1434 (gt50 — ADL_POWER = (73, 23, 30) and a close() after crkva's 7900 XTX validation), Dashboard: GPU load, VRAM, temperature, power and PCIe for AMD cards … #1871 (kaiw1102), serve: an AMD card's load, VRAM, temperature and power on Windows #2273 (YashwantBharaj, rebased on
  telemetry: GPU load, VRAM, temperature and power for AMD cards on Windows (#1380) #1434), serve: the Monitor's AMD GPU readings on Windows through ADLX (load, VRAM, temperature, power) #1224 (Annoosha-1, sensor 23 first and a fresh _Adl2() per call — the pattern telemetry: GPU load, VRAM, temperature and power for AMD cards on Windows (#1380) #1434 was called out
  for). This branch prefers sensor 73 (PMLOG_BOARD_POWER) for the reason crkva gave and releases handles the
  way telemetry: GPU load, VRAM, temperature and power for AMD cards on Windows (#1380) #1434 does; it reads the slot's generation from Device Manager rather than PMLog BUS_SPEED, which
  power-saves to Gen4 at idle on this board. Happy to rebase onto whichever lands first, or to port the
  SETUP/vision/Monitor parts onto that reader and drop this one.
• Card numbering: Fix Windows AMD telemetry and add optional Arabic UI #1152 (crkva) enumerates through HIP and [Bug] setup.sh can't setup dual nvidia GPUs #112 asks for multi-GPU on Windows. This branch
  enumerates through ADL2 and keeps HIP and ADL indices as separate namespaces; --gpus on Windows is the part
  [Bug] setup.sh can't setup dual nvidia GPUs #112 asks for.
• Windows AMD build/packaging: Strata Windows AMD HIP Build Guide (gfx1200 / gfx1201) AMD 9060XT16G strata-vision.exe #2442 is a community guide to building this by hand, including a hand-built
  strata-vision.exe; the zip encoder here is the same thing, shipped by the installer.
• Vision metrics: Monitor: one line per card on a layer split (load, VRAM, temperature, power, PCIe) #2317 and Monitor: the PCIe traffic from the bytes the engine moved, not from a driver counter #2272 touch /metrics and the Monitor. The strata:vision_* series are new
  names and emit nothing when the model has no vision.

Not here

• No CUDA/NVIDIA behaviour change beyond the power_limit guard.
• The Linux HIP GPU encoder compiles but has never been run on hardware — TEST_REQUESTS asks for reports;
  SETUP still offers it knowingly. (A hand-built Windows HIP encoder ran on gfx1201 — see above — but the zip
  still ships the CPU one and no published zip can select a GPU encoder yet.)
• No engine (C++/CUDA/HIP kernel) changes: this is setup, serve, tools and docs.

[Niko1221]

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.