Skip to content

hyperloom custom workload: the result can never say whether the graph stayed on HRX -- placement is null in all 147 artifacts #1008

Closed

Description

@bong-water-water-bong

Class: the artifact does not describe the measurement (same as #892, #930, #932, #947, #954, #977, #993 — all fixed),
but this one is a field that already exists and has never once held a value.

bench_summary.placement is null in 147 of 147 artifacts on this box (every result carrying a bench_summary), and
0 of the 71 serve.log files beneath ~/hyperloom-ws contain the line the parser is written for:

$ python3 - <<'PY'   # counts over ~/hyperloom-ws, 2026-11-04 13:2x
files scanned 7102  with bench_summary 147  placement null 147
PY
$ grep -rl "off HRX" ~/hyperloom-ws --include=serve.log | wc -l
0

placement_summary()/PLACEMENT_RE work — fed the engine's own documented line (docs/serve.md,
tests/hrx_placement_test.cpp:39):

placement_observer: placement: 512 nodes, 34 off HRX (MUL_MAT x 30, GET_ROWS x 4), 7 graph splits
-> {'graphs_reported': 1, 'nodes': 512, 'off_hrx': 34, 'worst_graph_off_hrx': 34}

The measurement simply never asks for the report. GGML_HRX_REPORT=2 (set by serve_measure) only makes ggml-hrx log
the counts, and llama-server maps the library's INFO lines to its trace level, so they reach serve's output only below
-lv 4. serve is the one that passes -lv 4, for --placement-report (and --strict-device)
(app/serve.cpp:1059, app/hrx_strict.h:60), and the workload passes neither. The comment that used to sit on that env
line asserted the opposite — "1bit serve exposes no -lv passthrough" — which is how the field stayed empty for the
whole series. (That sentence also went into #993's body; it is wrong and it is corrected there.)

Consequence: a result cannot say whether the graph it measured stayed on HRX. On this stack a node off HRX is a bug to
report (docs/hw-gate.md), and the one place the workload would say it is the field that is always null.

Wanted: the serve leg asks for the placement report, so every artifact records
{"graphs_reported", "nodes", "off_hrx", "worst_graph_off_hrx"}, and "not captured" (null) stops being the only value
the field ever has.

Note for comparability: --placement-report lifts the backend's verbosity, so absolute numbers after it are not directly
comparable with the numbers of record (measured at serve's default verbosity); it applies to the baseline and every
candidate alike, so lever deltas stay valid.

Activity

  1. added a commit that references this issue on Nov 4, 2026
    0953281
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions