You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
hyperloom custom workload: the quality-gate verdict does not name the reference it was measured against, so a KLD number cannot be compared across sessions #954
Class: the artifact does not describe the measurement (same as #892, #930, #932, #947 — all fixed).
benchmark.py decides pass/fail by comparing a candidate against two references kept in HL_STATE
(~/lb/hl-gate/<model-stem>): kld-base.kld (1.0 GB of base logits) and texts.json (the baseline's three greedy
replies). The result JSON records the candidate's own numbers only:
So a row that reads KLD 0.000e+00 <= 0.0001, texts identical cannot be tied to the reference it is a distance to.
That state is per box, outside the checkout, and kld()re-records the base whenever kld-base.kld is absent
(recording = not base.is_file()), so the same KLD number can be PASS against one base and meaningless against
another — while the reports of the two series look identical. ref.json does carry a model fingerprint and the
baseline's levers, but neither reaches the result.
Wanted: every compared result names the reference it was compared against.
Activity