Skip to content

tests: make iq_parity reproducible from a fresh checkout - #264

Closed
j-luwierski wants to merge 1 commit into
Niko1221:mainfrom
j-luwierski:todo24-iq-fixture
Closed

j-luwierski wants to merge 1 commit into
Niko1221:mainfrom
j-luwierski:todo24-iq-fixture

Conversation

@j-luwierski

Copy link
Copy Markdown
Contributor

The i-quant fixtures were never published, so iq_parity reported ten "missing fixture" failures on any fresh tree. tools/iq_fixture.py now generates them deterministically (seeded rows/cols, RandomState) into the BUILD directory: for each of the ten types it writes the raw block bytes (sane fp16 scales at each layout's scale offsets, seeded index and sub-scale bytes) plus the gguf-py dequantized reference - gguf-py is dequant-only for the i-quants, so no quantizer is needed, and Q2_0 (the repository's type 42) uses tools/gguf_writer.py's codec. The kernel type ids in the headers match the vendored gguf-py's enum one for one. Dependencies: python3 with numpy and the vendored gguf-py - nothing is pip-installed, and CMake probes the configured interpreter and .venv, registering the tests only when a usable interpreter is found and saying so otherwise.

CTest now runs iq_parity (previously built but never registered) with the fixtures as an explicit CTest fixture: generation completes before the test even under parallel execution, and a generation failure fails the run. The missing-fixture diagnostic names both candidate files and the generating command; the nonzero exit is preserved.

The i-quant fixtures were never published, so iq_parity reported ten
"missing fixture" failures on any fresh tree.  tools/iq_fixture.py now
generates them deterministically (seeded rows/cols, RandomState) into
the BUILD directory: for each of the ten types it writes the raw block
bytes (sane fp16 scales at each layout's scale offsets, seeded index and
sub-scale bytes) plus the gguf-py dequantized reference - gguf-py is
dequant-only for the i-quants, so no quantizer is needed, and Q2_0 (the
repository's type 42) uses tools/gguf_writer.py's codec.  The kernel
type ids in the headers match the vendored gguf-py's enum one for one.
Dependencies: python3 with numpy and the vendored gguf-py - nothing is
pip-installed, and CMake probes the configured interpreter and .venv,
registering the tests only when a usable interpreter is found and
saying so otherwise.

CTest now runs iq_parity (previously built but never registered) with
the fixtures as an explicit CTest fixture: generation completes before
the test even under parallel execution, and a generation failure fails
the run.  The missing-fixture diagnostic names both candidate files and
the generating command; the nonzero exit is preserved.

Co-Authored-By: Claude <noreply@anthropic.com>
Niko1221 added a commit that referenced this pull request Oct 1, 2026
…iling when the vendored gguf-py is absent; no unused SCALE_F16

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
Niko1221 added a commit that referenced this pull request Oct 1, 2026
…generated, as its generator does; a partly missing set still fails; the open handle of a half-present fixture is closed

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
@Niko1221

Niko1221 commented Oct 1, 2026

Copy link
Copy Markdown
Owner

Released in 0.1.31: with a follow-up: the fixture step and iq_parity skip (exit 3) instead of failing when the vendored gguf-py is absent. Merged under your name (GitHub doesn't show it as merged because it went in by hand). Thank you!

@Niko1221 Niko1221 closed this Oct 1, 2026
Niko1221 added a commit that referenced this pull request Oct 6, 2026
…iling when the vendored gguf-py is absent; no unused SCALE_F16
Niko1221 added a commit that referenced this pull request Oct 6, 2026
…generated, as its generator does; a partly missing set still fails; the open handle of a half-present fixture is closed
anon761 pushed a commit to anon761/Strata-upstream that referenced this pull request Oct 6, 2026
…ad of failing when the vendored gguf-py is absent; no unused SCALE_F16
anon761 pushed a commit to anon761/Strata-upstream that referenced this pull request Oct 6, 2026
…ure was generated, as its generator does; a partly missing set still fails; the open handle of a half-present fixture is closed
anon761 pushed a commit to anon761/Strata-upstream that referenced this pull request Oct 6, 2026
…ad of failing when the vendored gguf-py is absent; no unused SCALE_F16

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
anon761 pushed a commit to anon761/Strata-upstream that referenced this pull request Oct 6, 2026
…ure was generated, as its generator does; a partly missing set still fails; the open handle of a half-present fixture is closed

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0154bwiad8gY7wRehCXy27pN
ScrambledEggsOnToast pushed a commit to ScrambledEggsOnToast/Strata that referenced this pull request Oct 19, 2026
Niko1221#259: execution_plan.hpp resolves the option-level contract once - requested
and effective speculative widths, slot groups, proposal shape, lookup
eligibility, the foreground budget, concurrent-reader exclusions and prompt
suspension - and generate.cpp consumes the resolved widths and budget
decision. EPLAN reports it before READY; --report-plan prints it before any
model allocation. The frontend records it for /metrics and /v1/status.

Niko1221#260: ResidentSlots exposes a read-only identity and owns(). Legacy resident
admissions and terminals carry request=/incarnation=; BSTOP fences by that
identity and ignores a stale owner; the frontend records the admitted
identity, sends it on cancellation, and never applies a mismatched terminal.

Niko1221#264: opt-in STRATA_DECODE_SERVICE selects ready requests by a weighted 3:1
service cycle (one ticket = one decode window) and keeps an interactive
window free of background rows. Measured and left off by default.

Niko1221#265: a refused yielded-prefix continuation is preflight-only and ends one
request with an ERR terminal; the engine continues serving its peers.

Docs: DETAILS.md gains the resolved-plan and decode-service sections.
Tests: execution_plan_test, decode_service_test, resident identity checks,
frontend lifecycle/stale-terminal cases and the offline roundtrip fixture.
ScrambledEggsOnToast pushed a commit to ScrambledEggsOnToast/Strata that referenced this pull request Oct 19, 2026
Niko1221#259: lookup is conditional - it executes in a solo serial serving window
after the first window, not in resident decode or the BGEN single
admission window. Both earlier flat variants were too broad.

Niko1221#264: preserve the cursor's position within a request's weighted ticket
block across ready-set churn, so membership changes cannot starve a
continuously ready background request. Add the churn regression.

Niko1221#265: conversation_snapshot_save's internal synchronize failure is
classified as a device fault by its error reason, so the capture is not
silently dropped as durability-only.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants