feat(decision): add optional isolated scoring and reference benchmark - #592
Merged
Merged
Conversation
Expose numeric decisions without making consumers own raw-logit session state. Each handle shares weights but owns resettable KV/SSM state and creation-sized candidate storage. Keep default-off, linkable unsupported stubs; a build stamp and CI gate verify both modes in the same directories. DENSE preserves model softcaps and defines candidate-conditional probabilities separately from calibrated confidence. Add public consumer, adversarial and hybrid isolation tests plus a hashed reference protocol. Apple M1 Max / cpu_neon / 6 threads, single five-token arithmetic smoke: Bonsai 2 27B median decision 2.974 s, one-token generation 2.880 s, 16-token generation 5.642 s. This is a reference baseline, not a Jev or quality-matched speedup claim; selected-row work remains #586. Refs #583. Closes #585.
geisten
added
the
decision-inference
Linux targets enable -ffast-math, so GCC removed the isfinite guard and accepted NaN/Inf selected logits. Disable finite-only assumptions for the numeric decision TU, benchmark reference and adversarial test; custom enabled builds with finite-only assumptions fail compilation. Existing inference kernels keep their optimization flags. GCC 16 runtime tests now reject all three non-finite cases, feature on/off release and sanitizer tests pass, and optimized macOS decision code is byte-for-byte the same disassembly as the smoke baseline.
Strict fixture legs reject any unlisted skip. The optional real-model consumer intentionally skips with DECISION=0, so declare that build condition. Enabled and disabled public/hybrid tests remain mandatory through test-decision in release and sanitizer CI.
Keep the benchmark parser's pure size output first, initialize it on failure, and mark status/size helpers nodiscard per AGENT.md. The parser family is migrated together; no public signature or inference behavior changes.
On Intel macOS, decision.o rebuilt with DECISION=0 within the archive's one-second timestamp window. Make skipped archiving/linking and the capability witness detected an enabled stale binary. Force those targets only when the saved flag differs, and keep phony prerequisites out of the archive's object list. Unchanged-flag builds remain incremental.
8 tasks done
This was referenced Oct 4, 2026
geisten
marked this pull request as ready for review
October 4, 2026 10:34
This was referenced Oct 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Consumers currently have to manage a generation session and raw logits to score alternatives. This adds an isolated numeric decision handle with explicit candidate semantics, reset, ownership and failure behavior.
geist_decision.h; default-offDECISION=1, linkable unsupported stubs, and a content stamp plus forced transitions for reliable on/off builds with coarse timestamps.Validation on Apple M1 Max:
-O3 -cchecks (GCC 15 is not installed) and GCC runtime adversarial tests under fast math. The numeric TUs disable finite-only assumptions; enabled custom builds cannot silently remove NaN/Inf checks. All 23 executed CI checks pass on head69e91ca(coverage is intentionally skipped for draft PRs).One arithmetic smoke case, CPU NEON / six threads / five prompt tokens / four candidates / FP32 KV / concurrency one / five measured repeats:
Protocol and limits, all raw samples and hashes. These are workload-specific smoke measurements with uncontrolled caches, not representative task accuracy or Jev parity. Generation quality here measures only its first emitted token; final-answer quality needs downstream evaluation.
Closes #585. Part of #583. Output-row optimization continues in #586; representative Bonsai quality/calibration/latency evaluation in #587.