Skip to content

The canonical-form and arm-order checkers run in CI, gated on their findings - #1653

Merged
Rafael-SOWNet merged 2 commits into
masterfrom
canoncheck-and-confluence-gate-on-their-lists
Oct 1, 2026
Merged

Rafael-SOWNet merged 2 commits into
masterfrom
canoncheck-and-confluence-gate-on-their-lists

Conversation

@Rafael-SOWNet

@Rafael-SOWNet Rafael-SOWNet commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Part of #1256: the last two of its self-contained harnesses.

CanonCheck and Confluence report findings by design. InnerSimplified is a normalisation and does not reorder operands, so 2024 of its 2738 order checks fail. Three pairs of arms in Common and Power disagree where both fire. A gate that fails on a non-empty list would always be red.

So each commits its list of findings beside it, the way corpus-baseline.tsv does, and fails when the list changes in either direction. The list is the gate, and the counts can move freely.

harness list a line is run
CanonCheck 2091 stage, property, and the case written out in full 1.6 s
Confluence 3 the set and its two arms, by rule name 0.8 s

Arms are named by their rule names, such as a-variable-times-a-number-puts-the-number-first, rather than by index. A name survives an arm being inserted above it.

  • HARNESS_UPDATE_BASELINE=1 rewrites a list. The failure message says so, and lists what appeared and what went away.
  • A finding whose case timed out counts as unmeasured, not gone. A timeout still fails nothing.
  • The shared code is Shared/Baseline.cs.

Checked:

  • Each list is the same across three processes.
  • File-level control, both harnesses, both directions. I removed one committed line and the run exited 1 with "1 appeared". I added a line that cannot occur and the run exited 1 with "1 went away". With the list restored, both exit 0.
  • Library-level control, Confluence only. I made a-variable-times-a-number-puts-the-number-first hand its node back unchanged. Confluence exited 1 and named the Common pair that went away. CanonCheck's list was unchanged, because no finding of its involves that arm. Reverted.
  • Not exercised: the timeout path. No case timed out in any run.

AGENTS.md's two mentions of the harnesses in work/ now point to Sources/Tests/Harnesses. So do five code comments and CanonicalForm.md. Its section 7 had listed three candidate forms where the harness measures four. The catalogue's count of idempotence failures for the sort alone was 21, and it is 18 on master. BREAKING-CHANGES.md keeps its two mentions, which record where a change was found at the time.

After this, #1256's remaining harnesses each need something outside the repository:

  • docsamples needs the wiki and the site;
  • intbench needs the Rubi suite;
  • libcompare needs two other libraries from NuGet;
  • egraph is a measurement rather than a check.

🤖 Generated with Claude Code

https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura

…indings

CanonCheck and Confluence report findings by design. InnerSimplified is a normalisation and does
not reorder operands, and three pairs of arms in Common and Power disagree where both fire. Neither
can fail on a non-empty list. Each now commits its list of findings beside it, as
corpus-baseline.tsv does, and fails when the list changes in either direction.
HARNESS_UPDATE_BASELINE=1 rewrites the list. A finding whose case timed out counts as unmeasured,
not gone, so a timeout still fails nothing.

The lists hold 2091 lines for canoncheck and 3 for confluence. A canoncheck line names the stage,
the property and the case, written out in full. A confluence line names the set and the two arms
by their rule names, which survive an arm being inserted above them.

AGENTS.md's two mentions of the harnesses in work/ now point here.

Part of #1256.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura
@Rafael-SOWNet Rafael-SOWNet added this to the 2.6.0 milestone Oct 1, 2026
Five comments and CanonicalForm.md pointed at work/canoncheck, work/rulecheck and
work/simpsweep, which are Sources/Tests/Harnesses now. CanonicalForm.md's section 7 also listed
three candidate forms where the harness measures four, and now says how CI gates it. The
catalogue's count of idempotence failures for the sort alone is 18 on master, not 21, and the
comment now points to the list that holds every one.

BREAKING-CHANGES.md keeps its two mentions: they say where a change was found at the time.

Part of #1256.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura
@Rafael-SOWNet
Rafael-SOWNet merged commit 71eefc9 into master Oct 1, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant