Skip to content

Three things about a rule that can be measured rather than declared (#825) - #1170

Merged
Rafael-SOWNet merged 1 commit into
masterfrom
rule-definedness-and-cost
Sep 5, 2026
Merged

Rafael-SOWNet merged 1 commit into
masterfrom
rule-definedness-and-cost

Conversation

@Rafael-SOWNet

Copy link
Copy Markdown
Member

Whether a rule ever fires, whether it changes where its result has a value, and which way it moves
the cost the simplifier selects by.

Each could be a field beside RewriteRuleGrowth, and each would then be one more thing to write down
wrongly. Measuring costs nothing at run time — none of it runs in production — and it cannot go stale
against the rules it describes.

The definedness check finds one

k - k = 0 is declared Sound and assumes k has a value. At x = 0, x^(-2) - x^(-2) is
undefined and 0 is not, so the rewrite invents an answer. Its sibling for the same shape,
a / a = 1, attaches provided a is not zero; this one attaches nothing. Filed as
#1169, pinned here as the one known
exception so a new one fails and so does this one going away.

This is contract obligation O4, which is otherwise only as true as whoever wrote the rule
remembering the Provided.

Asked only of Sound rules, because that tier is what claims nothing is assumed.
SoundUnderAssumptions already says otherwise — a / (b / c) = a * c / b turning NaN into 0 at
c = 0 is a rule saying so, not a defect. Without that scoping the check reported 68 rewrites,
every one of them a declaration being read back.

Two things had to be right before it could see anything

Both were wrong on the first pass, and the check was silently vacuous:

  • Zero has to be among the check points. Arithmetic definedness changes where a denominator
    vanishes, a logarithm reaches its pole, a root turns negative. Four comfortable reals ran the whole
    check without ever asking the question it exists to ask.
  • NaN is a Number. Testing Evaled is Number calls the undefined case defined. Validated
    by removing the Provided from a / a = 1 — the check did not bite until it asked for a
    finite number.

The cost check says something growth cannot

growth counts nodes; candidate selection uses MathS.Settings.ComplexityCriteria. A rule can
shrink a tree and raise the number that decides whether its answer is taken.

Measured, they agree: rules declared Collects lower the cost far more often than they raise it.
Reported as a comparison rather than asserted to be perfect, because the two measures are allowed to
disagree and where they do it is a fact about the rule rather than a defect.

Hit rate

123 of 324 rules fire on this corpus, asserted as a count that moves. A rule the corpus never fires
is one the checks above say nothing about.

Verification

Full suite 9557 passed, 0 failed, 14 skipped.

Part of #825.

Whether it ever fires, whether it changes where its result has a value, and
which way it moves the cost the simplifier selects by. Each could have been a
field beside RewriteRuleGrowth and each would then be one more thing to write
down wrongly; measuring costs nothing at run time and cannot go stale against
the rules it describes.

The definedness check is contract obligation O4, and it finds one: `k - k = 0`
is declared Sound and assumes k has a value. At x = 0, `x^(-2) - x^(-2)` is
undefined and 0 is not, so the rewrite invents an answer. Its sibling for the
same shape, `a / a = 1`, attaches `provided a is not zero`; this one attaches
nothing. Filed as #1169 and pinned here as the one known exception, so a new one
fails and so does this one going away.

Asked only of Sound rules, because that tier is what claims nothing is assumed.
SoundUnderAssumptions already says otherwise, and `a / (b / c) = a * c / b`
turning NaN into 0 at c = 0 is a rule saying so rather than a defect. Without
that scoping the check reported sixty-eight rewrites, every one of them a
declaration being re-read back.

Two things had to be right before it could see anything, and both were wrong
first. Zero has to be among the check points, since arithmetic definedness
changes where a denominator vanishes and four comfortable reals never ask the
question. And NaN is a Number, so testing `Evaled is Number` calls the undefined
case defined -- removing the Provided from `a / a = 1` did not trip the check
until it asked for a finite number. It was run with that rule broken to make
sure it bites.

The cost check is the one that says something growth cannot: growth counts
nodes, and candidate selection uses ComplexityCriteria, so a rule can shrink a
tree and raise the number that decides whether its answer is taken. Measured,
they agree -- rules declared Collects lower the cost far more often than they
raise it -- which is worth knowing rather than assuming, and is reported as a
comparison rather than asserted to be perfect.

Full suite 9557 passed, 0 failed.

Part of #825.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura
@Rafael-SOWNet
Rafael-SOWNet merged commit 8cb3afe into master Sep 5, 2026
27 checks passed
@Rafael-SOWNet
Rafael-SOWNet deleted the rule-definedness-and-cost branch September 5, 2026 11:26
Rafael-SOWNet added a commit that referenced this pull request Sep 5, 2026
Four rules bring a negative factor out of a numerator or a denominator.
Given a magnitude of 1 they took it out anyway, leaving the 1 behind as a
literal `1 *` factor, and the numerator rule and the denominator rule then
undid each other around it: `-x / (-y)` grew by four nodes a pass for ever.
They now decline a factor of -1, which is not a factor to take out but the
sign the rewrite is about, and the two rules directly above them already
answer that case. Closes #1167.

Both spellings changed together -- the four MatchedRules and the eight
switch arms in Patterns.Common.cs that carry the same rules with their
commutative variants.

The corpus that had to find it is the second half of this. RuleSetsDoNotCycleTest
ran on hand-written seeds -- shapes somebody thought of -- and `-x / (-y)` was
in that list only because #1056 had already put it there. It now also generates
the arithmetic grammar the growth check uses: nine leaves, seven unary and five
binary shapes, composed two and three deep and sampled at the third level.

It found the next one on its first run, twice, and #1171 has it. `Power`
contains an inverse pair -- `two-powers-of-one-exponent-share-a-base` collects
`a ^ b * c ^ b` into `(a * c) ^ b` and `positive-power-of-a-product-distributes`
takes it straight back -- so applying the set to a fixed point alternates for
ever on `1 ^ (-2) * x ^ (-2)` and on `(-2) ^ 2 * (1/2) ^ 2`. Neither rule is
wrong and each is wanted in its own direction; a *set* holding both has no
normal form to reach, and what to do about that is a design question rather
than a fix, so both loops are pinned by name here with the issue.

Simplify is unaffected by that pair and it was measured rather than assumed:
`1 ^ (-2) * x ^ (-2)` answers `1 / x ^ 2` and `(-2) ^ 2 * (1/2) ^ 2` answers
`1`. The pipeline folds between passes, so `1 * x` collapses and the shape the
second rule needs stops existing. What makes the cycle visible is applying the
set alone, which is what the transformation layer lets a caller do.

Three recorded verdicts elsewhere moved, and each moved because the fix worked:

- RuleSetTerminationTest listed NumericNeat among the sets that settle only
  once the normalisation runs between passes, for exactly this reason -- its
  remark named the product of ones. It settles alone now and has left the list.
- RulePriorityTest recorded two conflicts left to declaration order in
  NumericNeat, one of them the numerator-and-denominator pair itself. Both are
  gone; four rules that decline -1 no longer fire on one node together.
- Its corpus had `-1` as its only negative leaf, which is the one case those
  rules now exclude, so they matched nothing at all and the subsumption claims
  about them went unwitnessed. `-2` is a leaf now. Measured four ways: on the
  old leaves the guard cost 24 witnesses, 513 to 489; with `-2` present it costs
  nothing, 501 either way, which is what says the leaf restores exactly what the
  guard removed. The rest of 513 to 501 is not coverage -- level3 samples every
  eleventh element of level2, so a longer level2 lands the sample elsewhere.

The corpus size is asserted alongside the witnessed count now. A coverage
number means nothing without what it was measured over, and both figures were
prose in a remark that nothing held to them.

Full suite 9558 passed, 0 failed, measured after rebasing onto #1170.


Claude-Session: https://claude.ai/code/session_012sonx8iAspMiwRwokT1Ura

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant