The integrator stops answering the same question over and over (#1156)

[Rafael-SOWNet]

Simplify of an integral it cannot evaluate took a minute and a half. It now takes about a second.

	master	now
sin(x)/(x^2 + 1)^2	114 851 ms	1163 ms	99×
e^x/(x^2 + 1)^2	85 683 ms	1517 ms	56×
ln(x)/(x^2 + 1)^2	51 072 ms	188 ms	272×
1/(x^4 + 1)^2	3811 ms	9 ms	423×

A single cold Integrate, in a fresh process, is faster too — on all nine tried:

	master	now
sqrt(tan(x))	154.5 ms	79.1 ms
x^2*sin(x)	21.0 ms	10.2 ms
1/(x^3 + 1)	49.8 ms	31.2 ms
x^2/(x^2 + 2)^2	17.4 ms	12.0 ms

The integrator was asked the same question 500 times

Tracing every entry to ComputeIndefiniteIntegral on sin(x)/(x^2 + 1)^2, which has no elementary
antiderivative and so runs the search to exhaustion:

top-level integrate calls        562
distinct top-level integrands      3

    500  sin(x) * 1 / (1 + x ^ 2) ^ 2
     50  sin(x) / (x ^ 2 + 1) ^ 2
     12  1 / (1 + x ^ 2) ^ 2 * sin(x)

Those three are one integral written three ways. And inside a single call there is a second layer
of the same thing — 5330 entries for 23 distinct integrands on e^x/(x^2 + 1)^2 — because the
solvers overlap: substitution, partial fractions, splitting a sum and by parts each decompose the
integrand differently and produce pieces that coincide.

So the answers are kept. The whole difficulty is knowing when to stop trusting them.

A settings state, not a change count

An answer depends on the ambient settings — Codomain decides whether the logarithms carry an
abs, MaxExpansionTermCount bounds the by-parts recursion — and a setting is scoped to a flow
rather than a thread, so "nothing has changed" is not something a per-thread cache may assume.

I built the counter version first, and it gave back almost none of the improvement. A scope
opened and closed leaves every setting exactly as it found it while moving a counter twice, and the
library opens such scopes constantly: one Simplify of one unevaluated integral opened 3520 of
them, every one from numeric downcasting in Number/Operators.cs. A counter therefore reports
"everything has changed" continuously and is useless here. That is measured, not predicted — it is
why SettingsState compares state.

The state of a setting is the reference to the frame on top of its stack, which is exact rather
than a hash: releasing a scope restores the very frame object that was there before it, since the
chain below a pushed frame is never rebuilt. So a comparison is a handful of reference comparisons
with no possibility of collision, a balanced open-and-close compares equal, and no enumeration of
which settings matter is needed — which would have been the fragile way to do this.

Reads are lock-free: registering happens while the settings are being constructed, and the reader is
on the integrator's hot path.

The cap

Emptied at 4096 entries. Nothing in it expires on its own — the settings holding still is the entire
condition for keeping an answer — so without a cap a long-lived process integrating a stream of
different expressions would hold every one of them for ever. The worst measured need is 23 distinct
integrands within a call.

Verification

Full suite 9550 passed, 0 failed, 14 skipped. 9 new tests.

The three that matter are the codomain ones, because a cache like this does not fail by
returning nonsense — it fails by returning a correct answer worked out under settings the caller is
no longer standing in. 1/x integrates to ln(abs(x)) on the reals and ln(x) on the complex
plane, so it separates them in one expression; it is asked in both orders, and with a scope entered
while an answer for the same integrand is already held.

Those three were verified to bite: with the state comparison disabled they fail, and with it they
pass. A test that passes either way would have proved nothing.

There is also a guard that sin(x)/(x^2 + 1)^2 finishes under 30 s — two orders of magnitude above
what it now takes and a quarter of what it used to, so it catches the regression without being a
benchmark that flakes.

What this replaced

Three earlier explanations of the same timings, each abandoned when measured:

• Unbounded growth in integration by parts — the cause originally written into Simplify spends up to 90s on an integral it cannot evaluate, asking the integrator the same question 500 times #1156. By parts
  never recurses into itself; both entry points pass integrateByParts: false downward.
• Caching within one top-level call. Collapses the 5330 to 23 and does not help, because it is
  the 562 that carry the cost.
• "Most of the time is outside the integrator." An artefact: the tracing that established it was
  doing 5330 stderr writes per call.

Closes #1156.