feat(options): move model hyperparameters onto their Options classes (#2090)

[ooples]

Progress on #2090: model Options classes exist, but models do not read them — tunable values live in constructor parameters instead.

This is a draft opened mid-flight so CI runs and the work is reviewable as it lands. It is not complete; see Remaining work below.

Ratchets

	Session start	Now	Floor
Baseline (name-credited constructor params)	350	0	0
ConstructorBaseline (strict constructor params)	461	0	0
UnreadBaseline (declared-but-unread properties)	3 (flattering)	3	0
AgreementBaseline (constructors disagreeing about an option)	—	4	4
DivergenceBaseline (a model's two implementations reading different options)	—	49	0
UncoveredBaseline (options classes with no Validate())	14	81	—

The "floor of 49" this table carried for several phases was wrong. It was an estimate of
how many parameters legitimately have no default and so must stay parameters, and the live
measurement went straight past it. The floor is 0 for both constructor ratchets.

Both constructor ratchets are now at 0, and two defect forms that previously had no detector
at all now have one. That does not close #2090.

AgreementBaseline's floor is 4, not 0, and the four are named in the guard: DropRate = 0.0
in the ONNX inference constructors of Mask2Former, MixedQueryTransformer, OneFormer and
XDecoder. Dropout must be off at inference, so driving that count to zero would introduce
dropout into inference in four models while appearing to improve consistency. It credits a name match, so it is satisfied by
a property existing and says nothing about whether the constructor reads it. See Remaining
work.

The unread ratchet rose because the detector twice stopped deceiving itself, not because anything regressed — see The detector was measuring the wrong thing, twice below.

What landed

Reservoir and volumetric models (79b8695d9b) — EchoStateNetwork, LiquidStateMachine, VoxelCNN, UNet3D onto two new family bases. Reservoir size deliberately stays a constructor argument: it has no default on either model, so moving it would mean inventing a published value that does not exist.

Speaker recognition (be8761d34b) — SpeakerEmbeddingExtractor and SpeakerVerifier onto a new SpeakerRecognitionOptions base. Three defect forms closed at once, each with a different correct disposition:

• FftSize/HopLength/NumMfcc were declared but never reached the MFCC front end, which always built from its own defaults — so setting NumMfcc changed the first layer's width while the front end kept emitting 40 coefficients. The two halves of the model could silently disagree.
• SpeakerVerifierOptions was referenced by nothing but the clone registry. Its VerificationThreshold = 0.7 was never applied; the model hardcoded 0.6. Both thresholds now default to 0.6 — the value actually in force — and both are read.
• numHeads was deleted, not moved. x-vector (Snyder et al. 2018) is a TDNN with statistics pooling and has no attention; CreateDefaultSpeakerEmbeddingLayers has no head count to pass it to. Moving it would have preserved exactly the false configurability this issue exists to remove.

SpeechEmotionRecognizer (8b045d7153) — 11 shadowing parameters and no options class at all. Its two constructors also disagreed about the model: the ONNX path hardcoded 4 convolutional blocks while the native path used 3 and cited the paper for three, and those fields are read back by GetModelMetadata.

Forecaster transformers (ba7ad0353a, 9cbb619918) — FEDformer, PatchTST, ITransformer, native and ONNX constructors, plus FinancialModelFactory, which was discarding AutoML's tuned parameters for PatchTST and ITransformer. The purest form of the defect: the options classes were already complete with identical defaults, and each constructor stored the options and then read every value from a shadowing parameter.

Neural fields (eaa56ae534, d72686da53) — NeRF, and InstantNGP whose parallel scalar convenience overload was deleted outright rather than migrated: it existed only to build an options object and chain.

The document family, complete (b83f72ffd3 … 7aab354dbb) — ABINet, TrOCR, DocBank, PSENet, DBNet, EAST, CRNN, TableTransformer. Four had a bare new AdamOptimizer<T, Tensor<T>, Tensor<T>>(this), so no configured learning rate could reach the model at all; each options class gained a LearningRate defaulting to Adam's own 1e-3 so behaviour is unchanged. EAST's geometryType was a string driving geometryType == "QUAD" ? 8 : 5 — a typo silently built the wrong-shaped geometry head, with no error anywhere. It is now EASTGeometryType.

VisionTransformer (04a5ee316a) — hiddenDim, numLayers, numHeads, mlpDim moved; imageHeight/imageWidth/channels/patchSize/numClasses deliberately kept as parameters, having no defaults to move.

UNet3D: real skip connections (9428b29933) — see below.

The segmentation family, 62 models (8fbd993835) — the largest single step: numClasses, dropRate and (for 32 of them) modelSize, against a new SegmentationModelOptions : ModelHyperparameterOptions. Both ratchets fell by the same 156, which is itself the finding — not one of those 62 options classes declared a property of a matching name, so none had ever drawn name credit. ModelSize is deliberately not hoisted: each model names its variants with its own enum, so there is no common type to declare. 376 tests pass across the segmentation suites, 0 failures.

The unread-options detector (d914534260) — see below.

The final tail, 29 model types (bb0ae367e3) — NeuralNetworks, PhysicsInformed, Document
and UncertaintyQuantification, plus the duplicate-clipping cleanup the re-parenting exposed.
Both constructor ratchets converged here, which resolves the 21-point spread the ratchet's
own remarks had recorded as probable: those parameters were ones whose options property
already existed while the constructor still took them — the UnifiedMultimodalNetwork shape —
not a disagreement between the two detectors.

Five defects found while doing it, each of which would have survived the migration silently:

• A ?? after a ??= un-does it. _options = options ?? new XOptions() placed after
  options ??= new XOptions() re-tests the parameter, and Roslyn joins both branches of the
  ??, so options becomes possibly-null again. That produced 168 CS8602 across ~20 files
  whose coalesce was demonstrably already present at the top of the body — so the errors looked
  exactly like the coalesce had never been applied. 69 residual ?? sites removed.
• SVTROptions' copy constructor did not copy Charset, so the defensive copy silently
  discarded a user-set charset while copying all 18 other properties.
• HyperbolicNeuralNetworkOptions.Validate() could never pass. It called
  Require(Curvature), which demands a value greater than zero — but hyperbolic space is by
  definition negatively curved, so the published default of -1.0 failed its own validation.
  Range-checked instead. Its doc example was itself a Epic: model Options classes expose no hyperparameters — 285 of 1,670 are empty #2090 defect three times over: it set
  Curvature = 1.0 (which throws), named two properties the class does not declare, and called
  a constructor overload that does not exist.
• Four bare optimizers. TabR, TabM and GANDALF each built a bare AdamOptimizer, so their
  declared WeightDecay reached nothing; Finch built no optimizer at all, leaving
  LearningRate, Beta1, Beta2 and WeightDecay all inert. Now AdamW, whose learning-rate
  default is identical to Adam's, so the swap adds the decay and changes nothing else.
• Clone() dropped the inherited MaxGradNorm on TabR, TabM, TabNet and FTTransformer.
  Survivable while each class had its own duplicate shadowing it; a live defect the moment the
  duplicate went — the copy-constructor hazard this branch already documents, reached from the
  opposite direction.

EnableGradientClipping and MaxGradientNorm were deleted from five options classes rather
than wired: MaxGradNorm on the shared base already documents zero-or-negative as "off", so the
separate boolean expressed no state the single double could not. TabNet's published bound of 2.0
moved onto MaxGradNorm in a new parameterless constructor rather than being dropped with the
duplicate.

49 test call sites moved from named constructor arguments to options object initializers, driven
from each constructor's accepted-parameter set rather than from the compiler's error list — the
compiler reports only the first offending argument per call, so the error list would have
needed one build per argument.

Two ratchet history blocks were also found sitting outside their </remarks>, which this
repo's test project compiles silently.

Concerto self-supervised pretraining (8178f54a6a) — the 10 ConcertoOptions properties
configured a pretraining pipeline whose parts all existed (two objectives, a paired-view type, a
teacher EMA) and referenced only each other, with no orchestrator. A closed ring of
mutually-referencing types is the signature of an unimplemented subsystem, which makes the
decision implement or remove a feature, not wire a property.

Both objectives computed in raw double and returned a bare scalar, so they carried no
gradient graph — a Pretrain built on them as written would have run, reported a falling
number, and left every weight untouched. They were reimplemented in engine operations first,
per this repo's contract that a loss defines its forward math once and GradientTape supplies
the backward. The test asserts the loss falls and the parameters move; a "does it run"
assertion passes against the broken version, which is why it is not the one used.

The audio cluster, 11 properties (8178f54a6a) — ReturnTimestamps now lives once on
AudioNeuralNetworkOptions with a resolver on the shared base. All 104 ISpeechRecognizer
implementations already branched on the flag, but it arrived only as a method parameter
defaulting to false, so no model could be configured to return timestamps. Declaring it on the
shared base was the measured choice: wiring only Whisper would have left 103 models with a
?? false that does nothing. OnnxOptions fell back to new OnnxModelOptions() rather than the
declared property — AudioGen passed no options to any of its three ONNX sessions at all. Six
ONNX model-path properties were deleted: each duplicates a required constructor parameter,
and a file the caller must supply is an input rather than a tunable.

Defect form (4), 129 fields across 33 files (8178f54a6a) — one model whose two
constructors describe it differently. The ONNX constructors take an options parameter and then
assign hardcoded literals to the same backing fields their native sibling reads from options, so
a caller configuring that path had their settings silently discarded. ConvTasNet's ONNX
constructor hardcoded 128/512/8/3/3 while its native one read all five from options. Every
literal was compared against its declared default first: 128 agreed and were swapped
behaviour-preservingly, and the four that did not are the deliberate inference cases above.
Every swapped line was also checked to read _options after it is assigned in that
constructor — a rewrite that puts the read above the assignment compiles clean and dereferences
null.

The last 21 constructor parameters (88e21d19b8) — 13 model types, closing the strict
measure. They reduced to exactly two shapes, both of which read as working configurability:

• Four probabilistic forecasters (CSDI, DiffusionTS, ScoreGrad, TSDiff) reconciled parameter
  and options with _numFeatures = numFeatures > 0 ? numFeatures : _options.NumFeatures; — so
  the options value applied only when a caller passed zero or less, and the parameter's own
  default of 1 shadowed it for everyone else. Each type's other constructor already read the
  options directly; the two now agree.
• Six models (DGCNN, PointNet, PointNetPlusPlus, GaussianSplatting, MeshCNN, SpiralNet) had
  convenience constructors forwarding scalars into an options object initializer, applying the
  parameter copy last.

All 11 defaults in the second shape were checked against their options property and matched, so
the removals change no behaviour. Parameters with no default (numClasses, samplingRates,
searchRadii, mlpDimensions) and collaborators stayed: a parameter with no default is a
required input, not a duplicated value. CRAFT separately inherited ImageSize/
BackboneChannels but left both unset, so its options object reported 0 while the model ran at
768/512.

SGPT was the last model carrying [ModelDimensionRole], and TestScaffoldGenerator emitted
its embedding-dimension / head-count divisibility test purely from those constructor parameters,
returning early on zero candidates — so migrating SGPT would have switched the feature off
silently. The attribute is retargeted to properties and the generator now reads values off a
default-constructed options instance, which is stronger than what it replaces: a parameter
default is a compile-time constant, whereas this exercises whatever the parameterless
constructor actually assigns.

That retarget caused a regression the same commit fixes, and it is the most transferable lesson
here: the properties are declared on EmbeddingModelOptions, so every descendant inherited
the annotation — including Word2Vec, GloVe and FastText, which have no attention, never assign
NumHeads, and failed a divisibility check against 0. An attribute that states a role is safe
to inherit from a family base; one a generator uses to assert an invariant is not. Candidates
are now gated on TransformerEmbeddingOptions, whose seven descendants are exactly the models
the invariant covers — still 7× the single-model coverage the feature had before. Skipping types
whose NumHeads is 0 was rejected: it passes, but makes the test vacuous for any attention model
that forgot to set it, which is the defect worth catching.

Two bugs this work exposed

Both were latent and unreachable because the options went unread. Neither was caused by the migration — each was confirmed by stashing to a clean tree at HEAD and re-running.

UNet3D emitted a bit-identical output for every input. CreateDefaultUNet3DLayers chose its output activation from numClasses alone and ignored architecture.TaskType, putting a bounded (0, 1) sigmoid head on a regression task whose targets span [-1, 1]. Chasing a target of -1 drives the pre-activation past about -88, where the float32 sigmoid underflows to exactly zero for every input. CreateDefaultVoxelCNNLayers, thirty lines above in the same file, already selected on task type — the UNet3D helper was the outlier.

ITransformer threw on non-contiguous tensor views. FinanceModelTestFactory.NormalizeOptions shrinks the options it passes (SequenceLength 8, PredictionHorizon 4, ModelDimension 24), and ITransformer had been ignoring all of it, running at its hardcoded 96/96/512. Once it honours those values the autoregressive path produces sliced views, and six methods reading .Data element-wise throw. TimeMachine in the same folder already guards this with x.IsContiguous ? x : x.Contiguous(); that idiom is applied.

UNet3D had no skip connections at all

CreateDefaultUNet3DLayers documented them and did not build them ("This implementation does NOT actually perform the concatenation"), so the decoder never saw encoder detail and the memorization test plateaued at exactly the variance of its target — the loss a constant predictor achieves. A 10× learning-rate probe moved it from 0.335760 to 0.335760, identical to six decimals over 26 steps, which ruled out slow optimization.

This was twice reported as blocked on the grounds that IEngine has no ConvTranspose3D. That verdict was wrong both times: the obstacle was real, the conclusion was not. The paper's 2×2×2 up-convolution (Çiçek et al. 2016) halves channels while doubling resolution, and for stride == kernelSize a transposed convolution is exactly a 1×1×1 convolution to C_out · stride³ channels followed by depth-to-space — the sub-pixel identity of Shi et al. 2016. Conv3DTransposeLayer is built from that identity, and the skip path is three coordinated pieces: encoder taps concatenated via Engine.TensorConcatenate, ForwardForTraining => Forward, and a ResolveLazyLayerShapes override that runs one inference-mode forward because the sequential base resolver cannot model a concatenation. 29/29 green.

The detector was measuring the wrong thing, twice

UnreadOptionsRatchetTests asks whether any method in the assembly calls a property's getter. Every one of these options classes carries a copy constructor, and TeacherMomentum = other.TeacherMomentum; is such a call — so having a copy constructor removed a class's entire property set from the detector's reach.

The discriminator is exact: MatryoshkaEmbeddingOptions.MaxEmbeddingDimension was one of the three properties the ratchet could see, and it is visible only because that class has no copy constructor. ConcertoOptions' ten never-read properties were invisible only because it has one.

Ignoring call sites declared on a ModelHyperparameterOptions subclass takes the count from 3 to 35 against 748 declared getters. Nothing became unread; 32 properties that were already unread stopped being hidden. Largest: ConcertoOptions 10 (an entire self-supervised pretraining configuration the model never consults), FinchOptions 6 (Beta1, Beta2, WeightDecay, MinLearningRate and the clipping pair, none of which reach an optimizer), WhisperOptions 5 and AudioGenOptions 4 (ONNX component paths nothing loads).

Second, it could not see generic options classes at all. The scan collected each getter's MethodDef token and compared it against raw IL call operands — but a call on a constructed generic type (CTGANOptions<T>) emits a MemberRef token, which never equals it. The old remark claimed raw comparison was "immune" to generics; it was blind to them, silently. Since nearly every tabular and synthetic-data options class is generic, re-parenting FinancialNeuralNetworkOptions appeared to expose 517 unread properties in one step. It had not: CTGANGenerator reads _options.EmbeddingDimension nine times, MedGANGenerator reads _options.AutoencoderDimensions three, TabPFNNetwork reads _options.EmbeddingDimension five — every one reported unread.

Call tokens now resolve with their enclosing method's generic context and are keyed on (declaring type definition, property name). Unresolvable tokens are counted and fail the test above a tenth of the total, and failures are deliberately not deduplicated — a token can fail under one generic context and resolve under another, so skipping the retry would rebuild the same silent "no" in a new shape.

The measured figure is 112 of 1236 declared getters. It was validated against a control rather than by looking better: ConcertoOptions is not generic and its ten unread properties were confirmed by grep beforehand, so it had to stay at ten — and did — while the suspected generic false positives moved. CTGANOptions now reports exactly one unread member, Epochs, which is precisely the one its model never reads.

Also fixed here, unrelated but blocking

SQLitePCLRaw.lib.e_sqlite3 2.1.12 dropped the win-arm (ARM32) runtime but its own buildTransitive/net461 targets still declare runtimes\win-arm\native\e_sqlite3.dll as Content, so every .NET Framework build fails MSB3030 on a file the package does not contain (2.1.11 shipped it). The stale item is removed rather than pinning the native SQLite binary back a version. It has to be removed in both csproj files, because the package's buildTransitive targets flow into every project downstream of it — fixing only the package owner left the test project failing identically.

Corrections to earlier commit messages on this branch

Two commits from earlier work carry claims that are not true, recorded here so review does not rely on them:

• 6f492735c claims it drops 8 duplicate _options.Validate() calls. It does not — that landed in 75f2ff420.
• f0c8a7e98 claims no other unread-LearningRate instance existed. 63 did; the search behind that claim used [^>]*, which cannot match a C# generic type argument.
• d1b942f951 (this branch, today) sets UnreadBaseline = 517 and states that "482 of 488 newly-visible properties are read by NOBODY". Both are wrong, and 78110b8974 corrects them: the measured figure is 112. 517 was an artefact of this test comparing raw metadata tokens, which cannot match a call on a constructed generic type — so every generic options class reported as wholly unconsumed. CTGANGenerator reads _options.EmbeddingDimension nine times while the ratchet called it unread. Note the recurring shape: this is the same generic-type-argument blind spot as f0c8a7e98 above, in a different tool. It has now defeated four separate scans in this issue.

Remaining work

The live ReportRemainingGaps drives this list, not an earlier hand-written backlog — the two had diverged badly.

1. 3 declared-but-unread properties, down from 112 measured at the start of this branch.
   What is left is not a backlog of wirings — each of the three needs a subsystem built, and
   each is named at the ratchet constant with what that would take:
   • TabNetOptions.EnablePreTraining / PreTrainingMaskingRatio — TabNet's self-supervised
     stage needs a decoder, and this implementation has none: encoder, attentive
     transformers and one output layer are all that exist.
   • SpikingNeuralNetworkOptions.StdpWindow — attempted and backed out, deliberately. The
     rule and the spike histories are both present and an entry point joining them is easy;
     where to WRITE the result is not. The supervised path indexes a layer's weights as
     post * preSize + pre into GetParameters, but a SpikingLayer stack reports 204
     parameters where synapses and biases account for ~80, and running a simulation moves that
     vector by 1.5 with no weight update applied. The vector carries neuron-model and membrane
     state alongside the synapses, so a flat index writes STDP deltas into the neuron model,
     silently. The unblock is the addressing, not the rule — and the supervised path makes the
     same assumption and should be checked with it.
2. Paper fidelity — docs/model-paper-defaults.tsv, a [PaperDefaults] attribute and a
   fidelity test. One discrepancy already logged: DepthAnythingV2 PatchSize is 16 where
   DINOv2's is 14. FidelityBaseline already covers the half of this that needs no external
   source: whether a declared default matches the default its own documentation states.
3. DivergenceBaseline (49) — every tabular model exists twice over one shared options
   class (XNetwork building layers via LayerHelper, XBase building them inline with
   XClassifier/XRegression derived), and the two read different options. The guard was
   written to assert zero and found 49, so it became a ratchet: curating an exclusion list until
   it passed would have hidden the finding and grown quietly in its place. It earned its keep
   this phase — wiring Mambular's DeltaMin/DeltaMax into MambularBase alone pushed it
   49 → 51, so MambularNetwork's LayerHelper path was wired too and it returned to 49.
4. UncoveredBaseline (81) is a deliberate consequence rather than a backlog: re-parenting
   onto ModelHyperparameterOptions enrols a class in the Validate()-coverage guard, and that
   guard's own comment records the precedent for raising it rather than adding Validate()
   methods that check nothing.

The invisible surface item that stood here through every earlier phase is now partly closed:
two of its forms are ratcheted. What remains undetected is hardcoded literals shadowing an
option where the model has only ONE constructor, factories discarding tuned parameters, and doc
<example> blocks that could never compile — three of which this branch found only by reading.

What the last phase found beyond the property count

Driving UnreadBaseline 112 → 3 surfaced four defects that are not unread properties, and are
the more interesting half of the result:

• MatryoshkaEmbedding was built 768 wide while documented and tested as 1536. Its options
  assigned 1536 to MaxEmbeddingDimension, which nothing read, leaving the inherited
  EmbeddingDimension at its 768 default — and the model handed the raw nullable options to
  its base while separately defaulting it, so with no options supplied the base built a
  TransformerEmbeddingOptions and the derived class a MatryoshkaEmbeddingOptions: two
  objects, and the one that sizes every layer was the base's. Three MatryoshkaEmbeddingTests
  had been failing on exactly this. Fixed; all 32 pass.
• OCTGAN advertised "WGAN-GP training" with an empty #region Gradient Penalty, keeping
  its critic Lipschitz by weight clipping against a hardcoded GanClip = 0.01. The penalty is
  implemented against the repo's existing double-backprop pattern and the clipping removed,
  because Gulrajani et al. replace clipping with the penalty rather than combining them.
• TimeGANOptions.NumFeatures was public new int, so options.NumFeatures returned 5
  while ((RiskModelOptions<T>)options).NumFeatures returned 10 for the same object.
• The unread scan had a false positive of its own. Its pass-through — a computed property
  carries consumption to what it derives from — was restricted to property getters, so a
  helper method did not carry it: SpeechEmotionRecognizer reads its labels through
  _options.GetEffectiveEmotionLabels() and EmotionLabels was reported unread while changing
  it demonstrably changes the model's ClassLabels. Extending the edge to methods required
  excluding ToString/Equals/GetHashCode/Clone/Validate in the same change, since those
  read every property without consuming any and one outside call would have marked a whole class
  consumed. The measured move was 21 → 20, not 21 → 3, which is the evidence that it did.

Also: the three guards added on this branch had only ever been compiled for net10.0. A full
multi-target build failed with five errors on net471 (double.IsFinite,
string.Replace(_,_,comparison), string.Contains(char, comparison)) — fixed.

🤖 Generated with Claude Code