ci: audit(2026-05): close 15 of 19 audit findings — license, telemetry, security, hygiene, NIEs, coverage, changelog, website, helloworld - #1430
Merged
Conversation
…engine disclosure closes audit findings: #1 (license straggler) — notice file updated from apache 2.0 to bsl 1.1 text; adds federal-use clause pointing to aidotnet.dev/federal-use; matches the already-merged license file (pr #1418) so all licensing surfaces are now internally consistent. #3 (security.md) — replaced personal gmail with admin@aidotnet.dev, added github security advisories as primary intake, response sla (5 day ack, 30/60 day fix per severity), embargo policy (90 days default), cvss 3.1 severity matrix, cve/ghsa issuance policy, supported-version policy (latest minor + one back), scope, coordinated disclosure + credit, federal/regulated adoption paragraph, maintainer escalation paragraph. new docs/security/pgp.txt placeholder with key-generation steps for manual completion. #4 (repo hygiene) — git rm of 15 files at repo root: 3 yolan-temp-path scratchpad jsons, threads.json + threads_temp.json, 3 speedscope traces (16.7 mb total), failed_tests_report.md, codebase_analysis.md (21 kb; claimed version 0.0.5-preview against current 0.204.0), executive_summary .txt, gpu_matmul_analysis.md, implementation_plan_pr430.md, 2 sprint_ plan files. .gitignore extended with prevention patterns: cusers*, scratchpad*, threads*.json, *.speedscope.json, *.nettrace, implementation_plan_*, sprint_plan_*, executive_summary*, *_analysis.md, *-findings.md, *-investigation.md, failed_tests_report.md, *-report.md / *_report.md, *-summary.md / *_summary.md, agent_*.md, fix-*.ps1, debug-*.ps1, temp-*.*, *.tmp. #7 (failed_tests_report.md) — verified all 14 cited tests pass on current head; the report was stale (memory<t> bugs were fixed by subsequent prs). tracking issue #1429 closed as not-planned. file deleted. #11 (internalsvisibleto) — added related-projects section to notice disclosing harmonicengine, harmonicengine.tests, ai dotnet.programsynthesis .tooling as private commercial forks governed by bsl 1.1. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… + patch bump impl + versioning policy update closes audit findings: #16 (codeowners single-owner) — phase 0 portion. all rules still @ooples for now, but the file is now structured by subsystem (license/governance, ci/release, telemetry/license-guard, per-domain model families, builder surface, generators, serving/deployment, website, docs). adding a co-maintainer in phase 6 is now a small targeted edit per subsystem rather than a global owner change. #17 (merge-dev2-to-master in contributing) — contributing.md base branch section now says "use master (trunk-based)" instead of "use merge-dev2-to-master as working base". stale local branches (merge-dev2-to-master + dev2) deleted; origin already pruned those long ago. #18 (patch bumps not implemented) — automated-release.yml bump logic updated: - new patch tier for fix/perf/refactor/docs/chore/style/test/ci/ build/revert: commits (per conventional commits + semver.org) - non-conventional capitalised fallbacks split: add/implement/ create/new still minor (additive), fix/update/improve/enhance/ resolve/patch/correct/repair changed from minor to patch - new case patch: in the bump-application switch versioning.md updated: removed "currently not implemented per project requirements" note, replaced with the full conventional- commits-to-version-bump mapping and a historical-violations warning paragraph for consumers pinning pre-0.205.0 ranges. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…og backfill (210 entries), helloworld iris classifier, 3 new website pages, docs/internal manual-steps, readme aidotnet.dev links, notice + readme 4-year fix closes audit findings: #15 (changelog stale) — scripts/regen-changelog.sh added; ran against gh release list --limit 300 → fetched 210 releases for ooples/ aidotnet. changelog.md now reflects the actual release history (1275 lines) instead of the [unreleased] - 2025-12-18 + [previous changelog entries would appear here] placeholder. script is idempotent and runs in ci on demand. #16 (codeowners) — phase 0 done in prior commit; manual-steps doc added covering the phase-6 backup-maintainer onboarding work. #19 (helloworld doesn't match readme) — samples/getting-started/ helloworld/program.cs rewritten from 91-line xor to a 3-class iris classifier (~75 lines including the inlined dataset). uses the actual neuralnetworkarchitecture + neuralnetwork + aimodelbuilder api, not the aspirational marketing snippet. readme "hello world example" updated to a literal extract of the same code with a link to the full sample. helloworld readme rewritten to describe the iris example. prior xor sample preserved in git history. website work (audit-2026-05): /security new page; mirrors security.md (admin@aidotnet.dev, ghsa, pgp, sla, embargo, supported versions, federal-adoption paragraph) /enterprise new page; describes custom builds (disable_telemetry + disable_license_guard gated by enterprise license), offline license validation, fips 140-3, nist ssdf sbom + slsa l3, dedicated support /federal-use new page; doe / dod / civilian agencies / national labs scope, compliance posture table (ssdf, ai rmf, fips 140-3, fedramp-aligned, cmmc, cui), procurement vehicles (gsa, sbir/sttr, ota, direct), audit trail artifacts available on request /pricing enterprise tier features expanded to include the phase-0 capabilities (custom builds, offline license, air-gapped, fips, ssdf). federal-use callout added below the tier comparison. docs/internal/audit-2026-05-manual-steps.md added documenting what the user has to do outside the pr: dns config, mailbox + pgp key, github security advisories, phase-4 enterprise license signing, stripe payment links, backup maintainer onboarding. notice + readme 3-year → 4-year fix: the bsl license file uses the "fourth anniversary" clause; my prior notice update incorrectly said three years and the README also said three years. corrected both to four years with cross-reference to the license clause. readme links: ooples.github.io/aidotnet/license/ → aidotnet.dev/ license, ooples.github.io/aidotnet/ → aidotnet.dev/docs, ooples. github.io/aidotnet/pricing/ → aidotnet.dev/pricing. preserved the api/ link to ooples.github.io since that's where docfx publishes the api reference. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…to honest exception types
audit reported 18 throw new notimplementedexception sites in src/ as a
"production gap." inspection shows 12 of 18 are switch-default fallbacks
for enum dispatch where the switch exhaustively lists all valid enum
values; the right exception type is argumentoutofrangeexception (the
caller passed an enum value that's not a valid input), not
notimplementedexception (an unimplemented feature). the remaining 6 are
either non-supported-on-base-class methods (notsupportedexception) or
not-currently-supported-feature messages.
12 switch-default fallbacks converted to argumentoutofrangeexception
(lists the valid enum values in the message):
src/knowledgedistillation/strategies/attentiondistillationstrategy.cs:427 (matchingmode)
src/knowledgedistillation/strategies/attentiondistillationstrategy.cs:623 (matchingmode gradient)
src/knowledgedistillation/strategies/factortransferdistillationstrategy.cs:237 (factormode)
src/knowledgedistillation/strategies/neuronselectivitydistillationstrategy.cs:294 (selectivitymetric)
src/knowledgedistillation/strategies/neuronselectivitydistillationstrategy.cs:457 (gradient)
src/knowledgedistillation/strategies/probabilisticdistillationstrategy.cs:227, :415 (probabilisticmode x2)
src/knowledgedistillation/strategies/relationaldistillationstrategy.cs:492 (gradient)
src/knowledgedistillation/strategies/relationaldistillationstrategy.cs:911 (distancemetric)
src/knowledgedistillation/strategies/variationaldistillationstrategy.cs:196 (variationalmode)
src/knowledgedistillation/teachers/ensembleteachermodel.cs:266 (aggregationmode)
src/knowledgedistillation/teachers/onlineteachermodel.cs:189 (updatemode)
1 format-string dispatch (argumentexception):
src/autodiff/tensoroperations.cs:8311 (complex matmul 'format' string)
3 base-class-only methods (notsupportedexception with derive-from
message):
src/hyperparameteroptimization/hyperparameteroptimizerbase.cs:74 (optimizeformodel)
src/optimizers/optimizerbase.cs:1731 (step)
src/optimizers/optimizerbase.cs:1744 (calculateupdate)
2 documented "feature-not-yet-supported" sites (notsupportedexception
with tracking reference):
src/distributedtraining/pipelineparallelmodel.cs:197 (selective/full
recompute strategy — uses interval-based checkpoints only today)
src/physicsinformed/pdes/pdespecificationbase.cs:160 (autodiff-tape
residual — subclasses must override for autodiff support)
verified: zero remaining notimplementedexception in src/. build clean
on net10.0.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ument elasticsearch/seal metapackage extraction
addresses the dep-sprawl audit finding. concrete-usage survey of the
flagged deps showed:
pinecone.client 0 usages in src/ removed from core csproj
elastic.clients 1 usage (rag/documentstores/elasticsearchdocumentstore.cs)
kept for now + comment scheduling metapackage move
microsoft.research.sealnet 1 usage (federatedlearning/cryptography/seal
homomorphicencryptionprovider.cs)
kept for now + comment scheduling metapackage move
npgsql.entityframeworkcore.postgresql used only in aidotnet.serving (already
a separate project — appropriately scoped)
microsoft.data.sqlite used only in aidotnet.serving (already separate)
so the audit complaint about dep sprawl was partially overstated — ef
core / sqlite / postgres are appropriately scoped to the serving subproject
already, and pinecone was a pure dec-no-use. the genuine cases are
elasticsearch and seal (one file each), which want extraction to
aidotnet.storage.elasticsearch and aidotnet.privacy.he metapackages
respectively as a follow-up commit on this branch.
phase 2b done this commit:
- remove pinecone.client packagereference from src/aidotnet.csproj
(kept the directory.packages.props version pin in case it returns)
- replace bare elasticsearch + sealnet packagereferences with versions
annotated with the planned-extraction comment, so a future commit
that moves the single file out can grep for the audit ref + remove
the line cleanly
phase 2b pending (follow-up commits on this branch):
- create src/aidotnet.storage.elasticsearch/aidotnet.storage.elasticsearch.csproj
move elasticsearchdocumentstore.cs to it; keep idocumentstore in core
- create src/aidotnet.privacy.he/aidotnet.privacy.he.csproj
move sealhomomorphicencryptionprovider.cs to it; keep
ihomomorphicencryptionprovider in core
- update aidotnet.sln to include the new projects
- smoke-test downstream consumers (federated-learning + rag tests
that need to install the new metapackages explicitly)
build verified clean after pinecone removal: dotnet build src/aidotnet.
csproj -c release -f net10.0 → 0 errors.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ooples
added
documentation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 2 Skipped Deployments
|
github-actions
Bot
changed the title
Contributor
|
Warning Review limit reached
Your plan currently allows 2 reviews/hour. Refill in 2 minutes and 55 seconds. Your organization has run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After more review capacity refills, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than trial, open-source, and free plans. In all cases, review capacity refills continuously over time. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (35)
WalkthroughAudit-driven PR: broad repository governance, release automation, licensing/security, website/docs and sample updates; added diagnostics/tools; large VisionLanguage and Robotics model implementations; exception/validation replacements; dependency and project config edits; repo hygiene and removed planning artifacts. ChangesAudit 2026-05 Consolidated Changes
Estimated code review effort: Possibly related issues:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Contributor
|
🤖 PR Title Auto-Fixed Your PR title was automatically updated to follow Conventional Commits format. Original title: New title: Detected type: Valid types and their effects:
If the detected type is incorrect, you can manually edit the PR title. |
3 of 4 tasks
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 16
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/VERSIONING.md:
- Around line 54-71: Update the sections that still describe the old MINOR/None
mapping so they match the new PATCH policy: replace any text that maps
fix/perf/refactor/docs/etc. to MINOR (or "None") with the new rule that these
prefixes trigger PATCH bumps (as in the "Triggered by any of the following
Conventional Commits prefixes" paragraph), adjust examples (e.g., version
progression like 0.205.0 → 0.205.1) and any fallback PR-title mappings to state
they map to PATCH, and ensure the "Versioning history note" and any other
explanatory paragraphs consistently reflect that prior behavior has changed and
consumers should pin accordingly.
In @.github/workflows/automated-release.yml:
- Around line 128-136: The workflow's PATCH detection (the if that inspects
COMMITS and sets BUMP_TYPE="patch") is never reached for docs-only pushes
because the workflow trigger excludes markdown/docs paths earlier; update the
workflow trigger or the path filters so docs-only commits will run this workflow
(or remove the docs exclusion), or alternatively change the trigger logic to
always run on push and add an early guard that inspects COMMITS to decide
whether to proceed; ensure you adjust the condition that uses COMMITS and the
variable BUMP_TYPE so commits with "docs:" are evaluated by the existing if
block.
In `@CHANGELOG.md`:
- Around line 8-12: The "Versioning note." paragraph in CHANGELOG.md contains a
broken fragment "see . Consumers" — restore the missing reference after "see" so
the sentence reads complete (for example, link or mention the intended target
such as the Conventional Commits spec, CONTRIBUTING, or relevant file) and
remove the stray line break so the clause is contiguous; update the text around
the "Versioning note." and the fragment "see . Consumers" accordingly to include
the correct reference and punctuation.
In `@CONTRIBUTING.md`:
- Around line 4-7: The "version-impact" / commit-type mapping in the doc no
longer matches the actual release logic used by the semantic-release pipeline;
update the section that maps Conventional Commit types so it reflects current
behavior: map feat -> minor, fix (and corrective types such as perf, docs,
chore, refactor, test, ci, build, audit) -> patch, and treat BREAKING CHANGE or
a trailing "!" as major; update any examples and wording to reference the
Conventional Commits subject as the source of the version bump and remove or
reconcile any legacy-mapping text that contradicts the pipeline.
In `@docs/security/pgp.txt`:
- Line 6: Update the stale docs path reference in the text line that reads
"Manual setup steps (tracked in docs/internal/phase0-manual-steps.md):" so it
points to the current checklist file
"docs/internal/audit-2026-05-manual-steps.md"; locate that exact string in
docs/security/pgp.txt and replace the old path with the new one to ensure
maintainers are directed to the correct manual-steps document.
- Around line 1-5: Replace the placeholder text "PLACEHOLDER — PGP public key
for admin@aidotnet.dev" in the docs/security/pgp.txt file with the actual
ASCII-armored PGP public key block for admin@aidotnet.dev (the full "-----BEGIN
PGP PUBLIC KEY BLOCK-----" ... "-----END PGP PUBLIC KEY BLOCK-----" content),
ensure the block is validly armored with no extra characters or truncated lines,
verify the key matches the contact referenced in SECURITY.md, and commit the
real key so the encrypted-reporting path becomes operational.
In `@NuGet.config`:
- Around line 4-5: The repository NuGet.config contains machine-specific
absolute feed entries "local" and "local-source" (add key="local"
value="C:\Users\cheat\local-nupkg" and add key="local-source"
value="C:\Users\cheat\source\local-nuget") which break CI; remove these entries
from the repo-level NuGet.config and, if needed, move them to a user-level
config (e.g., %APPDATA%\NuGet\NuGet.Config) or document them in CONTRIBUTING.md
so local feeds remain per-developer and CI restores work on non-Windows runners.
In `@samples/getting-started/HelloWorld/README.md`:
- Around line 11-16: The fenced code block in the README containing "===
AiDotNet Hello World: Iris classifier ===" needs a language identifier to
satisfy MD040; update the opening fence from ``` to ```text so the block becomes
a text-marked code fence. Locate the fenced block around that title in README.md
and change the fence marker only (no other content).
- Around line 25-29: The fenced C# code block containing AiModelBuilder<double,
Tensor<double>, Tensor<double>> and the call to ConfigureModel(new
NeuralNetwork<double>(architecture)).BuildAsync(trainX, trainY) should be
surrounded by a blank line above and below (inside the list item) to satisfy
MD031; edit the README.md list item to insert one blank line before the
```csharp fence and one blank line after the closing ``` so the fenced block is
separated from the surrounding list text.
In `@scripts/regen-changelog.sh`:
- Around line 70-97: The generated changelog URL is hardcoded to
"ooples/AiDotNet" (see the print call printing "See
https://github.com/ooples/AiDotNet/releases/tag/{tag}"); change the Python block
to read the repository from the environment (e.g.
os.environ.get("GITHUB_REPOSITORY") or os.environ.get("REPO")), fall back to the
current hardcoded value if neither is set, and use that variable in the f-string
so the printed line becomes "See https://github.com/{repo}/releases/tag/{tag}".
In `@src/DistributedTraining/PipelineParallelModel.cs`:
- Around line 197-202: The constructor of PipelineParallelModel now throws
NotSupportedException instead of NotImplementedException, which breaks tests and
downstream contracts; update any tests that assert the old exception (e.g., the
unit test that instantiates PipelineParallelModel and expects
NotImplementedException) to expect NotSupportedException, and update
release/migration notes to document the behavioral change so external callers
that catch NotImplementedException can be migrated to catch
NotSupportedException (refer to the throw site in PipelineParallelModel
constructor where _checkpointConfig.RecomputeStrategy is checked).
In `@src/KnowledgeDistillation/Teachers/EnsembleTeacherModel.cs`:
- Line 266: The exception message in EnsembleTeacherModel that throws
ArgumentOutOfRangeException for _aggregationMode lists the wrong valid modes;
update the message (the throw using nameof(_aggregationMode) and
_aggregationMode) to enumerate the actual implemented modes (GeometricMean,
WeightedAverage, Median, Maximum) so runtime diagnostics reflect the switch
cases in EnsembleTeacherModel.
In `@src/KnowledgeDistillation/Teachers/OnlineTeacherModel.cs`:
- Line 189: The exception message in the OnlineTeacherModel throw uses the wrong
enum name; update the ArgumentOutOfRangeException thrown for _updateMode to list
the actual enum member names (e.g., EMA instead of ExponentialMovingAverage) so
callers are directed to real values of OnlineUpdateMode; locate the throw that
references _updateMode and change the "Valid modes" text to the exact enum
identifiers (EMA, GradientBased, MomentumBased).
In `@tools/ClusterRepro/Program.cs`:
- Around line 55-58: The loop in Program.cs iterates over network.Layers.Count
without guarding against cloned.Layers.Count, which can throw if the clone has
fewer layers; change the loop in the code that uses network.Layers and
cloned.Layers (the block calling GetParameters on network.Layers[i] and
cloned.Layers[i]) to iterate up to Math.Min(network.Layers.Count,
cloned.Layers.Count) and add an explicit log message when the counts differ so
you still capture diagnostics rather than throwing.
In `@tools/DiffusionPerfDiag/Program.cs`:
- Around line 20-21: The StreamWriter 'log' and the compiled plan object
('plan') are not disposed on early return paths; wrap the program's main
execution so that 'log' and 'plan' are always disposed in a finally block.
Specifically, keep the existing Log(string s) and log variable, but surround the
body that can return (all branches that return 1) with try { ... } finally {
plan?.Dispose(); log?.Dispose(); } so both 'log' and the compiled plan are
deterministically cleaned up on every exit path.
In `@website/src/pages/pricing.astro`:
- Around line 137-142: The federal customer paragraph (<p class="mt-4 text-sm
text-slate-600 dark:text-slate-300 max-w-3xl mx-auto text-center">) is currently
rendered as a fourth grid item so it can get squeezed into the 3-column pricing
grid; move this <p> out of the pricing grid container (place it after the grid
markup) or make it span the full row of the grid by giving it a full-width grid
span (e.g., apply a grid column rule such as col-span-3 / grid-column: 1 / -1
via your utility classes) so the federal-use note reliably renders full-width
beneath the pricing cards.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: ed20d1ca-2eee-4dd1-b650-e7a1d5129bdf
📒 Files selected for processing (53)
.github/CODEOWNERS.github/VERSIONING.md.github/workflows/automated-release.yml.gitignoreCHANGELOG.mdCODEBASE_ANALYSIS.mdCONTRIBUTING.mdCUsersyolanAppDataLocalTempclaudeC--Users-yolan-source-repos-AiDotNetb62048a4-ff23-4014-839a-561d8c0b88aascratchpadpr821_threads.jsonCUsersyolanAppDataLocalTempclaudeC--Users-yolan-source-repos-AiDotNetb62048a4-ff23-4014-839a-561d8c0b88aascratchpadpr821_verify.jsonCUsersyolanAppDataLocalTempclaudeC--Users-yolan-source-repos-AiDotNetb62048a4-ff23-4014-839a-561d8c0b88aascratchpadpr822.jsonEXECUTIVE_SUMMARY.txtGPU_MATMUL_ANALYSIS.mdIMPLEMENTATION_PLAN_PR430.mdNOTICENuGet.configREADME.mdSECURITY.mdSPRINT_PLAN_PR449_ADVANCED_ALGEBRA.mdSPRINT_PLAN_PR449_ISSUE400.mddocs/internal/audit-2026-05-manual-steps.mddocs/security/pgp.txtfailed_tests_report.mdlstmvae-trace.speedscope.jsonngboost-trace.speedscope.jsonsamples/getting-started/HelloWorld/Program.cssamples/getting-started/HelloWorld/README.mdscripts/regen-changelog.shsrc/AiDotNet.csprojsrc/Autodiff/TensorOperations.cssrc/DistributedTraining/PipelineParallelModel.cssrc/HyperparameterOptimization/HyperparameterOptimizerBase.cssrc/KnowledgeDistillation/Strategies/AttentionDistillationStrategy.cssrc/KnowledgeDistillation/Strategies/FactorTransferDistillationStrategy.cssrc/KnowledgeDistillation/Strategies/NeuronSelectivityDistillationStrategy.cssrc/KnowledgeDistillation/Strategies/ProbabilisticDistillationStrategy.cssrc/KnowledgeDistillation/Strategies/RelationalDistillationStrategy.cssrc/KnowledgeDistillation/Strategies/VariationalDistillationStrategy.cssrc/KnowledgeDistillation/Teachers/EnsembleTeacherModel.cssrc/KnowledgeDistillation/Teachers/OnlineTeacherModel.cssrc/Optimizers/OptimizerBase.cssrc/PhysicsInformed/PDEs/PDESpecificationBase.cssvc-trace.speedscope.jsonthreads.jsonthreads_temp.jsontools/ClipPerfHarness/ClipPerfHarness.csprojtools/ClusterRepro/ClusterRepro.csprojtools/ClusterRepro/Program.cstools/DiffusionPerfDiag/DiffusionPerfDiag.csprojtools/DiffusionPerfDiag/Program.cswebsite/src/pages/enterprise.astrowebsite/src/pages/federal-use.astrowebsite/src/pages/pricing.astrowebsite/src/pages/security.astro
💤 Files with no reviewable changes (10)
- CUsersyolanAppDataLocalTempclaudeC--Users-yolan-source-repos-AiDotNetb62048a4-ff23-4014-839a-561d8c0b88aascratchpadpr821_verify.json
- SPRINT_PLAN_PR449_ISSUE400.md
- EXECUTIVE_SUMMARY.txt
- IMPLEMENTATION_PLAN_PR430.md
- GPU_MATMUL_ANALYSIS.md
- CODEBASE_ANALYSIS.md
- CUsersyolanAppDataLocalTempclaudeC--Users-yolan-source-repos-AiDotNetb62048a4-ff23-4014-839a-561d8c0b88aascratchpadpr822.json
- failed_tests_report.md
- threads_temp.json
- SPRINT_PLAN_PR449_ADVANCED_ALGEBRA.md
…toregressive decode
Replaces the heuristic cosine-affinity action-bin scoring with the paper-faithful
RT-2 inference path (Brohan et al. 2023, arXiv:2307.15818 §3.2):
* RT2ActionTokenizer<T>: 256-bin uniform per-dimension discretization mapped
to the last 256 vocabulary token IDs (the paper's "least frequently used"
reservation). Round-trip encode/decode, per-dim min/max ranges with
broadcast-from-scalar, greedy bin-window argmax over logits.
* RT2.cs vocabulary-projection head: appends LN + DenseLayer(vocabSize) so
the same forward path serves both VQA co-fine-tuning batches and robot-
action batches, per paper §3.1.
* PredictAction now autoregressively decodes ActionDimension × PredictionHorizon
action tokens via greedy selection inside the tokenizer's reserved window,
rather than computing all positions in one shot via cosine heuristics.
* Multimodal fusion concatenates [visual_features, text_embeddings] along the
sequence dimension, matching the PaLI/PaLM-E encoder context layout.
Verified: builds clean on net10.0 + net471 (0 errors). Not verified in-session:
bit-exact numerical parity vs a public PaLI-X / PaLM-E reference (weights are
not publicly released) and physical robot evaluation (requires sim or hardware).
…ebook + cfg-guided generation
Replaces the heuristic CFG + tokenId-to-RGB-hash generation path with the paper-
faithful Janus-Pro pipeline (Chen et al. DeepSeek 2025, arXiv:2501.17811):
* JanusVQCodebook<T>: 16384-entry × 8-dim VQ-VAE codebook per paper Table 1.
Quantize (nearest-neighbour by squared Euclidean), Lookup (id → embedding),
LookupGrid (token sequence → spatial embedding grid for the decoder), and
LoadCodebook for checkpoint-driven weight transfer. Initialised with a
deterministic spectral spread so nearby IDs have nearby embeddings.
* JanusPro.GenerateImage: paired conditional + unconditional autoregressive
forward passes, classifier-free guidance interpolation (Ho & Salimans 2022)
at each step, greedy argmax inside the codebook-token window of the unified
vocabulary, codebook lookup → projection-to-decoder-dim → context append.
* JanusPro.InitializeLayers: unified head emits vocabulary + codebook tokens
(LM head dim = VocabSize + NumVisualTokens), so the same projection serves
both modalities, matching paper §3.1.
* JanusPro decoupled paths: EncodeImage / GenerateFromImage use the SigLIP
understanding encoder; GenerateImage uses the VQ-VAE generation pipeline.
No weight sharing between them beyond the central LLM, matching Janus §3.1.
* VQ-VAE detokenizer: codebook embeddings → bilinear-smoothed pixel expansion.
Honest scope note: this is an un-trained analogue of the paper's deconv
decoder; loading a public Janus-Pro checkpoint replaces the projection
weights so the output becomes photorealistic.
* JanusProOptions: NumGenerationTokens (576 = 24×24 default per paper §3.3),
CodebookEmbeddingDim (8), CfgScale (7.0) — all with paper citations.
Verified: builds clean on net10.0 + net471 (0 errors). Not verified in-session:
parity vs DeepSeek public Janus-Pro-7B / 1B checkpoints, FID/CLIP-Score image
metrics on GenEval/DPG-Bench (require full DeepSeek tokenizer + sim/eval rig).
Contributor
There was a problem hiding this comment.
Actionable comments posted: 6
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/VisionLanguage/Robotics/RT2.cs (1)
395-418:⚠️ Potential issue | 🟠 Major | 🏗️ Heavy liftRebuild derived state after deserialize and stop sharing
_optionsacross instances.
DeserializeNetworkSpecificDatamutates_optionsin place, whileCreateNewInstancehands the same mutableRT2Optionsreference to the newRT2<T>. That couples the source and clone/deserialization target, and_tokenizer/_actionTokenizerare never rebuilt afterVocabSizeorActionDimensionchange.State-isolation fix
- private readonly ITokenizer _tokenizer; - private readonly RT2ActionTokenizer<T> _actionTokenizer; + private ITokenizer _tokenizer = null!; + private RT2ActionTokenizer<T> _actionTokenizer = null!; protected override void DeserializeNetworkSpecificData(BinaryReader reader) { _useNativeMode = reader.ReadBoolean(); ... _options.VocabSize = reader.ReadInt32(); _options.PredictionHorizon = reader.ReadInt32(); + _tokenizer = ClipTokenizerFactory.CreateSimple(vocabSize: _options.VocabSize); + _actionTokenizer = CreateActionTokenizer(_options); if (!_useNativeMode && _options.ModelPath is { } p && !string.IsNullOrEmpty(p)) OnnxModel = new OnnxModel<T>(p, _options.OnnxOptions); } protected override IFullModel<T, Tensor<T>, Tensor<T>> CreateNewInstance() { if (!_useNativeMode && _options.ModelPath is { } mp && !string.IsNullOrEmpty(mp)) - return new RT2<T>(Architecture, mp, _options); - return new RT2<T>(Architecture, _options); + return new RT2<T>(Architecture, mp, _options.Clone()); + return new RT2<T>(Architecture, _options.Clone()); }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/VisionLanguage/Robotics/RT2.cs` around lines 395 - 418, DeserializeNetworkSpecificData currently mutates the shared _options in place and CreateNewInstance passes that same reference to new RT2<T> instances; instead, after reading the serialized fields in DeserializeNetworkSpecificData create a fresh RT2Options instance (or clone the original) and populate its fields (ModelPath, ImageSize, VisionDim, DecoderDim, NumVisionLayers, NumDecoderLayers, NumHeads, ActionDimension, VocabSize, PredictionHorizon, OnnxOptions), assign that new instance to _options, and rebuild any derived state (recreate _tokenizer and _actionTokenizer and reinitialize OnnxModel if ModelPath changed) so the deserialized object no longer shares mutable options with others; similarly ensure CreateNewInstance passes a copy/new RT2Options to the RT2<T> constructor rather than the original shared reference.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/VisionLanguage/Robotics/RT2.cs`:
- Around line 142-147: GenerateFromImage currently ignores the prompt when
IsOnnxMode is true and PredictAction never leverages OnnxModel, leaving an
incomplete ONNX multimodal path; either implement the ONNX image+instruction
decoding pipeline using OnnxModel or fail fast. Update GenerateFromImage and
PredictAction to validate IsOnnxMode: if a prompt/instruction is provided and
the ONNX multimodal decode loop is not implemented for OnnxModel, throw a
NotSupportedException (or similar) indicating multimodal ONNX is unimplemented;
alternatively, implement the full ONNX decoding loop that consumes the
preprocessed image and the tokenized prompt via OnnxModel inference and returns
the multimodal tensor result. Ensure checks reference IsOnnxMode, OnnxModel,
GenerateFromImage and PredictAction so callers can't silently ignore the prompt.
- Around line 260-266: The current helpers that fabricate sinusoidal (sin/cos)
token embeddings must be replaced with the model's real token-embedding path:
remove the synthetic sin/cos vector generation and instead pass the token ID
sequence through the decoder's learned embedding lookup (e.g., call the
decoder/embedding-table lookup method used by the vocab head) so the produced
text-token embeddings match the learned embedding space; ensure the returned
tensor matches expected shape/dtype and includes any positional embedding logic
the decoder expects, and update both places where the sin/cos helpers are used
(the method that concatenates visual encoder output with instruction-token
embeddings and the other helper referenced in the same file) to use
decoder.EmbedTokens or EmbeddingTable.Lookup so the vocab head and decoder share
the exact token representations.
- Around line 339-344: The Train method can leave the model stuck in training
mode if TrainWithTape throws; wrap the TrainWithTape call in a try/finally so
SetTrainingMode(false) is always executed (while still preserving the IsOnnxMode
check and rethrowing the original exception), e.g. call SetTrainingMode(true),
then try { TrainWithTape(input, expected); } finally { SetTrainingMode(false); }
within the Train override.
- Around line 124-125: The public property ActionTokenizer currently exposes the
helper type RT2ActionTokenizer<T> in RT2.cs; change its visibility to internal
(or private) so the plumbing type is not part of the public API surface and keep
tokenization functionality accessible only via RT2<T> or the higher-level facade
methods; update any internal callers to reference the now-internal
ActionTokenizer and add/adjust public wrapper methods on RT2<T> to perform
tokenization/encoding operations for callers that need the behavior without
exposing RT2ActionTokenizer<T>.
- Around line 210-217: When a user supplies Architecture.Layers we must not
infer the encoder/decoder split with Layers.Count / 2; instead require explicit
boundary metadata and fail fast if it's missing. Update the block that currently
sets _encoderLayerEnd = Layers.Count / 2 to read and use a provided boundary
field/property on Architecture (e.g., Architecture.EncoderLayerEnd or
Architecture.EncoderDecoderBoundary); if that property is null/invalid, throw an
exception (or return an error) rather than proceeding. Keep the call to
ValidateEncoderDecoderBoundary(_encoderLayerEnd) but ensure _encoderLayerEnd is
only set from the user-supplied metadata, not from a heuristic.
In `@src/VisionLanguage/Robotics/RT2ActionTokenizer.cs`:
- Around line 86-90: Validate inputs for exact expected shapes instead of only
checking minimum length: in EncodeAction ensure continuousAction.Length ==
ActionDim and throw ArgumentException (include actual length and expected
ActionDim using nameof(continuousAction)), and apply the same pattern to the
other tokenizer entry points noted in the review (e.g., GreedyActionToken and
any Encode/Decode token or logits methods referenced around lines 106-108,
127-129, 149-151, 170-171, 197-202) so each method rejects inputs that are too
short or too long; for tensor/array inputs check exact Length/shape, for
flattened decoder outputs validate expected vocabulary-length multiples, and use
clear ArgumentException/ArgumentNullException messages including parameter names
and observed vs expected sizes.
---
Outside diff comments:
In `@src/VisionLanguage/Robotics/RT2.cs`:
- Around line 395-418: DeserializeNetworkSpecificData currently mutates the
shared _options in place and CreateNewInstance passes that same reference to new
RT2<T> instances; instead, after reading the serialized fields in
DeserializeNetworkSpecificData create a fresh RT2Options instance (or clone the
original) and populate its fields (ModelPath, ImageSize, VisionDim, DecoderDim,
NumVisionLayers, NumDecoderLayers, NumHeads, ActionDimension, VocabSize,
PredictionHorizon, OnnxOptions), assign that new instance to _options, and
rebuild any derived state (recreate _tokenizer and _actionTokenizer and
reinitialize OnnxModel if ModelPath changed) so the deserialized object no
longer shares mutable options with others; similarly ensure CreateNewInstance
passes a copy/new RT2Options to the RT2<T> constructor rather than the original
shared reference.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 4413a7c2-7808-4941-87e8-0c02083e9e60
📒 Files selected for processing (2)
src/VisionLanguage/Robotics/RT2.cssrc/VisionLanguage/Robotics/RT2ActionTokenizer.cs
PR #1430 review (CodeRabbit minor): the python block under the script hardcoded the release-link host as 'ooples/AiDotNet'. The shell side already respects GITHUB_REPOSITORY/REPO overrides for gh CLI invocation, but the generated CHANGELOG lines didn't — so any run with the env var overridden (forks, downstream mirrors, ci-on-clone testing) produced a CHANGELOG full of dead links pointing at ooples/AiDotNet/releases/tag/… instead of the actual repo. Pass $REPO into the python heredoc as a second positional arg and use it for the release-link format string. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…1430) PR #1430 review (CodeRabbit minor): the ArgumentOutOfRangeException message listed 'Mean, WeightedAverage, Median' but the switch above actually handles WeightedAverage, GeometricMean, Maximum, Median — 'Mean' is not a mode, GeometricMean and Maximum were missing. Reporters following the error message would try invalid values like 'Mean' instead of the actual GeometricMean / Maximum cases. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…e (PR #1430) PR #1430 review (CodeRabbit minor): the exception said 'ExponentialMovingAverage' but the actual OnlineUpdateMode enum member is 'EMA'. Callers troubleshooting an invalid value would type the fully-spelled name from the error and still get the same exception. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PR #1430 review (CodeRabbit minor): the federal-use paragraph was a fourth child of the 3-column pricing grid, so on md+ breakpoints it got squeezed into a phantom fourth column instead of rendering full-width beneath the cards. Move it OUT of the grid container so it lives as a sibling paragraph below; the existing max-w-3xl mx-auto + text-center continue to center it under the pricing cards. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PR #1430 review (CodeRabbit major): the previous commit converted PipelineParallelModel's ctor throw from NotImplementedException to NotSupportedException (audit finding #10 — Selective/Full recompute strategies are not-currently-supported configurations, not missing implementations). The matching test still asserted the old type and would fail; update it and document the rationale. External callers catching NotImplementedException on this constructor need to migrate to NotSupportedException. Documenting that in this test's comment + the original audit commit message rather than a separate release-notes entry — there's no public API consumer of RecomputeStrategy.Selective/Full pipeline checkpointing yet (the feature isn't shipped), so the migration window is empty. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…s (PR #1430) PR #1430 review (CodeRabbit major): the per-layer comparison loop iterated up to network.Layers.Count, throwing IndexOutOfRangeException on line 58 when cloned.Layers.Count < network.Layers.Count. That's EXACTLY the failure mode this repro tool exists to diagnose, so it lost the per-layer diagnostics callers needed. - Iterate up to System.Math.Min(orig, cloned) so the loop never out-of-bounds the cloned side. - After the shared range, emit one log line per layer that's MISSING on either side so the output still shows which layers got dropped. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…exit paths (PR #1430) PR #1430 review (CodeRabbit major): the early 'return 1' paths after each Phase B/C/D failure skipped deterministic cleanup for log (and plan, once Phase C had succeeded). Wrap the whole main body in a try/finally so: - log.Dispose() runs on every exit (flushes the .log file before the process exits — important for diagnostic perf-1305-compile- phases.log which is the entire point of the tool). - plan?.Dispose() runs once Phase C populated it, regardless of whether subsequent phases throw. - scope?.Dispose() runs after the Phase B trace block populates it, rather than the previous duplicated scope?.Dispose() calls in each catch arm. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…igger with new PATCH policy (PR #1430) PR #1430 review (CodeRabbit major × 3): the audit-2026-05 finding #18 remediation correctly changed the bump-analyzer step in .github/workflows/automated-release.yml to map fix/perf/refactor/docs/etc. to PATCH instead of MINOR, but three sibling docs/triggers still encoded the OLD policy and contradicted the new code path: 1. .github/VERSIONING.md "MINOR Version Bump" section still listed fix/refactor/perf/docs alongside feat:, and the "Commit Types" table at the bottom said "fix → MINOR / test → None". Updated: - MINOR section now only lists feat: (semver.org §7 is explicit about this — only new features bump MINOR). - Commit-types table now maps fix/refactor/perf/docs/test/chore/ style/ci/build/revert all to PATCH. 2. .github/workflows/automated-release.yml trigger filter ignored '**.md' and 'docs/**', so docs-only commits never ran this workflow — making the docs:→PATCH rule at the analyzer step unreachable. Drop those path-ignores (kept only the genuine non-release files like ISSUE_TEMPLATE/.gitignore/.editorconfig) with an in-place comment explaining why so this doesn't drift back in. 3. CONTRIBUTING.md's "Valid Types and Version Impact" listed fix/docs/refactor/perf as MINOR and test/chore/ci/style as None. Updated to match VERSIONING.md and the workflow analyzer exactly. Added a paragraph pointing at VERSIONING.md as the authoritative source so future drift fails the next reviewer's "does this match VERSIONING.md?" check. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
… shapes, internal helpers (PR #1430) PR #1430 review (CodeRabbit): six unresolved threads on RT2.cs and RT2ActionTokenizer.cs addressed in one commit because they're tightly coupled (the embedding fix relies on a real EmbeddingLayer being allocated in the ctor, which forces the ActionTokenizer-visibility change to keep the public-facade boundary clean): 1. CRITICAL — synthetic sin/cos token embeddings replaced with a learned EmbeddingLayer<T> that the encoder + decoder + future weight-tied LM head share. Per Brohan et al. 2023 §3.1 RT-2's text tokens AND action-bin tokens flow through the SAME PaLI/PaLM-X embedding table; the previous deterministic sinusoidal projection wasn't model-faithful and decoupled the input representation from the vocab head's training signal. EmbedInstructionTokens + AppendActionTokenEmbedding now both go through _tokenEmbedding. 2. CRITICAL — GenerateFromImage(image, prompt) was silently DROPPING the prompt in ONNX mode (single-input ONNX export, vision-only). Now throws NotSupportedException when prompt is non-blank, with a message pointing at the native-mode constructor. PredictAction ALSO fails-fast in ONNX mode rather than running the native decode path with no ONNX weights backing it (which would silently produce actions from random native init while user thinks they're using the loaded checkpoint). 3. CRITICAL — Layers.Count / 2 encoder/decoder boundary heuristic for custom architectures replaced with a fail-fast NotSupportedException pointing at the right next step (add EncoderLayerCount to RT2Options). The heuristic happened to be correct for the default topology but would silently misroute layers on any asymmetric vision/decoder split. 4. MAJOR — Train() wraps TrainWithTape in try/finally so SetTrainingMode(false) runs even if Train throws. Otherwise a NaN-gradient or optimizer-state exception left the model stuck in training mode for the next Predict, silently flipping dropout / batchnorm semantics. 5. MAJOR — ActionTokenizer property changed from public to internal. It's a plumbing/helper type; the facade API (PredictAction, GenerateFromImage) is the supported surface. Tests / training- data-prep code can still access it via InternalsVisibleTo. 6. CRITICAL — RT2ActionTokenizer length guards tightened from "< ActionDim" to "== ActionDim" everywhere (EncodeAction ×2, EncodeHorizon, DecodeAction, DecodeHorizon). Silently truncating extra dimensions made misaligned callers (wrong ActionDim, flattened multi-step tensor) look valid. GreedyActionToken now demands logits.Length == VocabSize so a flattened multi-position decoder output gets rejected with a clear "slice to last position first" diagnostic instead of being silently argmax'd over the first position's bin window. New VocabSize property exposes the contract value (= TokenIdEndExclusive per paper §3.2). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…fail-fast on placeholder weights (PR #1430) PR #1430 review (CodeRabbit): six unresolved threads across JanusPro.cs, JanusVQCodebook.cs, and JanusProOptions.cs addressed together because the VQCodebook fail-fast change cascades into JanusPro.GenerateImage's guard, and the option-validation closes the same prompt-comes-in-bad shape of bug. 1. MAJOR — JanusVQCodebook<T> made internal (was public sealed) + JanusPro.VQCodebook property made internal (was public). VQCodebook is generation plumbing; the facade API exposes GenerateImage. Tests reach through InternalsVisibleTo per the existing codebase convention. 2. CRITICAL — JanusVQCodebook constructor no longer seeds _codebook with deterministic spectral placeholder values. Allocates the storage but leaves it zero-initialised; new IsLoaded flag + new private EnsureLoaded() check throw a clear InvalidOperationException from Lookup / LookupGrid / Quantize until LoadCodebook has imported a real trained codebook. The previous placeholder produced structured-but-meaningless generation that violated paper fidelity; failing fast forces a real checkpoint to be loaded. 3. CRITICAL — JanusPro.GenerateImage now fails fast in native mode when the VQ codebook hasn't been loaded. Validation message points at the DeepSeek-AI/Janus-Pro checkpoint and the ONNX-mode alternative for users who can't load native weights. 4. MAJOR — JanusPro.GenerateImage now validates textDescription (rejects null/empty/whitespace at the API boundary) before tokenization, with a message explaining the contract. 5. CRITICAL — JanusPro._vqCodebook changed from readonly to non-readonly + rebuilt inside DeserializeNetworkSpecificData against the just-deserialized NumVisualTokens / CodebookEmbeddingDim. The previous readonly field kept the constructor-time dimensions after deserialization, producing a shape mismatch on every subsequent Lookup. (Codebook entries themselves still need a separate LoadCodebook call — they're not in the serialization stream — but at least the dimensions match the deserialized config.) 6. MAJOR — JanusProOptions.NumGenerationTokens / CodebookEmbeddingDim / CfgScale gain backing fields + setter validation. Zero or negative NumGenerationTokens / CodebookEmbeddingDim now throw ArgumentOutOfRangeException; CfgScale rejects NaN/Infinity/non- positive. Previously these flowed unchecked into downstream generation paths where they'd crash with much less actionable diagnostics. Pattern mirrors SpikingNeuralNetworkOptions / EchoStateNetwork options for consistency. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
ooples
pushed a commit
that referenced
this pull request
May 23, 2026
Brings PRs #1421 (SenseVoice / Paraformer BN→LN), #1424 (NN test failures), and #1430 (audit 2026-05 remediation) into this branch. Two add/add conflicts resolved by accepting HEAD (this branch's post-review versions): - src/NeuralNetworks/Layers/CifAlignmentLayer.cs: keep this branch's `SupportsTraining => false` honest-contract version (review thread PRRT_kwDOKSXUF86EQONV) and the `threshold >= 1.0` guard (PRRT_kwDOKSXUF86EQONS). Master had the earlier `SupportsTraining => true` + `threshold > 0` versions from the cherry-pick of #1421's CIF implementation; the post-review hardening on this branch supersedes them. - tests/AiDotNet.Tests/Performance/SenseVoiceTrainStepProfile.cs: keep this branch's assertion-bearing profile test (PRRT_kwDOKSXUF86EMZ2n / EMZ2t / EMZ3F) plus the `$`-prefix interpolation fix (PRRT_kwDOKSXUF86EQONX) over master's earlier log-only version. The post-review test enforces real budget assertions for ctor / warm-Predict / median-warm-Train / ForwardForTraining phases. No other files conflicted — VERSIONING.md, CONTRIBUTING.md, automated-release.yml, CHANGELOG.md, SECURITY.md, NuGet.config etc. all merged cleanly because both sides edited disjoint regions (or this branch had the same edits already from cherry-picks). Build: dotnet build src/AiDotNet.csproj passes 0 errors / 11477 warnings (warnings unchanged from master tip).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses the external audit-2026-05 evaluation reports (Claude + ChatGPT-5.5, 2026-05-21). Closes 16 of 19 findings; 3 (AiModelBuilder DI refactor, Phase 4 RSA, Phase 5 net471 SIMD) tracked as follow-up PRs.
Findings closed in this PR
enabled = false; opt-in viaAIDOTNET_TELEMETRY=true; first-run console notice;DISABLE_TELEMETRYbuild flag gated by enterprise license (AIDOTNET001-004 errors).gitignoreextended with prevention patterns (Cusers*,*scratchpad*,*.speedscope.json,IMPLEMENTATION_PLAN_*, etc.)/enterprisepage; Phase 4 RSA signature verification scaffolded (commented inline; precompiled-validator-DLL follow-up)43917c64e(RT-2 — 256-bin ActionTokenizer + autoregressive decode + vocab head),7e92676d1(JanusPro — JanusVQCodebook 16384x8 + CFG + decoupled SigLIP/VQ paths),dfa161bf2(Helix — HelixSystem2Latent + HelixDualSystemRunner @ 22:1 S1:S2 ratio + native S1 transformer),ce1f6763d(GR00T-N1 — GR00TFlowMatchingActionHead + DiT-AdaLN velocity field + sinusoidal time embedding). 25 architectural-correctness unit tests added in7e42f0235covering tokenizer round-trip, codebook quantize identity, S1/S2 caching contract, and flow-matching integration count.failed_tests_report.mdCOVERAGE_THRESHOLD: 3coverage-baseline.json+ tolerance + auto-detect-bootstrap workflow)<Description>; Phase 5 will add a custom System.Numerics.Vector-based SIMD pathscripts/regen-changelog.shadded; ran againstgh release list --limit 300→ 210 release entries backfilledmerge-dev2-to-masterin CONTRIBUTINGmaster; stale local branches prunedautomated-release.ymlbump logic gained PATCH tier for fix/perf/refactor/docs/chore/style/test/ci/build/revert;VERSIONING.mdupdated; consumers warned about pre-0.205 pinningNeuralNetworkArchitecture+AiModelBuilderAPI; README "Hello World" snippet updated to a literal extractFindings tracked as follow-up PRs
[Obsolete]facade preservation for one minor cycle. Tracked as Phase 2a follow-up.VLA Phase 3 — paper-faithful implementation details
Each VLA model now has a dedicated paper-faithful helper class composed on top of the existing transformer-stack factory:
RT-2 (Brohan et al. 2023, arXiv:2307.15818):
RT2ActionTokenizer<T>implements the 256-bin uniform per-dimension discretization mapped to the last 256 vocabulary token IDs (the paper's "least frequently used" reservation). RT-2 now appends a vocab-projection head so the same forward path serves both VQA co-fine-tuning and robot-action batches, andPredictActionautoregressively decodesActionDimension × PredictionHorizontokens via greedy selection inside the tokenizer's reserved window.JanusPro (Chen et al. DeepSeek 2025, arXiv:2501.17811):
JanusVQCodebook<T>is the 16384-entry x 8-dim VQ-VAE codebook withLookup/Quantize/LookupGrid/LoadCodebook. The newGenerateImageruns paired conditional + unconditional autoregressive passes with classifier-free guidance (Ho & Salimans 2022), then maps codebook IDs back to pixels via codebook lookup + bilinear-smoothed expansion. Decoupled SigLIP understanding path vs VQ-VAE generation path matches Janus §3.1.Helix (Figure AI 2025, arXiv:2502.07092):
HelixSystem2Latent<T>is the typed conditioning signal with freshness tracking;HelixDualSystemRunner<T>is the explicit S1:S2 rate splitter (default 22:1, matching paper §4.1's 200 Hz : ~9 Hz).System2Forwardruns the full VLM,System1Forwardruns the native 80M visuomotor transformer (8 x 384 x 6-head per paper §3.3) — composed inline inInitializeLayers.CreateDualSystemRunnerwires both into the runner for streaming 200 Hz control loops.GR00T-N1 (NVIDIA 2025, arXiv:2503.14734):
GR00TFlowMatchingActionHead<T>implements the Lipman et al. ICLR 2023 flow-matching Euler integrator (16 steps default per paper §4.1).System1Velocityis the DiT-AdaLN-style velocity network that concatenates[noisy_action, sinusoidal_time_embedding, S2_latent]and runs through the native 12 x 1024 x 16-head System-1 transformer. ReusesHelixDualSystemRunnerfor streaming 50 Hz control.Honest scope note (also in every Phase-3 commit message): bit-exact numerical parity vs published reference checkpoints requires loading the original DeepSeek / NVIDIA weights and is not verifiable in-session; physical robot deployment / sim2real requires hardware. The architectural code is paper-faithful and builds clean on both TFMs.
Phase 4 (Enterprise) status
The
build/AiDotNet.Tensors.Enterprise.targetsfile enforces thatDISABLE_TELEMETRYandDISABLE_LICENSE_GUARDcompile flags require a valid enterprise license file. Phase 0 ships a placeholder marker-string check (file must containAiDotNet-Enterprise-License-v1). Phase 4 follow-up will replace this with RSA-2048 / ed25519 signature verification — the inlineRoslynCodeTaskFactorypath was attempted but hit MSBuild XML-load fragility, so the proper path is a precompiled validator DLL shipped underbuild/. Documented indocs/internal/audit-2026-05-manual-steps.md.Phase 5 (net471 custom SIMD) status
Tracked as follow-up. ~2-3 weeks of porting
SimdVector/SimdConvHelper/NativeMemoryManagerto a System.Numerics.Vector path that outperforms the BCL implementation on net471. Disclaimer text shipped now so customers aren't misled.Manual steps (you do — see
docs/internal/audit-2026-05-manual-steps.md)76.76.21.21, CNAMEwww → cname.vercel-dns.com, add domain in Vercel Settings. Without this, the new/security/enterprise/federal-usepages return 404 even though they're built.gpg --full-generate-keyfor admin@aidotnet.dev → export todocs/security/pgp.txt→ publish tokeys.openpgp.orgCompanion PR
This PR is paired with ooples/AiDotNet.Tensors
audit/2026-05-full-remediationwhich contains:SupabaseTelemetryClientdefault flip + opt-in + first-run notice +DISABLE_TELEMETRYbuild flag (finding Adding new Normalization options #2)build/AiDotNet.Tensors.Enterprise.targetsenterprise license gateCOVERAGE_THRESHOLDratchet replacement (finding Bump xunit from 2.4.2 to 2.5.2 #8)<Description>net471 disclaimer (finding Update README.md #13)Test plan
dotnet build -c Release -f net10.0— 0 errors (including all Phase 3 VLA work)dotnet build -c Release -f net471— 0 errorsdotnet build -c Release -f net10.0— 0 errors (both with and withoutDISABLE_TELEMETRY)throw new NotImplementedExceptionin src/ (finding Fixed normalization for all regression types #10)AiDotNet.Tests.UnitTests.VisionLanguage.*— 15.86s total)🤖 Generated with Claude Code