feat: Test Scaffold Generator - warn on untested models, generate skeleton tests

[ooples]

Parent Epic

Part of #956 — Model Domain Attributes

Overview

Create a Roslyn incremental source generator that cross-references model classes (decorated with [ModelDomain]) against test classes to identify untested models. Optionally generates skeleton test files for models that lack test coverage.

Problem Statement

With ~700 model classes, it is easy for new models to be added without corresponding tests. This generator provides:

1. Compile-time warnings for any model class without a matching test class
2. Optional skeleton test generation to jumpstart test writing

Generator Requirements

Input

1. Model classes — all classes with [ModelDomain(...)] attribute
2. Test classes — classes in the test project with [Fact] or [Theory] attributes, or classes whose name matches {ModelName}Tests
3. Convention matching — ResNet<T> matches ResNetTests, ResNet_Tests, or any test class containing a test method referencing ResNet

Output 1: Diagnostics

// Compile-time warnings
warning AIDNT010: Model 'WaveGrad<T>' (Audio/Generation) has no corresponding test class.
warning AIDNT010: Model 'DiffusionTransformer<T>' (Vision/Generation) has no corresponding test class.
// Info-level for models with tests
info AIDNT011: 642 of 700 models have test coverage (91.7%).

Output 2: Test Coverage Report (TestCoverage.g.cs)

// Auto-generated
public static partial class TestCoverage
{
    /// <summary>
    /// Models that have corresponding test classes.
    /// </summary>
    public static IReadOnlyList<Type> TestedModels { get; }
    
    /// <summary>
    /// Models that do NOT have corresponding test classes.
    /// </summary>
    public static IReadOnlyList<Type> UntestedModels { get; }
    
    /// <summary>
    /// Coverage percentage (0-100).
    /// </summary>
    public static double CoveragePercent { get; }
    
    /// <summary>
    /// Get untested models by domain.
    /// </summary>
    public static IReadOnlyList<Type> GetUntestedByDomain(ModelDomain domain);
}

Output 3: Optional Skeleton Tests

When enabled via MSBuild property <GenerateTestSkeletons>true</GenerateTestSkeletons>, generates skeleton test files:

// Auto-generated: WaveGradTests.g.cs
using Xunit;

namespace AiDotNet.Tests.Audio;

/// <summary>
/// Auto-generated test skeleton for WaveGrad.
/// Domain: Audio | Category: NeuralNetwork | Task: Generation
/// TODO: Implement actual test logic.
/// </summary>
public class WaveGradTests
{
    [Fact]
    public void Constructor_CreatesInstance()
    {
        // Arrange & Act
        // var model = new WaveGrad<double>(...);
        
        // Assert
        // Assert.NotNull(model);
        throw new NotImplementedException("TODO: Implement test for WaveGrad");
    }
    
    [Fact]
    public void Forward_ProducesFiniteOutput()
    {
        // Arrange
        // var model = new WaveGrad<double>(...);
        // var input = ...; // TODO: Create appropriate test input
        
        // Act
        // var output = model.Forward(input);
        
        // Assert
        // Assert.All output values are finite
        throw new NotImplementedException("TODO: Implement forward pass test for WaveGrad");
    }
}

Test Matching Heuristics

The generator uses these rules to match models to tests (in priority order):

1. Exact name match: ResNet<T> → ResNetTests class exists
2. Partial name match: ResNet<T> → any test class containing ResNet in its name
3. Method reference: Any test method that references new ResNet< or typeof(ResNet<
4. Attribute reference: [InlineData(typeof(ResNet<double>))] in any theory

Configuration via MSBuild

<PropertyGroup>
    <!-- Enable/disable test skeleton generation (default: false) -->
    <GenerateTestSkeletons>false</GenerateTestSkeletons>
    
    <!-- Severity for untested model warnings (default: Warning) -->
    <UntestedModelSeverity>Warning</UntestedModelSeverity>
    
    <!-- Minimum coverage threshold before error (default: 0, disabled) -->
    <MinTestCoveragePercent>80</MinTestCoveragePercent>
</PropertyGroup>

Implementation Notes

• Use IIncrementalGenerator for performance
• Requires cross-project analysis: model project + test project
• The generator runs in the test project (where it has visibility into both model types via references and test types via the current compilation)
• Use AdditionalTextsProvider or assembly attribute scanning for cross-project data if needed
• net471 compatible output

Acceptance Criteria

• Generator emits AIDNT010 warning for each model without a test class
• Generator emits AIDNT011 info with coverage summary
• TestCoverage.g.cs is generated with correct tested/untested lists
• Optional skeleton test generation works when GenerateTestSkeletons=true
• Test matching heuristics correctly identify tested models
• Coverage percentage is accurate
• Builds on both net10.0 and net471

Dependencies

• Create model metadata attributes and enums, remove ModelType enum #957 (attributes and enums must exist first)
• Annotate all ~700 model classes with metadata attributes #958 (models must be annotated for generator to discover them)