Generic test rows that violate type constraints are silently dropped in source-gen mode but fail in reflection mode #7097

Summary

When a generic test method infers its type argument from a data source, and the inferred type breaks the method's generic constraints, source-generated mode and reflection mode do different things:

• Source-generated mode drops the row. No test is discovered, so nothing tells you the data source and the constraint disagree.
• Reflection mode discovers the test and reports it as failed, with a message naming the constraint that was broken.

So the same test class gives a different number of tests in each mode. TUnit requires metadata collection to behave the same in both modes. This was found while working on #7094.

Repro

tests/TUnit.TestProject/GenericConstraintValidationTests.cs (it uses no [GenerateGenericTest]):

[Test]
[IntDataSourceForConstraint]   // yields 42
[Skip("Expected to fail - int violates class constraint")]
public async Task GenericMethodWithClassConstraint_InvalidType<T>(T value) where T : class { ... }

[Test]
[NonImplementingClassDataSource]
[Skip("Expected to fail - NonImplementingClass doesn't implement ITestInterface")]
public async Task GenericMethodWithInterfaceConstraint_InvalidType<T>(T value) where T : ITestInterface { ... }

[Test]
[Arguments("invalid")]
[Skip("Expected to fail - string violates struct constraint")]
public async Task GenericMethodWithStructConstraint_InvalidType<T>(T value) where T : struct { ... }

cd tests/TUnit.TestProject
dotnet run -f net10.0 -- --treenode-filter "/*/*/GenericConstraintValidationTests/*"
dotnet run -f net10.0 -- --treenode-filter "/*/*/GenericConstraintValidationTests/*" --reflection

What each mode does (on main at f41b8ce)

Source-generated: 3 tests, 3 passed. None of the three *_InvalidType tests exist.

passed GenericMethodWithStructConstraint_ValidType<int>(5)
passed GenericMethodWithClassConstraint_ValidType<string>(valid)
passed GenericMethodWithInterfaceConstraint_ValidType<global::TUnit.TestProject.TestClass>(TestClass)
total: 3, failed: 0, succeeded: 3

Reflection: 6 tests, 3 passed, 3 failed.

failed GenericMethodWithClassConstraint_InvalidType
  GenericTypeResolutionException: Type 'Int32' does not satisfy the 'class' constraint for generic parameter 'T'
failed GenericMethodWithInterfaceConstraint_InvalidType
  GenericTypeResolutionException: Type 'NonImplementingClass' does not implement interface 'ITestInterface' required for generic parameter 'T'
failed GenericMethodWithStructConstraint_InvalidType
  GenericTypeResolutionException: Type 'String' does not satisfy the 'struct' constraint for generic parameter 'T'
passed GenericMethodWithStructConstraint_ValidType(5)
passed GenericMethodWithInterfaceConstraint_ValidType(TestClass)
passed GenericMethodWithClassConstraint_ValidType(valid)
total: 6, failed: 3, succeeded: 3

The failures come from TestGenericTypeResolver.ValidateAgainstConstraints while the test is being built. Build errors are reported as failures before [Skip] is considered, which is why the [Skip] attributes have no effect in reflection mode.

Root cause

TestMetadataGenerator.CollectConcreteInstantiations infers the type arguments from each data attribute. It then calls ValidateTypeConstraints and returns without recording anything when the check fails. It has to skip these combinations, because emitting instance.Method<int>(...) against where T : class would not compile. But nothing else is emitted in their place. If every inferred instantiation is invalid, GenerateTestMetadata hits concreteInstantiations.Count == 0 and emits nothing at all for the method.

Reflection mode keeps the open generic method and resolves T for each row at build time. When the resolved type breaks a constraint, it turns the GenericTypeResolutionException into a failed test.

Correct behaviour, and why

Both modes should report a failed test that says which constraint was broken. Neither mode should drop the row.

• Test intent. The test file came from the engine rewrite (A bit of a rewrite #2600) together with the constraint-validation code in both modes. Every invalid case has the comment "This should fail during constraint validation", and the skip reasons read "Expected to fail - …". A failure is the documented intent. Dropping the row was a side effect of the generator not being able to emit code that compiles.
• Silent drops hide mistakes. A data source that yields the wrong type is a bug in the user's code. Making the test disappear is the worst way to report it, especially when it was the method's only row.
• Reflection-only users. Projects that build with EnableTUnitSourceGeneration=false (and VB/F# projects) never run the source generator. A compile-time diagnostic alone would never reach them, so the runtime has to report the problem.

User-visible change

In source-generated mode (the default), a generic test whose data-inferred type argument breaks the method's constraints now shows up as one failed test per offending data attribute, instead of disappearing. The failure message matches reflection mode, for example Type 'Int32' does not satisfy the 'class' constraint for generic parameter 'T'. As in reflection mode today, this is a build-time failure, so it is reported even when the test has [Skip]. Suites that passed only because these rows were dropped will now report failures that point at the bad data.

Scope: this covers method type arguments inferred from method-level data attributes ([Arguments], typed data sources, [MethodDataSource]). If an inferred class type argument breaks a constraint, the generator cannot emit a TestEntry<T> for the closed type, so that case is unchanged. It can be handled in a follow-up.