Fix assert and TypeDesc misread in CastHelpers.IsNullableForType for open Nullable

[Copilot]

CastHelpers.IsNullableForType asserted on a checked/Debug runtime whenever typeMT was an open Nullable<> (or Nullable<T> over a generic variable), because the assert called InstantiationArg0(), whose FCall does TypeHandle::AsMethodTable() on what is actually a TypeVarTypeDesc. Reachable from ordinary managed code, e.g. typeof(Nullable<>).IsAssignableFrom(typeof(int)).

Changes

• CastHelpers.IsNullableForType — the assert now short-circuits when the type argument read out of PerInstInfo is a TypeDesc, so InstantiationArg0() is only called when it is valid.
• Latent bug on the same path — on FEATURE_TYPEEQUIVALENCE builds the mismatch path passed the tagged TypeDesc pointer to AreTypesEquivalent, which read HasTypeEquivalence flags off a non-MethodTable pointer. A TypeDesc argument can never be equal or equivalent to a boxed type, so it now returns false before that read. This matches native Nullable::IsNullableForTypeHelper, which goes through TypeHandle::IsEquivalentTo.
• The fast path (pMTNullableArg == boxedMT) is untouched — no added work on the hot unboxing path.

MethodTable *pMTNullableArg = **typeMT->PerInstInfo;
Debug.Assert(new TypeHandle(pMTNullableArg).IsTypeDesc || typeMT->InstantiationArg0() == pMTNullableArg);

Tests

• NullableTests: [Theory] over typeof(Nullable<>).IsAssignableFrom(...) for int, int?, GStruct<int>, plus Nullable<!0> constructed over another type's generic parameter (all false), and a [Fact] for the true cases (Nullable<> from Nullable<>, int? from int).
• System.Reflection.MetadataLoadContext TypeTests.TestIsAssignableFrom: the same open-Nullable<> and Nullable<!0> cases in the projected metadata world.

Verified against a checked CoreCLR: the repro aborts with ASSERT FAILED Expression: !IsTypeDesc() before the change and prints the expected True/False/False/True/False/False after.

• Fixes Assert failure in CastHelpers.IsNullableForType for an open Nullable<T> #133448

[azure-pipelines]

Azure Pipelines:
16 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[dotnet-policy-service]

Tagging subscribers to this area: @steveisok, @dotnet/area-system-reflection
See info in area-owners.md if you want to be subscribed.

[jkotas]

@copilot Delete this test - it is not interesting

[Copilot]

Deleted in 58e56c2.

Note

This reply was generated by GitHub Copilot.

[UnityAlex]

@jkotas is the fix on this branch generally correct? I fixed it differently locally by adjusting the assert to be: Debug.Assert(typeMT->ContainsGenericVariables || typeMT->InstantiationArg0() == **typeMT->PerInstInfo); which is different than what copilot did here.

[jkotas]

is the fix on this branch generally correct?

It is not. Also, I am not convinced whether the reflection behavior for this corner case make sense.

[jkotas]

@copilot Let's avoid using MethodTable* for values that are not guaranteed to be MethodTable* . Check what type code in System.Private.CoreLib typically uses in these situations and update to that. Do not hesitate to change the type of PerInstInfo as appropriate.

Also, change InstantiationArg0 FCall to be only used for debug-only verification, and introduce NullableType property on MethodTable that will fully managed impl optimized for Nulalble . InstantiationArg0 FCall should be only used for debug-only validation in the new property.

[jkotas]

Stuck on "Error: Model "claude-opus-5" is not available."