NativeAOT compiler crashes if assembly missing

[kant2002]

Create console application which targets net7.0 and attempt to add Microsoft.Extensions.Hosting with explicit dependencies and missing reference to Microsoft.Extensions.Configuration.EnvironmentVariables. ILC crashes.

  System.AggregateException: One or more errors occurred. (Code generation failed for method '[Microsoft.Extensions.Hosting]Microsoft.Extensions.Hosting.HostingHostBuilderExtensions.ConfigureDefaults(IHostBui
  lder,string[])')
   ---> ILCompiler.CodeGenerationFailedException: Code generation failed for method '[Microsoft.Extensions.Hosting]Microsoft.Extensions.Hosting.HostingHostBuilderExtensions.ConfigureDefaults(IHostBuilder,stri
  ng[])'
   ---> System.NullReferenceException: Object reference not set to an instance of an object.
     at ILCompiler.LazyGenericsSupport.GraphBuilder.WalkMethod(EcmaMethod method) in /_/src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/LazyGenerics/GraphBuilder.cs:line 204
     at ILCompiler.LazyGenericsSupport.GraphBuilder..ctor(EcmaModule assembly) in /_/src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/LazyGenerics/GraphBuilder.cs:line 59
     at ILCompiler.LazyGenericsSupport.CycleInfoHashtable.CreateValueFromKey(EcmaModule key) in /_/src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/LazyGenerics/ModuleCycleInfo.cs:line 50
     at Internal.TypeSystem.LockFreeReaderHashtable`2.CreateValueAndEnsureValueIsInTable(TKey key) in /_/src/coreclr/tools/Common/TypeSystem/Common/Utilities/LockFreeReaderHashtable.cs:line 559
     at ILCompiler.LazyGenericsSupport.GenericCycleDetector.DetectCycle(TypeSystemEntity owner, TypeSystemEntity referent) in /_/src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/LazyGenerics/ModuleCycleInfo
  .cs:line 193
     at Internal.IL.ILImporter.GetMethodEntrypoint(MethodDesc method) in /_/src/coreclr/tools/aot/ILCompiler.Compiler/IL/ILImporter.Scanner.cs:line 253
     at Internal.IL.ILImporter.ImportCall(ILOpcode opcode, Int32 token) in /_/src/coreclr/tools/aot/ILCompiler.Compiler/IL/ILImporter.Scanner.cs:line 685
     at Internal.IL.ILImporter.ImportBasicBlock(BasicBlock basicBlock) in /_/src/coreclr/tools/Common/TypeSystem/IL/ILImporter.cs:line 600
     at Internal.IL.ILImporter.ImportBasicBlocks() in /_/src/coreclr/tools/Common/TypeSystem/IL/ILImporter.cs:line 311
     at Internal.IL.ILImporter.Import() in /_/src/coreclr/tools/aot/ILCompiler.Compiler/IL/ILImporter.Scanner.cs:line 156
     at ILCompiler.ILScanner.CompileSingleMethod(ScannedMethodNode methodCodeNodeNeedingCode) in /_/src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/ILScanner.cs:line 123
     --- End of inner exception stack trace ---
     at ILCompiler.ILScanner.CompileSingleMethod(ScannedMethodNode methodCodeNodeNeedingCode) in /_/src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/ILScanner.cs:line 134
     at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
  --- End of stack trace from previous location ---
     at System.Threading.Tasks.Parallel.<>c__DisplayClass19_0`1.<ForWorker>b__1(RangeWorker& currentWorker, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
     at System.Threading.Tasks.TaskReplicator.Replica.Execute()
     --- End of inner exception stack trace ---
     at System.Threading.Tasks.TaskReplicator.Run[TState](ReplicatableUserAction`1 action, ParallelOptions options, Boolean stopOnFirstFailure)
     at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit
  , Action`1 localFinally)
  --- End of stack trace from previous location ---
     at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Func`4 bodyWithLocal, Func`1 localInit
  , Action`1 localFinally)
     at ILCompiler.ILScanner.CompileMultiThreaded(List`1 methodsToCompile) in /_/src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/ILScanner.cs:line 97
     at ILCompiler.ILScanner.ComputeDependencyNodeDependencies(List`1 obj) in /_/src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/ILScanner.cs:line 86
     at ILCompiler.DependencyAnalysisFramework.DependencyAnalyzer`2.ComputeMarkedNodes() in /_/src/coreclr/tools/aot/ILCompiler.DependencyAnalysisFramework/DependencyAnalyzer.cs:line 315
     at ILCompiler.ILScanner.ILCompiler.IILScanner.Scan() in /_/src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/ILScanner.cs:line 140
     at ILCompiler.Program.Run(String[] args) in /_/src/coreclr/tools/aot/ILCompiler/Program.cs:line 828
     at ILCompiler.Program.Main(String[] args) in /_/src/coreclr/tools/aot/ILCompiler/Program.cs:line 1085

Addition to console app.

<ItemGroup>
    <Reference Include="Microsoft.Extensions.Hosting">
      <HintPath>$(RuntimeRepoRoot)artifacts\bin\Microsoft.Extensions.Hosting\Release\net7.0\Microsoft.Extensions.Hosting.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.Hosting.Abstractions">
      <HintPath>$(RuntimeRepoRoot)artifacts\bin\Microsoft.Extensions.Hosting.Abstractions\Release\net7.0\Microsoft.Extensions.Hosting.Abstractions.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.Logging">
      <HintPath>$(RuntimeRepoRoot)artifacts\bin\Microsoft.Extensions.Logging\Release\net7.0\Microsoft.Extensions.Logging.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.Logging.Abstractions">
      <HintPath>$(RuntimeRepoRoot)artifacts\bin\Microsoft.Extensions.Logging.Abstractions\Release\net7.0\Microsoft.Extensions.Logging.Abstractions.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.DependencyInjection">
      <HintPath>$(RuntimeRepoRoot)artifacts\bin\Microsoft.Extensions.DependencyInjection\Release\net7.0\Microsoft.Extensions.DependencyInjection.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.DependencyInjection.Abstractions">
      <HintPath>$(RuntimeRepoRoot)artifacts\bin\Microsoft.Extensions.DependencyInjection.Abstractions\Release\net7.0\Microsoft.Extensions.DependencyInjection.Abstractions.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.Configuration">
      <HintPath>$(RuntimeRepoRoot)artifacts\bin\Microsoft.Extensions.Configuration\Release\net7.0\Microsoft.Extensions.Configuration.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.Configuration.Abstractions">
      <HintPath>$(RuntimeRepoRoot)artifacts\bin\Microsoft.Extensions.Configuration.Abstractions\Release\net7.0\Microsoft.Extensions.Configuration.Abstractions.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.FileProviders.Physical">
      <HintPath>$(RuntimeRepoRoot)artifacts\bin\Microsoft.Extensions.FileProviders.Physical\Release\net7.0\Microsoft.Extensions.FileProviders.Physical.dll</HintPath>
    </Reference>
    <Reference Include="Microsoft.Extensions.FileProviders.Abstractions">
      <HintPath>$(RuntimeRepoRoot)artifacts\bin\Microsoft.Extensions.FileProviders.Abstractions\Release\net7.0\Microsoft.Extensions.FileProviders.Abstractions.dll</HintPath>
    </Reference>
  </ItemGroup>

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[MichalStrehovsky]

The instructions will have to be more specific. I tried locally, but ran into:

  Unhandled exception. Internal.TypeSystem.TypeSystemException+FileNotFoundException: Failed to load assembly 'Microsof
  t.Extensions.Primitives'
     at Internal.TypeSystem.ThrowHelper.ThrowFileNotFoundException(ExceptionStringID id, String fileName) in C:\git\run
  time\src\coreclr\tools\Common\TypeSystem\Common\ThrowHelper.cs:line 35
     at Internal.TypeSystem.ResolutionFailure.Throw() in C:\git\runtime\src\coreclr\tools\Common\TypeSystem\Common\Reso
  lutionFailure.cs:line 105
     at Internal.TypeSystem.Ecma.EcmaModule.GetObject(EntityHandle handle, NotFoundBehavior notFoundBehavior) in C:\git
  \runtime\src\coreclr\tools\Common\TypeSystem\Ecma\EcmaModule.cs:line 387
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ResolveHandle(EntityHandle handle) in C:\git\runtime\src\coreclr\t
  ools\Common\TypeSystem\Ecma\EcmaSignatureParser.cs:line 65
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseTypeImpl(SignatureTypeCode typeCode) in C:\git\runtime\src\co
  reclr\tools\Common\TypeSystem\Ecma\EcmaSignatureParser.cs:line 150
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseType(SignatureTypeCode typeCode) in C:\git\runtime\src\corecl
  r\tools\Common\TypeSystem\Ecma\EcmaSignatureParser.cs:line 102
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseTypeImpl() in C:\git\runtime\src\coreclr\tools\Common\TypeSys
  tem\Ecma\EcmaSignatureParser.cs:line 335
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseType() in C:\git\runtime\src\coreclr\tools\Common\TypeSystem\
  Ecma\EcmaSignatureParser.cs:line 325
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseMethodSignatureImpl(Boolean skipEmbeddedSignatureData) in C:\
  git\runtime\src\coreclr\tools\Common\TypeSystem\Ecma\EcmaSignatureParser.cs:line 406
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseMethodSignatureInternal(Boolean skipEmbeddedSignatureData) in
   C:\git\runtime\src\coreclr\tools\Common\TypeSystem\Ecma\EcmaSignatureParser.cs:line 372
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseMethodSignature() in C:\git\runtime\src\coreclr\tools\Common\
  TypeSystem\Ecma\EcmaSignatureParser.cs:line 354
     at Internal.TypeSystem.Ecma.EcmaMethod.InitializeSignature() in C:\git\runtime\src\coreclr\tools\Common\TypeSystem
  \Ecma\EcmaMethod.cs:line 86
     at Internal.TypeSystem.Ecma.EcmaMethod.get_Signature() in C:\git\runtime\src\coreclr\tools\Common\TypeSystem\Ecma\
  EcmaMethod.cs:line 95
     at ILCompiler.DependencyAnalysis.EETypeNode.get_InterestingForDynamicDependencyAnalysis() in C:\git\runtime\src\co
  reclr\tools\aot\ILCompiler.Compiler\Compiler\DependencyAnalysis\EETypeNode.cs:line 166

or

 Internal.TypeSystem.TypeSystemException+FileNotFoundException: Failed to load assembly 'Microsoft.Extensions.Options'
     at Internal.TypeSystem.ThrowHelper.ThrowFileNotFoundException(ExceptionStringID id, String fileName) in /_/src/cor
  eclr/tools/Common/TypeSystem/Common/ThrowHelper.cs:line 35
     at Internal.TypeSystem.ResolutionFailure.Throw() in /_/src/coreclr/tools/Common/TypeSystem/Common/ResolutionFailur
  e.cs:line 105
     at Internal.TypeSystem.Ecma.EcmaModule.GetObject(EntityHandle handle, NotFoundBehavior notFoundBehavior) in /_/src
  /coreclr/tools/Common/TypeSystem/Ecma/EcmaModule.cs:line 387
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ResolveHandle(EntityHandle handle) in /_/src/coreclr/tools/Common/
  TypeSystem/Ecma/EcmaSignatureParser.cs:line 65
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseTypeImpl(SignatureTypeCode typeCode) in /_/src/coreclr/tools/
  Common/TypeSystem/Ecma/EcmaSignatureParser.cs:line 150
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseType(SignatureTypeCode typeCode) in /_/src/coreclr/tools/Comm
  on/TypeSystem/Ecma/EcmaSignatureParser.cs:line 102
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseType() in /_/src/coreclr/tools/Common/TypeSystem/Ecma/EcmaSig
  natureParser.cs:line 325
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseTypeImpl(SignatureTypeCode typeCode) in /_/src/coreclr/tools/
  Common/TypeSystem/Ecma/EcmaSignatureParser.cs:line 227
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseType(SignatureTypeCode typeCode) in /_/src/coreclr/tools/Comm
  on/TypeSystem/Ecma/EcmaSignatureParser.cs:line 102
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseType() in /_/src/coreclr/tools/Common/TypeSystem/Ecma/EcmaSig
  natureParser.cs:line 325
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseMethodSignatureImpl(Boolean skipEmbeddedSignatureData) in /_/
  src/coreclr/tools/Common/TypeSystem/Ecma/EcmaSignatureParser.cs:line 415
     at Internal.TypeSystem.Ecma.EcmaSignatureParser.ParseMethodSignature() in /_/src/coreclr/tools/Common/TypeSystem/E
  cma/EcmaSignatureParser.cs:line 354
     at Internal.TypeSystem.Ecma.EcmaMethod.InitializeSignature() in /_/src/coreclr/tools/Common/TypeSystem/Ecma/EcmaMe
  thod.cs:line 86
     at Internal.TypeSystem.Ecma.EcmaType.GetDefaultConstructor() in /_/src/coreclr/tools/Common/TypeSystem/Ecma/EcmaTy
  pe.cs:line 377
     at ILCompiler.DependencyAnalysis.ConstructedEETypeNode.ComputeNonRelocationBasedDependencies(NodeFactory factory)

It depended on timing.

The compiler can generally recover from garbage inputs in some spots, but in other spots it can't. The fixes usually involve doing more upfront validation so that we find the problem in a recoverable spot. Some of which slow down compilation. We're okay with doing the cheap validation, but failures that involve expensive upfront validation are usually won't fixed.

[kant2002]

apireflection.zip

Just running dotnet publish --self-contained trigger issue for me. I understand that you cannot validate everything, but NRE seems wierd. I worry that making app run in NativeAOT is somewhat laborous process because most people do not jump on R2R + trimming ship, and will have all issues from these modes + NativeAOT issues. So NRE would trigger a lot of people. Let me know if you was able to repro or not.

[MichalStrehovsky]

Thanks for the repro. Yep, this is in the bucket of "we can validate cheaply". The NullRef was just stupid (we ask for the API to return null instead of throwing and then don't check the null). There was one more issue that can be fixed with a try/catch.

The other two stacks I pasted above wouldn't be so easy, but we don't hit them.

[kant2002]

At least these 2 issues which you post are pretty explanatory, and even if they look scary, they actionable on dev side.