Test failure baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.cmd

[v-wenyuxu]

Failed in Run: runtime-coreclr outerloop 20230318.3

Failed tests:

R2R-CG2 windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open
    - baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.cmd

Error message:

Return code:      1
Raw output file:      C:hwAC570944wB68409ADuploads
egressions2164oreground-shutdownoutput.txt
Raw output:
BEGIN EXECUTION
foreground-shutdown.dll
       1 file(s) copied.
20:26:15.68
Response file: C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll.rsp
C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownIL-CG2oreground-shutdown.dll
-o:C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll
--targetarch:x64
--targetos:windows
--verify-type-and-field-layout
--method-layout:random
-r:C:hwAC570944pSystem.*.dll
-r:C:hwAC570944pMicrosoft.*.dll
-r:C:hwAC570944p\xunit.*.dll
-r:C:hwAC570944pmscorlib.dll
-r:C:hwAC570944p
etstandard.dll
-O
" "dotnet" "C:hwAC570944pcrossgen2crossgen2.dll" @"C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll.rsp"   -r:C:hwAC570944wB68409ADe�aseservices	hreading	hreading_group1IL-CG2*.dll  -r:C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownIL-CG2*.dll"
Emitting R2R PE file: C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll
" "dotnet" "C:hwAC570944p
2rdump
2rdump.dll" --header --sc --in C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll --out C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll.r2rdump --val"
20:26:17.71
"C:hwAC570944pcorerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false"  foreground-shutdown.dll 
Xunit.Sdk.EqualException: Assert.Equal() Failure
Expected: 100
Actual:   101
  at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) in /_/src/xunit.assert/Asserts/EqualityAsserts.cs:line 96
  at __GeneratedMainWrapper.Main()
Expected: 100
Actual: 101
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:cores since 3/18/2023 8:26:15 PM
Test Harness Exitcode is : 1
To run the test:
Set up CORE_ROOT and run.
> C:hwAC570944wB68409ADe�aseservices	hreading	hreading_group1../regressions/2164/foreground-shutdown/foreground-shutdown.cmd
Expected: True
Actual:   False

Stack trace:

  at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
  at Xunit.Assert.True(Boolean condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 116
  at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String basePath, String assemblyPath)
  at Program.<<Main>$>g__TestExecutor2|0_1(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2022-11-19T20:51:51.061Z	runtime-coreclr ilasm	Checked-ilasmroundtrip	ubuntu.1804.amd64.open.rt	x64	baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.sh	Failed	{"System.StageName":"Build","DefinitionId":"140","BuildId":"88711","Project":"public","System.JobAttempt":"1","configuration":"Checked-ilasmroundtrip","System.StageAttempt":"1","AzurePipelinesTestRunId":"1867338","CollectionUri":"https://dev.azure.com/dnceng-public/","DefinitionName":"runtime-coreclr ilasm","BuildNumber":"20221119.1","System.PhaseName":"run_test_p1__Linux_x64_checked","operatingSystem":"Ubuntu.1804.Amd64.Open","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"x64","System.JobId":"6dfec640-5b96-5d95-2a5b-7c236ea2d5c5","Reason":"Schedule"}	Schedule
2022-11-19T20:51:17.451Z	runtime-coreclr ilasm	Checked-ilasmroundtrip	ubuntu.1804.armarch.open	arm64	baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.sh	Failed	{"architecture":"arm64","System.JobId":"34b2f53b-add1-5e74-7a39-5a11483297fc","BuildId":"88711","Project":"public","System.PhaseName":"run_test_p1__Linux_arm64_checked","DefinitionId":"140","System.JobName":"__default","CollectionUri":"https://dev.azure.com/dnceng-public/","AzurePipelinesTestRunId":"1867336","Reason":"Schedule","operatingSystem":"(Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8","System.StageAttempt":"1","System.PhaseAttempt":"1","BuildNumber":"20221119.1","System.StageName":"Build","System.JobAttempt":"1","configuration":"Checked-ilasmroundtrip","DefinitionName":"runtime-coreclr ilasm"}	Schedule
2022-11-19T20:48:25.822Z	runtime-coreclr ilasm	Checked-ilasmroundtrip	ubuntu.1804.armarch.open	arm	baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.sh	Failed	{"BuildNumber":"20221119.1","operatingSystem":"(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7","AzurePipelinesTestRunId":"1867310","DefinitionId":"140","System.StageName":"Build","System.JobId":"860c965f-b28e-50d5-65d2-e40bd59f3a69","architecture":"arm","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobAttempt":"1","DefinitionName":"runtime-coreclr ilasm","System.PhaseName":"run_test_p1__Linux_arm_checked","BuildId":"88711","System.JobName":"__default","Reason":"Schedule","System.StageAttempt":"1","Project":"public","System.PhaseAttempt":"1","configuration":"Checked-ilasmroundtrip"}	Schedule

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Failed in Run: runtime-coreclr outerloop 20230318.3

Failed tests:

R2R-CG2 windows x64 Checked no_tiered_compilation @ Windows.10.Amd64.Open
    - baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.cmd

Error message:

Return code:      1
Raw output file:      C:hwAC570944wB68409ADuploads
egressions2164oreground-shutdownoutput.txt
Raw output:
BEGIN EXECUTION
foreground-shutdown.dll
       1 file(s) copied.
20:26:15.68
Response file: C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll.rsp
C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownIL-CG2oreground-shutdown.dll
-o:C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll
--targetarch:x64
--targetos:windows
--verify-type-and-field-layout
--method-layout:random
-r:C:hwAC570944pSystem.*.dll
-r:C:hwAC570944pMicrosoft.*.dll
-r:C:hwAC570944p\xunit.*.dll
-r:C:hwAC570944pmscorlib.dll
-r:C:hwAC570944p
etstandard.dll
-O
" "dotnet" "C:hwAC570944pcrossgen2crossgen2.dll" @"C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll.rsp"   -r:C:hwAC570944wB68409ADe�aseservices	hreading	hreading_group1IL-CG2*.dll  -r:C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownIL-CG2*.dll"
Emitting R2R PE file: C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll
" "dotnet" "C:hwAC570944p
2rdump
2rdump.dll" --header --sc --in C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll --out C:hwAC570944wB68409ADe�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll.r2rdump --val"
20:26:17.71
"C:hwAC570944pcorerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false"  foreground-shutdown.dll 
Xunit.Sdk.EqualException: Assert.Equal() Failure
Expected: 100
Actual:   101
  at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) in /_/src/xunit.assert/Asserts/EqualityAsserts.cs:line 96
  at __GeneratedMainWrapper.Main()
Expected: 100
Actual: 101
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:cores since 3/18/2023 8:26:15 PM
Test Harness Exitcode is : 1
To run the test:
Set up CORE_ROOT and run.
> C:hwAC570944wB68409ADe�aseservices	hreading	hreading_group1../regressions/2164/foreground-shutdown/foreground-shutdown.cmd
Expected: True
Actual:   False

Stack trace:

  at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
  at Xunit.Assert.True(Boolean condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 116
  at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String basePath, String assemblyPath)
  at Program.<<Main>$>g__TestExecutor2|0_1(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)

Queued	Pipeline	Pipeline_Configuration	OS	Arch	Test	Outcome	Properties	BuildReason
2022-11-19T20:51:51.061Z	runtime-coreclr ilasm	Checked-ilasmroundtrip	ubuntu.1804.amd64.open.rt	x64	baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.sh	Failed	{"System.StageName":"Build","DefinitionId":"140","BuildId":"88711","Project":"public","System.JobAttempt":"1","configuration":"Checked-ilasmroundtrip","System.StageAttempt":"1","AzurePipelinesTestRunId":"1867338","CollectionUri":"https://dev.azure.com/dnceng-public/","DefinitionName":"runtime-coreclr ilasm","BuildNumber":"20221119.1","System.PhaseName":"run_test_p1__Linux_x64_checked","operatingSystem":"Ubuntu.1804.Amd64.Open","System.JobName":"__default","System.PhaseAttempt":"1","architecture":"x64","System.JobId":"6dfec640-5b96-5d95-2a5b-7c236ea2d5c5","Reason":"Schedule"}	Schedule
2022-11-19T20:51:17.451Z	runtime-coreclr ilasm	Checked-ilasmroundtrip	ubuntu.1804.armarch.open	arm64	baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.sh	Failed	{"architecture":"arm64","System.JobId":"34b2f53b-add1-5e74-7a39-5a11483297fc","BuildId":"88711","Project":"public","System.PhaseName":"run_test_p1__Linux_arm64_checked","DefinitionId":"140","System.JobName":"__default","CollectionUri":"https://dev.azure.com/dnceng-public/","AzurePipelinesTestRunId":"1867336","Reason":"Schedule","operatingSystem":"(Ubuntu.1804.Arm64.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm64v8","System.StageAttempt":"1","System.PhaseAttempt":"1","BuildNumber":"20221119.1","System.StageName":"Build","System.JobAttempt":"1","configuration":"Checked-ilasmroundtrip","DefinitionName":"runtime-coreclr ilasm"}	Schedule
2022-11-19T20:48:25.822Z	runtime-coreclr ilasm	Checked-ilasmroundtrip	ubuntu.1804.armarch.open	arm	baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.sh	Failed	{"BuildNumber":"20221119.1","operatingSystem":"(Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.Open@mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-helix-arm32v7","AzurePipelinesTestRunId":"1867310","DefinitionId":"140","System.StageName":"Build","System.JobId":"860c965f-b28e-50d5-65d2-e40bd59f3a69","architecture":"arm","CollectionUri":"https://dev.azure.com/dnceng-public/","System.JobAttempt":"1","DefinitionName":"runtime-coreclr ilasm","System.PhaseName":"run_test_p1__Linux_arm_checked","BuildId":"88711","System.JobName":"__default","Reason":"Schedule","System.StageAttempt":"1","Project":"public","System.PhaseAttempt":"1","configuration":"Checked-ilasmroundtrip"}	Schedule

Author:	v-wenyuxu
Assignees:	-
Labels:	area-System.Threading, os-windows, arch-x64, untriaged, blocking-outerloop
Milestone:	-

[v-wenyuxu]

Failed again in: runtime-coreclr r2r-extra 20230326.1

Failed test:

R2R-CG2 windows x64 Checked jitstressregs0x80 @ Windows.10.Amd64.Open

- baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.cmd

R2R-CG2 windows x64 Checked jitstressregs1 @ Windows.10.Amd64.Open

- baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.cmd

R2R-CG2 windows x64 Checked jitstressregs4 @ Windows.10.Amd64.Open

- baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.cmd

R2R-CG2 windows x64 Checked jitstressregs3 @ Windows.10.Amd64.Open

- baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.cmd

R2R-CG2 windows x64 Checked jitstress2 @ Windows.10.Amd64.Open

- baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.cmd

R2R-CG2 windows x64 Checked jitstressregs2 @ Windows.10.Amd64.Open

- baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.cmd

Error message:

Return code:      1
Raw output file:      C:hwA9FF0918wB31A09C1uploads
egressions2164oreground-shutdownoutput.txt
Raw output:
BEGIN EXECUTION
foreground-shutdown.dll
1 file(s) copied.
4:04:36.29
Response file: C:hwA9FF0918wB31A09C1e�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll.rsp
C:hwA9FF0918wB31A09C1e�aseservices	hreading
egressions2164oreground-shutdownIL-CG2oreground-shutdown.dll
-o:C:hwA9FF0918wB31A09C1e�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll
--targetarch:x64
--targetos:windows
--verify-type-and-field-layout
--method-layout:random
-r:C:hwA9FF0918pSystem..dll
-r:C:hwA9FF0918pMicrosoft..dll
-r:C:hwA9FF0918p\xunit..dll
-r:C:hwA9FF0918pmscorlib.dll
-r:C:hwA9FF0918p
etstandard.dll
-O
" "dotnet" "C:hwA9FF0918pcrossgen2crossgen2.dll" @"C:hwA9FF0918wB31A09C1e�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll.rsp"   -r:C:hwA9FF0918wB31A09C1e�aseservices	hreading	hreading_group1IL-CG2.dll  -r:C:hwA9FF0918wB31A09C1e�aseservices	hreading
egressions2164oreground-shutdownIL-CG2*.dll"
Emitting R2R PE file: C:hwA9FF0918wB31A09C1e�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll
" "dotnet" "C:hwA9FF0918p
2rdump
2rdump.dll" --header --sc --in C:hwA9FF0918wB31A09C1e�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll --out C:hwA9FF0918wB31A09C1e�aseservices	hreading
egressions2164oreground-shutdownoreground-shutdown.dll.r2rdump --val"
4:04:38.15
"C:hwA9FF0918pcorerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false"  foreground-shutdown.dll
Xunit.Sdk.EqualException: Assert.Equal() Failure
Expected: 100
Actual:   101
at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) in /_/src/xunit.assert/Asserts/EqualityAsserts.cs:line 96
at __GeneratedMainWrapper.Main()
Expected: 100
Actual: 101
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:cores since 3/27/2023 4:04:36 AM
Test Harness Exitcode is : 1
To run the test:
Set up CORE_ROOT and run.

C:hwA9FF0918wB31A09C1e�aseservices	hreading	hreading_group1../regressions/2164/foreground-shutdown/foreground-shutdown.cmd
Expected: True
Actual:   False


Stack trace
   at Xunit.Assert.True(Nullable`1 condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 132
   at Xunit.Assert.True(Boolean condition, String userMessage) in /_/src/xunit.assert/Asserts/BooleanAsserts.cs:line 116
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String basePath, String assemblyPath)
   at Program.<<Main>$>g__TestExecutor2|0_1(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)

[markples]

fyi - This test was switched to running in a merged test group around the time of this failure. However, it sets RequiresProcessIsolation (because of the Environment.Exit), so there is very little change in behavior. There is a new autogenerated wrapper method that calls the test method rather than the test method itself being the process entry point. I don't think this is the cause, but it's probably something to keep in mind.

[BruceForstall]

@mangod9

[mangod9]

Is this an intermittent failure? @kouvel if he has any insights into this test.

[JulieLeeMSFT]

Failed in runtime-coreclr pgostress pipeline
Failed test

[v-wenyuxu]

Failed again in: runtime-coreclr jitstress 20230402.1

Failed test:

coreclr windows x64 Checked jitstress1_tiered @ Windows.10.Amd64.Open

- baseservices/threading/regressions/2164/foreground-shutdown/foreground-shutdown.cmd

Error message:

Return code:      1
Raw output file:      C:hwB5D10A03wA87F08D9uploads
egressions2164oreground-shutdownoutput.txt
Raw output:
BEGIN EXECUTION
"C:hwB5D10A03pcorerun.exe" -p "System.Reflection.Metadata.MetadataUpdater.IsSupported=false"  foreground-shutdown.dll
Xunit.Sdk.EqualException: Assert.Equal() Failure
Expected: 100
Actual:   101
at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) in //src/xunit.assert/Asserts/EqualityAsserts.cs:line 96
at Xunit.Assert.Equal[T](T expected, T actual) in //src/xunit.assert/Asserts/EqualityAsserts.cs:line 63
at __GeneratedMainWrapper.Main()
Expected: 100
Actual: 101
END EXECUTION - FAILED
FAILED
Test failed. Trying to see if dump file was created in C:cores since 4/3/2023 5:47:01 AM
Test Harness Exitcode is : 1
To run the test:
Set up CORE_ROOT and run.

C:hwB5D10A03wA87F08D9e�aseservices	hreading	hreading_group1../regressions/2164/foreground-shutdown/foreground-shutdown.cmd
Expected: True
Actual:   False


Stack trace
   at TestLibrary.OutOfProcessTest.RunOutOfProcessTest(String basePath, String assemblyPath)
   at Program.<<Main>$>g__TestExecutor2|0_1(StreamWriter tempLogSw, StreamWriter statsCsvSw, <>c__DisplayClass0_0&)