Repository navigation
Test failure: JIT/Methodical/xxobj/sizeof/_il_dbgsizeof_Target_32Bit/_il_dbgsizeof_Target_32Bit.sh #37470
Description
failed in job: runtime-coreclr outerloop 20200604.7
failed tests:
JIT/Methodical/xxobj/sizeof/_il_dbgsizeof_Target_32Bit/_il_dbgsizeof_Target_32Bit.sh
JIT/Methodical/xxobj/sizeof/_il_relsizeof_Target_32Bit/_il_relsizeof_Target_32Bit.sh
JIT/Methodical/xxobj/sizeof/_il_relsizeof32_Target_32Bit/_il_relsizeof32_Target_32Bit.sh
JIT/Methodical/xxobj/sizeof/_il_dbgsizeof32_Target_32Bit/_il_dbgsizeof32_Target_32Bit.sh
JIT/Methodical/xxobj/sizeof/_il_relsizeof64_Target_32Bit/_il_relsizeof64_Target_32Bit.sh
Error message
Return code: 1
Raw output file: /root/helix/work/workitem/JIT/Methodical/Reports/JIT.Methodical/xxobj/sizeof/_il_dbgsizeof_Target_32Bit/_il_dbgsizeof_Target_32Bit.output.txt
Raw output:
BEGIN EXECUTION
/root/helix/work/correlation/corerun _il_dbgsizeof_Target_32Bit.dll ''
sizeof(RefComplexStruct) failed.
Expected: 100
Actual: 104
END EXECUTION - FAILED
Test Harness Exitcode is : 1
To run the test:
> set CORE_ROOT=/root/helix/work/correlation
> /root/helix/work/workitem/JIT/Methodical/xxobj/sizeof/_il_dbgsizeof_Target_32Bit/_il_dbgsizeof_Target_32Bit.sh
Expected: True
Actual: False
Stack trace
at JIT_Methodical._xxobj_sizeof__il_dbgsizeof_Target_32Bit__il_dbgsizeof_Target_32Bit_._xxobj_sizeof__il_dbgsizeof_Target_32Bit__il_dbgsizeof_Target_32Bit_sh() in /__w/1/s/artifacts/tests/coreclr/Linux.arm.Checked/TestWrappers/JIT.Methodical/JIT.Methodical.XUnitWrapper.cs:line 201318
Activity
BruceForstall commentedon Jun 5, 2020
CoreCLR Linux arm Checked @ (Ubuntu.1804.Arm32.Open)Ubuntu.1804.Armarch.OpenMany are also failing on Windows x86
BruceForstall commentedon Jun 5, 2020
@dotnet/jit-contrib
BruceForstall commentedon Jun 13, 2020
still failing:
https://dev.azure.com/dnceng/public/_build/results?buildId=685414&view=ms.vss-test-web.build-test-results-tab
BruceForstall commentedon Jun 15, 2020
The most important one (fails in most runs):
JIT\Methodical\xxobj\sizeof_il_dbgsizeof64_il_dbgsizeof64.cmd
AndyAyersMS commentedon Jun 16, 2020
Looking at
_il_dbgsizeof64, there is just one novel methodMainjitted and it is debug codegen. CI history shows failures starting around 6/1, so I build an x86 jit from just before then at 967d216 and diffed codegen forMainwith current master; they're identical.13 remaining items
jashook commentedon Jun 24, 2020
As @trylek has mentioned conditional compilation of managed tests using bitness is not supported as we will always build the managed components of the test on x64 OSX in CI.
Specifically below would not work in CI:
jashook commentedon Jun 24, 2020
Seems like this specific ilproj should just be deleted. There is already an ilproj for _il_dbgsizeof64_target_64bit and _il_dbgsizeof64_target_32bit
AndyAyersMS commentedon Jun 24, 2020
@trylek @jashook the x86 case seems to be in hand. Now trying to figure out what gets built and what gets run for the arm32 failures.
I have core dumps for these on arm32 from https://dev.azure.com/dnceng/public/_build/results?buildId=701494&view=ms.vss-test-web.build-test-results-tab&runId=21752388&resultId=107271&paneView=dotnet-dnceng.dnceng-build-release-tasks.helix-test-information-tab but haven't gotten to extracting the test IL just yet... my lldb/sos combo is not happy.
AndyAyersMS commentedon Jun 27, 2020
Ah, think I've perhaps finally figured this out.
Arm defines
FEATURE_64BIT_ALIGNMENTand so the class sizes on arm don't match those on x86.Before the changes in #37274 we were checking for arch=x86 when doing test version selection, and so on arm we were running the "64 bit" flavors of these tests. Now we're running the 32 bit flavors and they're failing.
That also explains why some other bit-size specific tests are ok and these particular tests are not.
I don't think these tests are all that valuable overall so instead of adding yet another category of exclusion/selection I'm tempted to just disable them for arm.
Thoughts?
BruceForstall commentedon Jun 27, 2020
So it looks like "x86" was right all along, and it wasn't a 32-bit/64-bit question.
@jashook @trylek @sdmaclea Is there an existing mechanism for determining a test run time whether a test should be run or not, based on architecture (or other criteria) (in the metadata; without adding that logic to the test)?
AndyAyersMS commentedon Jun 27, 2020
Seems a little odd that arm32 has the same class sizes as x64/arm64. Perhaps that was intentional, or perhaps a happy accident.
BruceForstall commentedon Jun 27, 2020
That's only because there are no pointers involved.
Actually, I'm thinking maybe these tests are valuable specifically for ARM, because ARM is the only platform that defines
FEATURE_64BIT_ALIGNMENTand has that particular alignment requirements, so protecting against regression there makes sense.BruceForstall commentedon Jun 27, 2020
Note that these tests were previously disabled for arm32 in
issues.targetswith the note "needs triage". After #37274, the issues.targets references were renamed for the new test names, however, the directories were written with incorrect casing. So presumably case-insensitive Windows has them disabled but case-sensitive Linux does not. Then #38186 disabled them for all platforms by adding another set of disabling text, this time with the correct casing (but note that the bad casing is still there).sdmaclea commentedon Jun 29, 2020
This is the mechanism to control whether the script is generated for a given platform
sdmaclea commentedon Jun 29, 2020
There was a class of tests which had 64-bit, 32-bit, and arm32-bit flavors of tests.
AndyAyersMS commentedon Jun 29, 2020
In these test cases, arm happens to have the same class sizes as arm64/x64.
So my proposal is to add another set of ILPROJs for arm, splitting the "target_32bit" cases into "target_x86" and "target_arm", and include the "64 bit" il sources for the latter. I won't rename the sources unless we feel it can add clarity.
Fix sizeof tests for arm and x86
Fix sizeof tests for arm and x86 (#38577)