[Perf] Regressions from stp/ldp in prolog on 5/3/2023 7:05:50 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	4ead8078eef158f466b693b2a395b811aa184a1c
Compare	f510c5ab2d8a9b94b84f63ff9d2d8318243e390f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Devirtualization.EqualityComparer

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ValueTupleCompareCached - Duration of single invocation	2.75 ns	4.96 ns	1.80	0.58	False
ValueTupleCompareWrapped - Duration of single invocation	2.60 ns	4.37 ns	1.68	0.53	False
ValueTupleCompareNoOpt - Duration of single invocation	10.82 ns	15.46 ns	1.43	0.22	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Devirtualization.EqualityComparer*'

Details

Payloads

Baseline
Compare

Histogram

Devirtualization.EqualityComparer.ValueTupleCompareCached

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.958335409439925 > 2.8636749530062544.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/3/2023 1:18:01 AM, 5/10/2023 10:36:48 PM falls between 5/2/2023 4:25:00 AM and 5/10/2023 10:36:48 PM.
IsRegressionStdDev: Marked as regression because -10.935309328648138 (T) = (0 -4.744384756963931) / Math.Sqrt((0.31194613751441336 / (11)) + (0.06960278016735186 / (24))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (24) - 2, .025) and -0.6877906950679686 = (2.811003029479832 - 4.744384756963931) / 2.811003029479832 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Devirtualization.EqualityComparer.ValueTupleCompareWrapped

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.372593836734031 > 2.6918711047385973.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/3/2023 1:18:01 AM, 5/10/2023 10:36:48 PM falls between 5/2/2023 4:25:00 AM and 5/10/2023 10:36:48 PM.
IsRegressionStdDev: Marked as regression because -26.91975324410644 (T) = (0 -4.663320013382269) / Math.Sqrt((0.05710386793103032 / (11)) + (0.03794529349665089 / (24))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (24) - 2, .025) and -0.9049635551497784 = (2.4479838476571874 - 4.663320013382269) / 2.4479838476571874 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Devirtualization.EqualityComparer.ValueTupleCompareNoOpt

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.463634381678597 > 11.352321919604611.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/3/2023 1:18:01 AM, 5/10/2023 10:36:48 PM falls between 5/2/2023 4:25:00 AM and 5/10/2023 10:36:48 PM.
IsRegressionStdDev: Marked as regression because -42.04969701146113 (T) = (0 -15.399074055936403) / Math.Sqrt((0.05914737499588529 / (11)) + (0.15781869927594283 / (23))) is less than -2.03693334345674 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (23) - 2, .025) and -0.4328493000111299 = (10.747169333032293 - 15.399074055936403) / 10.747169333032293 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	4ead8078eef158f466b693b2a395b811aa184a1c
Compare	f510c5ab2d8a9b94b84f63ff9d2d8318243e390f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in BenchmarksGame.KNucleotide_1

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	308.75 ms	346.34 ms	1.12	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.KNucleotide_1*'

Details

Payloads

Baseline
Compare

Histogram

BenchmarksGame.KNucleotide_1.RunBench

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 346.3394214285714 > 325.42319299999997.
IsChangePoint: Marked as a change because one of 5/3/2023 1:18:01 AM, 5/10/2023 10:36:48 PM falls between 5/2/2023 4:25:00 AM and 5/10/2023 10:36:48 PM.
IsRegressionStdDev: Marked as regression because -41.15592177236883 (T) = (0 -347074934.57875454) / Math.Sqrt((6347502920928.425 / (11)) + (4057659750735.734 / (24))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (24) - 2, .025) and -0.1141147713811062 = (311525296.58008665 - 347074934.57875454) / 311525296.58008665 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	4ead8078eef158f466b693b2a395b811aa184a1c
Compare	f510c5ab2d8a9b94b84f63ff9d2d8318243e390f
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IndexerSet<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Span - Duration of single invocation	393.35 ns	418.47 ns	1.06	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet<String>*'

Details

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<String>.Span(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 418.4681186408938 > 412.9108186412027.
IsChangePoint: Marked as a change because one of 5/3/2023 1:18:01 AM, 5/10/2023 10:36:48 PM falls between 5/2/2023 4:25:00 AM and 5/10/2023 10:36:48 PM.
IsRegressionStdDev: Marked as regression because -104.26852350283777 (T) = (0 -419.1804344002914) / Math.Sqrt((0.3006893229336772 / (11)) + (0.8201290901603625 / (24))) is less than -2.034515297446192 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (24) - 2, .025) and -0.06574606719749097 = (393.32111776173605 - 419.1804344002914) / 393.32111776173605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[EgorBo]

We suspect #85657