Skip to content

Perf regressions in Perf_Matrix benchmarks #59415

Closed
Closed
Perf regressions in Perf_Matrix benchmarks#59415
Assignees
kunalspathakalexcovington
Labels
arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-linuxLinux OS (any supported distro)tenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark
Milestone
7.0.0

Description

@performanceautofiler
performanceautofiler
bot
opened on Sep 21, 2021

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10b2f7c858934c27c56aa13845cb47064bfe87e9
Compare a842e7a1dc6c241c928c6291411393cdb2516608
Diff Diff

Regressions in System.IO.Tests.BinaryWriterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteAsciiChar - Duration of single invocation 3.44 ns 4.71 ns 1.37 0.17 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.BinaryWriterTests*'
Details

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterTests.WriteAsciiChar

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS ubuntu 18.04
Baseline 10b2f7c858934c27c56aa13845cb47064bfe87e9
Compare a842e7a1dc6c241c928c6291411393cdb2516608
Diff Diff

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InequalityOperatorBenchmark - Duration of single invocation 6.33 ns 12.21 ns 1.93 0.07 True
LerpBenchmark - Duration of single invocation 8.16 ns 13.30 ns 1.63 0.04 True
NegationOperatorBenchmark - Duration of single invocation 6.35 ns 10.87 ns 1.71 0.10 False
AddBenchmark - Duration of single invocation 8.96 ns 15.89 ns 1.77 0.08 True

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'
Details

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Matrix3x2.InequalityOperatorBenchmark

System.Numerics.Tests.Perf_Matrix3x2.LerpBenchmark

System.Numerics.Tests.Perf_Matrix3x2.NegationOperatorBenchmark

System.Numerics.Tests.Perf_Matrix3x2.AddBenchmark

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10b2f7c858934c27c56aa13845cb47064bfe87e9
Compare a842e7a1dc6c241c928c6291411393cdb2516608
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 130.22 ns 140.77 ns 1.08 0.05 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Nullable&lt;DateTimeOffset&gt;&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Nullable<DateTimeOffset>>.DeserializeFromUtf8Bytes

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10b2f7c858934c27c56aa13845cb47064bfe87e9
Compare a842e7a1dc6c241c928c6291411393cdb2516608
Diff Diff

Regressions in System.Collections.Sort<BigStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array_ComparerStruct - Duration of single invocation 33.94 μs 46.92 μs 1.38 0.02 True
Array_Comparison - Duration of single invocation 30.67 μs 45.16 μs 1.47 0.03 True
Array_ComparerClass - Duration of single invocation 31.22 μs 44.95 μs 1.44 0.07 True

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;BigStruct&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<BigStruct>.Array_ComparerStruct(Size: 512)

System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512)

System.Collections.Sort<BigStruct>.Array_ComparerClass(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10b2f7c858934c27c56aa13845cb47064bfe87e9
Compare a842e7a1dc6c241c928c6291411393cdb2516608
Diff Diff

Regressions in System.Collections.IterateForEachNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 3.61 μs 3.85 μs 1.07 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEachNonGeneric<Int32>.Queue(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10b2f7c858934c27c56aa13845cb47064bfe87e9
Compare a842e7a1dc6c241c928c6291411393cdb2516608
Diff Diff

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ConcurrentStack - Duration of single invocation 2.49 μs 2.69 μs 1.08 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ConcurrentStack(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10b2f7c858934c27c56aa13845cb47064bfe87e9
Compare a842e7a1dc6c241c928c6291411393cdb2516608
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedSet - Duration of single invocation 28.60 μs 31.25 μs 1.09 0.00 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.ImmutableSortedSet(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10b2f7c858934c27c56aa13845cb47064bfe87e9
Compare a842e7a1dc6c241c928c6291411393cdb2516608
Diff Diff

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TransformNormalByMatrix3x2Benchmark - Duration of single invocation 1.99 ns 8.09 ns 4.06 0.14 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_Vector2*'
Details

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_Vector2.TransformNormalByMatrix3x2Benchmark

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10b2f7c858934c27c56aa13845cb47064bfe87e9
Compare a842e7a1dc6c241c928c6291411393cdb2516608
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SequenceCompareTo - Duration of single invocation 9.92 ns 12.15 ns 1.23 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.SequenceCompareTo(Size: 512)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS ubuntu 18.04
Baseline 10b2f7c858934c27c56aa13845cb47064bfe87e9
Compare a842e7a1dc6c241c928c6291411393cdb2516608
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Uri_IsMatch - Duration of single invocation 132.85 ns 143.51 ns 1.08 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
Details

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: IgnoreCase, Compiled)

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. kunalspathak
    changed the title [-][Perf] Changes at 9/18/2021 1:42:06 AM[/-] [+]Perf regressions in Perf_Matrix benchmarks[/+] on Sep 21, 2021
  2. kunalspathak
    transferred this issue fromdotnet/perf-autofiling-issueson Sep 21, 2021
  4. ghost added
    untriagedNew issue has not been triaged by the area owner
    on Sep 21, 2021
  5. kunalspathak
    added
    arch-x64
    os-linuxLinux OS (any supported distro)
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    and removed
    untriagedNew issue has not been triaged by the area owner
    on Sep 21, 2021

  7. kunalspathak commented on Sep 21, 2021

    @kunalspathak
    kunalspathak
    on Sep 21, 2021
    Contributor

    Introduced in #55604

  8. kunalspathak commented on Sep 21, 2021

    @kunalspathak
    kunalspathak
    on Sep 21, 2021
    Contributor

    @alexcovington

  9. kunalspathak commented on Sep 21, 2021

    @kunalspathak
    kunalspathak
    on Sep 21, 2021
    Contributor

    windows/x64 regression - dotnet/perf-autofiling-issues#1487

  10. DrewScoggins
    added
    tenet-performancePerformance related issue
    tenet-performance-benchmarksIssue from performance benchmark
    on Sep 21, 2021

  11. kunalspathak commented on Sep 21, 2021

    @kunalspathak
    kunalspathak
    on Sep 21, 2021
    Contributor

    windows/x86 regression - dotnet/perf-autofiling-issues#1492

  12. 9 remaining items

  13. JulieLeeMSFT
    modified the milestones: 6.0.0, 7.0.0 on Sep 24, 2021

  14. JulieLeeMSFT commented on Sep 24, 2021

    @JulieLeeMSFT
    JulieLeeMSFT
    on Sep 24, 2021
    Member

    Moving this to .NET 7 since we passed .NET 6 RC2 snap.

  15. adamsitnik commented on Feb 11, 2022

    @adamsitnik
    adamsitnik
    on Feb 11, 2022
    Member

    I confirm that System.Numerics.Tests.Perf_Matrix4x4.NegationOperatorBenchmark regression is still a thing for 7.0:

    Result Base Diff Ratio Operating System Bit Processor Name
    Slower 4.58 7.99 0.57 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
    Slower 3.62 6.99 0.52 Windows 11 X64 AMD Ryzen 9 5900X
    Slower 5.80 7.89 0.73 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
    Same 11.58 10.50 1.10 Windows 11 X64 Intel Core i5-4300U CPU 1.90GHz (Haswell)
    Same 5.58 7.01 0.80 Windows 10 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
    Same 5.20 6.54 0.80 Windows 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
    Same 5.06 7.08 0.71 Windows 11 X64 Intel Core i9-9900T CPU 2.10GHz
    Slower 8.45 10.77 0.78 Windows 11 X64 Unknown processor
    Same 8.83 9.41 0.94 Windows 11 X64 Unknown processor
    Slower 4.28 7.33 0.58 ubuntu 20.04 X64 AMD Ryzen 9 5900X
    Same 7.32 8.63 0.85 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
    Same 8.10 8.35 0.97 centos 7 X64 Intel Xeon CPU E5530 2.40GHz
    Same 7.44 8.73 0.85 ubuntu 18.04 X64 Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
    Same 6.03 8.02 0.75 alpine 3.13 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
    Slower 5.95 18.62 0.32 ubuntu 18.04 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
    Slower 5.95 8.28 0.72 ubuntu 20.04 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
    Same 11.59 11.56 1.00 Windows 10 Arm64 Microsoft SQ1 3.0 GHz
    Slower 6.18 38.48 0.16 Windows 10 X86 Intel Xeon CPU E5-1650 v4 3.60GHz
    Slower 27.73 32.56 0.85 Windows 10 Arm Microsoft SQ1 3.0 GHz
    Slower 7.14 11.38 0.63 macOS Big Sur 11.6.3 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell)
    Slower 6.26 10.57 0.59 macOS Big Sur 11.4 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

  16. adamsitnik commented on Feb 11, 2022

    @adamsitnik
    adamsitnik
    on Feb 11, 2022
    Member

    Same goes for System.Collections.Sort<BigStruct>.Array_Comparison(Size: 512) but it's more hardware-specific

    Result Base Diff Ratio Operating System Bit Processor Name
    Same 24599.98 23124.16 1.06 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
    Faster 22267.59 12209.82 1.82 Windows 11 X64 AMD Ryzen 9 5900X
    Same 36100.30 33451.88 1.08 Windows 10 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
    Slower 47183.04 66218.78 0.71 Windows 11 X64 Intel Core i5-4300U CPU 1.90GHz (Haswell)
    Faster 40294.54 28493.22 1.41 Windows 10 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
    Same 30304.77 27743.34 1.09 Windows 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
    Same 32046.90 32203.04 1.00 Windows 11 X64 Intel Core i9-9900T CPU 2.10GHz
    Same 46669.91 43923.64 1.06 Windows 11 X64 Unknown processor
    Slower 42951.49 50995.70 0.84 Windows 11 X64 Unknown processor
    Slower 14197.04 37533.26 0.38 ubuntu 20.04 X64 AMD Ryzen 9 5900X
    Slower 38299.66 55484.61 0.69 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
    Same 72365.12 70844.77 1.02 centos 7 X64 Intel Xeon CPU E5530 2.40GHz
    Same 48875.84 52673.91 0.93 ubuntu 18.04 X64 Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
    Slower 36496.10 52447.27 0.70 alpine 3.13 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
    Slower 36711.46 52210.52 0.70 ubuntu 18.04 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
    Slower 37068.52 55272.38 0.67 ubuntu 20.04 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
    Slower 34329.86 49660.84 0.69 ubuntu 20.04 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
    Same 57540.97 56836.27 1.01 ubuntu 20.04 Arm64 Unknown processor
    Same 58085.44 58366.52 1.00 Windows 10 Arm64 Microsoft SQ1 3.0 GHz
    Same 59580.26 57693.12 1.03 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
    Slower 26869.67 50771.14 0.53 Windows 11 X86 AMD Ryzen Threadripper PRO 3945WX 12-Cores
    Slower 39058.55 68290.00 0.57 Windows 10 X86 Intel Xeon CPU E5-1650 v4 3.60GHz
    Same 82255.74 82176.45 1.00 Windows 10 Arm Microsoft SQ1 3.0 GHz
    Slower 45699.41 71023.83 0.64 macOS Big Sur 11.6.3 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell)
    Slower 42134.09 68595.79 0.61 macOS Big Sur 11.4 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)

  17. kunalspathak commented on Feb 11, 2022

    @kunalspathak
    kunalspathak
    on Feb 11, 2022
    Contributor

    @alexcovington - do you mind taking a look at fresh numbers and see it with and without #59497 ?

  18. alexcovington commented on Feb 14, 2022

    @alexcovington
    alexcovington
    on Feb 14, 2022
    Contributor

    @kunalspathak Sorry for the delay. I'm not able to reproduce the regression on either my 3600 or 5900X systems.

    I pulled the latest main and built two versions, one with and without #59497.

    Base commit is 77d6833.

    To remove #59497 to use as a comparison, I reverted commit b82e838:

    git revert --strategy resolve b82e8389715b275

    Here are my numbers on my Ryzen 5 3600 system (the results are similar on my 5900X system and don't see much change either):

    System.Collections.Sort.Array_Comparison 
    BenchmarkDotNet=v0.13.1.1694-nightly, OS=Windows 10 (10.0.19044.1466/21H2/November2021Update)
AMD Ryzen 5 3600, 1 CPU, 12 logical and 6 physical cores
.NET SDK=7.0.100-preview.2.22111.4
  [Host]     : .NET 7.0.0 (7.0.22.10302), X64 RyuJIT
  Job-VNDCGQ : .NET 7.0.0 (42.42.42.42424), X64 RyuJIT
  Job-ZZYUDP : .NET 7.0.0 (42.42.42.42424), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000  Arguments=/p:DebugType=portable,-bl:benchmarkdotnet.binlog  InvocationCount=5000  
IterationCount=30  IterationTime=250.0000 ms  LaunchCount=5  
MaxIterationCount=20  MinIterationCount=15  MinWarmupIterationCount=6  
UnrollFactor=1  WarmupCount=-1  

|           Method |        Job |                                                                                                      Toolchain | Size |     Mean |    Error |   StdDev |   Median |      Min |      Max | Ratio | RatioSD | Allocated | Alloc Ratio |
|----------------- |----------- |--------------------------------------------------------------------------------------------------------------- |----- |---------:|---------:|---------:|---------:|---------:|---------:|------:|--------:|----------:|------------:|
| Array_Comparison | Job-VNDCGQ | \runtime-base\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\7.0.0\corerun.exe |  512 | 28.25 μs | 0.191 μs | 0.679 μs | 28.32 μs | 26.85 μs | 30.38 μs |  1.00 |    0.00 |         - |          NA |
| Array_Comparison | Job-ZZYUDP | \runtime-diff\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\7.0.0\corerun.exe |  512 | 27.84 μs | 0.106 μs | 0.380 μs | 27.80 μs | 26.92 μs | 29.05 μs |  0.99 |    0.03 |         - |          NA |
    System.Numerics.Tests.Perf_Matrix4x4.NegationOperatorBenchmark 
    BenchmarkDotNet=v0.13.1.1694-nightly, OS=Windows 10 (10.0.19044.1466/21H2/November2021Update)
AMD Ryzen 5 3600, 1 CPU, 12 logical and 6 physical cores
.NET SDK=7.0.100-preview.2.22111.4
  [Host]     : .NET 7.0.0 (7.0.22.10302), X64 RyuJIT
  Job-HRURMT : .NET 7.0.0 (42.42.42.42424), X64 RyuJIT
  Job-KNGSMF : .NET 7.0.0 (42.42.42.42424), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000  Arguments=/p:DebugType=portable,-bl:benchmarkdotnet.binlog  IterationCount=30  
IterationTime=250.0000 ms  LaunchCount=5  MaxIterationCount=20  
MinIterationCount=15  WarmupCount=1  

|                    Method |        Job |                                                                                                      Toolchain |     Mean |     Error |    StdDev |   Median |      Min |      Max | Ratio | RatioSD | Allocated | Alloc Ratio |
|-------------------------- |----------- |--------------------------------------------------------------------------------------------------------------- |---------:|----------:|----------:|---------:|---------:|---------:|------:|--------:|----------:|------------:|
| NegationOperatorBenchmark | Job-HRURMT | \runtime-base\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\7.0.0\corerun.exe | 9.258 ns | 0.1898 ns | 0.6657 ns | 9.368 ns | 8.492 ns | 10.24 ns |  1.00 |    0.00 |         - |          NA |
| NegationOperatorBenchmark | Job-KNGSMF | \runtime-diff\artifacts\bin\testhost\net7.0-windows-Release-x64\shared\Microsoft.NETCore.App\7.0.0\corerun.exe | 9.318 ns | 0.1338 ns | 0.4709 ns | 9.373 ns | 8.502 ns | 10.02 ns |  1.01 |    0.06 |         - |          NA |

    Please let me know if I can clarify anything.

  19. JulieLeeMSFT
    assigned
    kunalspathak
    on Jun 3, 2022

  20. kunalspathak commented on Jun 4, 2022

    @kunalspathak
    kunalspathak
    on Jun 4, 2022
    Contributor

    Perf_Matrix3x2 seems to have spikes historically:

    image

    image

    image

  21. kunalspathak
    closed this as completedon Jun 4, 2022
  22. ghost locked as resolved and limited conversation to collaborators on Jul 4, 2022

  23. jozkee commented on Oct 14, 2022

    @jozkee
    jozkee
    on Oct 14, 2022
    Member

    A regression for System.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark showed up in the 6.0 vs 7.0-rc2 report, potentially x64 specific. The historical data only goes back until october 2021 so we are unable to determine if this is just noise nor to see what might've caused it.

    Details

    System.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark

    Result Ratio Alloc Delta Operating System Bit Processor Name Modality
    Faster 1.15 +0 ubuntu 18.04 Arm64 Unknown processor
    Slower 0.67 +0 Windows 11 Arm64 Unknown processor
    Faster 1.23 +0 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
    Faster 1.17 +0 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
    Noise - +0 macOS Monterey 12.6 Arm64 Apple M1
    Noise - +0 macOS Monterey 12.6 Arm64 Apple M1 Max
    Same 0.96 +0 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
    Slower 0.73 +0 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
    Slower 0.63 +0 Windows 11 X64 AMD Ryzen 9 5900X
    Slower 0.67 +0 Windows 11 X64 AMD Ryzen 9 7950X
    Same 0.91 +0 Windows 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) several?
    Same 0.92 +0 debian 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
    Slower 0.29 +0 ubuntu 18.04 X64 AMD Ryzen 9 5900X
    Slower 0.31 +0 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
    Slower 0.67 +0 ubuntu 20.04 X64 AMD Ryzen 9 5900X
    Same 0.92 +0 ubuntu 20.04 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
    Same 1.06 +0 ubuntu 20.04 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
    Same 0.92 +0 macOS Big Sur 11.7 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell)
    Noise - +0 macOS Monterey 12.6 X64 Intel Core i7-4870HQ CPU 2.50GHz (Haswell)
  24. jozkee
    reopened this on Oct 14, 2022

  25. jozkee commented on Oct 14, 2022

    @jozkee
    jozkee
    on Oct 14, 2022
    Member

    Actually, this can be closed as duplicate of #65191 which is tracking System.Numerics.Tests.Perf_Matrix4x4.NegateBenchmark and other regressions in the same area.

  26. jozkee
    closed this as completedon Oct 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-linuxLinux OS (any supported distro)tenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions