Skip to content

[Perf] Linux/x64: 35 Regressions on 9/10/2024 6:39:57 AM #107925

Closed
Closed
[Perf] Linux/x64: 35 Regressions on 9/10/2024 6:39:57 AM#107925
Assignees
amanasifkhalid
Labels
arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-linuxLinux OS (any supported distro)runtime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark
Milestone
10.0.0

Description

@performanceautofiler
performanceautofiler
bot
opened on Sep 17, 2024

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline c5340807568ba33edc74f61ed05dafe8e030362e
Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.74 ns 20.06 ns 1.07 0.02 False
11.87 ns 12.89 ns 1.09 0.03 False
19.10 ns 20.73 ns 1.08 0.03 False
22.71 ns 26.15 ns 1.15 0.14 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'
Details

System.Tests.Perf_Int64.TryParse(value: "9223372036854775807")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int64.TryParseSpan(value: "12345")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int64.Parse(value: "9223372036854775807")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int64.ToString(value: -9223372036854775808)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline c5340807568ba33edc74f61ed05dafe8e030362e
Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.44 ns 11.35 ns 1.20 0.04 False
14.22 ns 15.86 ns 1.11 0.03 False
9.15 ns 10.24 ns 1.12 0.07 False
7.65 ns 8.70 ns 1.14 0.05 False
13.01 ns 14.93 ns 1.15 0.03 False
10.49 ns 11.59 ns 1.11 0.09 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'
Details

System.Tests.Perf_Int32.ParseHex(value: "3039")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "2147483647")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "12345")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "4")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "2147483647")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "12345")

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline c5340807568ba33edc74f61ed05dafe8e030362e
Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
87.00 μs 94.14 μs 1.08 0.02 False
10.56 ms 12.89 ms 1.22 0.11 False
64.31 μs 69.00 μs 1.07 0.02 False
61.60 μs 66.91 μs 1.09 0.02 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'
Details

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: Compiled)

ETL Files

Histogram

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: None)

ETL Files

Histogram

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: Compiled)

ETL Files

Histogram

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: Compiled)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline c5340807568ba33edc74f61ed05dafe8e030362e
Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.11 ms 3.40 ms 1.10 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'
Details

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: Compiled)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline c5340807568ba33edc74f61ed05dafe8e030362e
Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Uri

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
38.30 ns 43.11 ns 1.13 0.03 False
5.40 μs 6.24 μs 1.16 0.10 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Uri*'
Details

System.Tests.Perf_Uri.UnescapeDataString(input: "abc%20def%20ghi%20")

ETL Files

Histogram

JIT Disasms

System.Tests.Perf_Uri.EscapeDataString(input: "{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{")

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline c5340807568ba33edc74f61ed05dafe8e030362e
Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
379.48 ns 404.59 ns 1.07 0.00 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'
Details

System.Collections.IterateForEach<Int32>.HashSet(Size: 512)

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline c5340807568ba33edc74f61ed05dafe8e030362e
Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Burgers

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
178.66 ms 223.19 ms 1.25 0.04 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Burgers*'
Details

Burgers.Test1

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline c5340807568ba33edc74f61ed05dafe8e030362e
Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
633.40 ns 713.53 ns 1.13 0.02 False
634.02 ns 704.45 ns 1.11 0.02 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'
Details

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))

ETL Files

Histogram

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False))

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture x64
OS ubuntu 22.04
Queue TigerUbuntu
Baseline c5340807568ba33edc74f61ed05dafe8e030362e
Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
103.60 ns 112.33 ns 1.08 0.05 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'
Details

System.Tests.Perf_Enum.Parse_Flags(text: "Red, Orange, Yellow, Green, Blue")

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. performanceautofiler
    added
    arch-x64
    os-linuxLinux OS (any supported distro)
    runtime-coreclrspecific to the CoreCLR runtime
    untriagedNew issue has not been triaged by the area owner
    on Sep 17, 2024

  2. performanceautofiler commented on Sep 17, 2024

    @performanceautofiler
    performanceautofilerBot
    on Sep 17, 2024 – with PerformanceAutoFiler
    Author

    Run Information

    Name Value
    Architecture x64
    OS ubuntu 22.04
    Queue TigerUbuntu
    Baseline c5340807568ba33edc74f61ed05dafe8e030362e
    Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    63.82 ns 72.00 ns 1.13 0.08 False
    399.82 ns 450.44 ns 1.13 0.17 False
    63.34 ns 71.69 ns 1.13 0.05 True
    115.80 ns 132.32 ns 1.14 0.03 False
    118.36 ns 133.88 ns 1.13 0.03 False
    115.76 ns 131.53 ns 1.14 0.04 False
    117.11 ns 132.19 ns 1.13 0.05 False

    graph
    graph
    graph
    graph
    graph
    graph
    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'
    Details

    System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: Compiled)

    ETL Files

    Histogram

    JIT Disasms

    System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: None)

    ETL Files

    Histogram

    JIT Disasms

    System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: IgnoreCase, Compiled)

    ETL Files

    Histogram

    JIT Disasms

    System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: Compiled)

    ETL Files

    Histogram

    JIT Disasms

    System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: Compiled)

    ETL Files

    Histogram

    JIT Disasms

    System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: IgnoreCase, Compiled)

    ETL Files

    Histogram

    JIT Disasms

    System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: IgnoreCase, Compiled)

    ETL Files

    Histogram

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Name Value
    Architecture x64
    OS ubuntu 22.04
    Queue TigerUbuntu
    Baseline c5340807568ba33edc74f61ed05dafe8e030362e
    Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    183.08 ns 221.95 ns 1.21 0.03 False
    55.72 ns 60.99 ns 1.09 0.33 False

    graph
    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'
    Details

    System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Dequeue_And_Enqueue(Size: 10)

    ETL Files

    Histogram

    JIT Disasms

    System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.K_Max_Elements(Size: 10)

    ETL Files

    Histogram

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Name Value
    Architecture x64
    OS ubuntu 22.04
    Queue TigerUbuntu
    Baseline c5340807568ba33edc74f61ed05dafe8e030362e
    Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in Benchstone.BenchI.EightQueens

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    1.59 μs 1.70 μs 1.07 0.01 False

    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.EightQueens*'
    Details

    Benchstone.BenchI.EightQueens.Test

    ETL Files

    Histogram

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Name Value
    Architecture x64
    OS ubuntu 22.04
    Queue TigerUbuntu
    Baseline c5340807568ba33edc74f61ed05dafe8e030362e
    Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in System.Memory.ReadOnlySpan

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    18.46 ns 20.69 ns 1.12 0.04 False
    180.27 ns 240.76 ns 1.34 0.00 True

    graph
    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'
    Details

    System.Memory.ReadOnlySpan.IndexOfString(input: "ABCDE", value: "c", comparisonType: InvariantCultureIgnoreCase)

    ETL Files

    Histogram

    JIT Disasms

    System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: InvariantCultureIgnoreCase)

    ETL Files

    Histogram

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

    Run Information

    Name Value
    Architecture x64
    OS ubuntu 22.04
    Queue TigerUbuntu
    Baseline c5340807568ba33edc74f61ed05dafe8e030362e
    Compare ad6c18f6874b3382a8f96a6021616fa71c918aeb
    Diff Diff
    Configs CompilationMode:tiered, RunKind:micro

    Regressions in Benchstone.BenchI.BubbleSort

    Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
    8.77 μs 11.87 μs 1.35 0.02 True

    graph
    Test Report

    Repro

    General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

    git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BubbleSort*'
    Details

    Benchstone.BenchI.BubbleSort.Test

    ETL Files

    Histogram

    JIT Disasms

    Docs

    Profiling workflow for dotnet/runtime repository
    Benchmarking workflow for dotnet/runtime repository

  3. LoopedBard3
    transferred this issue fromdotnet/perf-autofiling-issueson Sep 17, 2024
  4. ghost added
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Sep 17, 2024

  5. LoopedBard3 commented on Sep 17, 2024

    @LoopedBard3
    LoopedBard3
    on Sep 17, 2024
    Member

    @amanasifkhalid Likely caused by: #107483
    Related Regressions:
    Windows x64: dotnet/perf-autofiling-issues#41295, dotnet/perf-autofiling-issues#41283
    More regressions in PR.

    Seeing some improvements:
    Windows x64: dotnet/perf-autofiling-issues#41302

  6. LoopedBard3
    added
    tenet-performancePerformance related issue
    tenet-performance-benchmarksIssue from performance benchmark
    on Sep 17, 2024
  7. amanasifkhalid
    self-assigned this
    on Sep 17, 2024
  8. jeffschwMSFT
    added
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    on Sep 18, 2024

  9. dotnet-policy-service commented on Sep 18, 2024

    @dotnet-policy-service
    dotnet-policy-serviceBot
    on Sep 18, 2024 – with Dotnet Policy Service
    Contributor

    Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
    See info in area-owners.md if you want to be subscribed.

  10. JulieLeeMSFT
    removed
    untriagedNew issue has not been triaged by the area owner
    on Sep 19, 2024
  11. JulieLeeMSFT
    added this to the 9.0.0 milestone on Sep 19, 2024
  12. amanasifkhalid
    modified the milestones: 9.0.0, 10.0.0 on Sep 20, 2024
  13. vcsjones
    removed
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Sep 25, 2024

  14. amanasifkhalid commented on Apr 28, 2025

    @amanasifkhalid
    amanasifkhalid
    on Apr 28, 2025
    Contributor

    Regression table, with Intel machines filtered out to ignore JCC errata:

    Notes Recent Score Orig Score arm64 Ubuntu amd64 Ubuntu amd64 Windows Benchmark
    1.42 1.25 1.42
    1.25    		 Benchstone.BenchI.BubbleSort.Test
    1.12 1.51 1.12
    1.51    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: Compiled)
    1.00 1.20 1.00
    1.20    		 System.Collections.IterateForEach(Int32).HashSet(Size: 512)
    0.96 1.16 0.90
    1.11    		 1.00
    1.19    		 1.00
    1.19    		 System.Collections.IterateForEach(Int32).Dictionary(Size: 512)
    0.79 1.07 0.79
    1.07    		 System.Tests.Perf_Uri.EscapeDataString(input: "{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{
    0.73 1.15 0.73
    1.15    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "aei", Options: None)

    The Regex test is quite noisy, though it has a clear downward trajectory. The bubble sort test improved from 3-opt work, and then regressed from profile maintenance work (probably #111498), and then was further regressed by #114821. Since there are newer issues tracking the latter regression, and the former regression doesn't seem actionable as we don't plan to further churn our layout strategy, I think we ought to close this.

  15. amanasifkhalid
    closed this as completedon Apr 28, 2025
  16. github-actions
    locked and limited conversation to collaborators on May 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

arch-x64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-linuxLinux OS (any supported distro)runtime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions