Skip to content

[Perf] Linux/arm64: 1 Regression in Perf_HashCode on 11/12/2024 1:47:19 AM #109834

Closed
Closed
[Perf] Linux/arm64: 1 Regression in Perf_HashCode on 11/12/2024 1:47:19 AM#109834
Assignees
AndyAyersMS
Labels
arch-arm64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-linuxLinux OS (any supported distro)runtime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark
Milestone
10.0.0

Description

@performanceautofiler
performanceautofiler
bot
opened on Nov 14, 2024

Run Information

Name Value
Architecture arm64
OS ubuntu 22.04
Queue AmpereUbuntu
Baseline 80cb4583c9079cf052e6148b51b43590771b4353
Compare ab7e12334f07b72e67b897e9ae7a09d04fd1efb5
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_HashCode

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.59 μs 2.82 μs 1.09 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_HashCode*'
Details

System.Tests.Perf_HashCode.Add

ETL Files

Histogram

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. performanceautofiler
    added
    arch-arm64
    os-linuxLinux OS (any supported distro)
    runtime-coreclrspecific to the CoreCLR runtime
    untriagedNew issue has not been triaged by the area owner
    on Nov 14, 2024
  2. DrewScoggins
    transferred this issue fromdotnet/perf-autofiling-issueson Nov 14, 2024
  3. ghost added
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Nov 14, 2024
  4. DrewScoggins
    changed the title [-][Perf] Linux/arm64: 1 Regression on 11/12/2024 1:47:19 AM[/-] [+][Perf] Linux/arm64: 1 Regression in Perf_HashCode on 11/12/2024 1:47:19 AM[/+] on Nov 14, 2024
  5. DrewScoggins
    added
    tenet-performancePerformance related issue
    tenet-performance-benchmarksIssue from performance benchmark
    on Nov 14, 2024

  6. DrewScoggins commented on Nov 14, 2024

    @DrewScoggins
    DrewScoggins
    on Nov 14, 2024
    Member

    These seem to be the most likely commits.
    #109670
    #109661

  7. jeffschwMSFT
    added
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    on Nov 19, 2024

  8. dotnet-policy-service commented on Nov 19, 2024

    @dotnet-policy-service
    dotnet-policy-serviceBot
    on Nov 19, 2024 – with Dotnet Policy Service
    Contributor

    Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
    See info in area-owners.md if you want to be subscribed.

  9. LoopedBard3 commented on Nov 19, 2024

    @LoopedBard3
    LoopedBard3
    on Nov 19, 2024
    Member

    Related Regressions:

    Improvements:

  10. JulieLeeMSFT
    assigned
    AndyAyersMS
    on Nov 19, 2024
  11. JulieLeeMSFT
    removed
    untriagedNew issue has not been triaged by the area owner
    on Nov 19, 2024
  12. JulieLeeMSFT
    added this to the 10.0.0 milestone on Nov 19, 2024
  13. vcsjones
    removed
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Nov 20, 2024

  14. AndyAyersMS commented on Feb 15, 2025

    @AndyAyersMS
    AndyAyersMS
    on Feb 15, 2025
    Member

    Collated results

    Notes Recent Score Orig Score Ampere Ubuntu Tiger Ubuntu Tiger Windows Viper Ubuntu Viper Windows Benchmark
    1.61 1.32 1.61
    1.32    		 System.Memory.Span(Byte).SequenceCompareTo(Size: 4)
    1.35 1.29 1.35
    1.29    		 Microsoft.Extensions.Logging.LoggingOverhead.TwoArguments_DefineMessage_FilteredByLevel
    1.21 1.12 1.21
    1.12    		 Microsoft.Extensions.Primitives.StringSegmentBenchmark.Trim
    1.20 1.16 1.21
    1.16    		 1.39
    1.21    		 1.01
    1.10    		 System.Numerics.Tests.Perf_VectorOf(SByte).GetHashCodeBenchmark
    1.18 1.11 1.06
    1.08    		 1.32
    1.14    		 System.Runtime.Intrinsics.Tests.Perf_Vector128Of(SByte).GetHashCodeBenchmark
    1.17 1.19 1.21
    1.25    		 1.31
    1.19    		 1.01
    1.14    		 System.Numerics.Tests.Perf_VectorOf(Byte).GetHashCodeBenchmark
    1.15 1.17 1.15
    1.17    		 System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False))
    1.15 0.72 1.15
    0.72    		 System.Reflection.Invoke.Field_SetStatic_int
    1.15 1.17 1.15
    1.17    		 System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False))
    1.15 1.17 1.15
    1.17    		 System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, None, False))
    1.14 1.12 1.11
    1.12    		 1.29
    1.13    		 1.03
    1.11    		 System.Numerics.Tests.Perf_VectorOf(UInt16).GetHashCodeBenchmark
    1.12 1.09 1.09
    1.09    		 1.15
    1.09    		 System.Tests.Perf_HashCode.Add
    1.12 1.08 1.12
    1.08    		 System.Numerics.Tests.Perf_VectorOf(Int64).GetHashCodeBenchmark
    1.12 1.15 1.12
    1.15    		 System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Int16).GetHashCodeBenchmark
    1.11 1.15 1.08
    1.17    		 1.14
    1.14    		 System.Numerics.Tests.Perf_VectorOf(UInt32).GetHashCodeBenchmark
    1.10 1.55 1.10
    1.55    		 System.Memory.Span(Int32).IndexOfAnyFourValues(Size: 512)
    1.10 1.10 1.09
    1.09    		 1.12
    1.11    		 System.Numerics.Tests.Perf_VectorOf(Int32).GetHashCodeBenchmark
    1.10 0.91 1.10
    0.91    		 PerfLabTests.CastingPerf2.CastingPerf.FooObjIsDescendant
    1.10 1.14 1.01
    1.08    		 1.28
    1.17    		 1.02
    1.18    		 System.Runtime.Intrinsics.Tests.Perf_Vector128Of(Byte).GetHashCodeBenchmark
    1.09 1.12 1.08
    1.07    		 1.21
    1.16    		 1.00
    1.12    		 System.Numerics.Tests.Perf_VectorOf(Int16).GetHashCodeBenchmark
    1.09 1.10 1.09
    1.10    		 System.Net.Primitives.Tests.IPAddressPerformanceTests.TryFormat(address: 143.24.20.36)
    1.09 1.06 1.09
    1.06    		 System.Tests.Perf_Enum.InterpolateIntoStringBuilder_NonFlags(value: Saturday)
    1.09 1.05 1.09
    1.05    		 System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None)
    1.08 1.10 1.08
    1.10    		 System.Tests.Perf_Int32.ParseSpan(value: "-2147483648")
    1.07 1.12 1.07
    1.12    		 System.Collections.CreateAddAndClear(String).Queue(Size: 512)
    1.07 1.10 1.07
    1.10    		 System.Collections.IndexerSet(Int32).SortedDictionary(Size: 512)
    1.04 1.06 1.04
    1.06    		 System.Threading.Channels.Tests.BoundedChannelPerfTests.ReadAsyncThenWriteAsync
    1.04 1.22 1.04
    1.22    		 System.Collections.TryGetValueFalse(Int32, Int32).ConcurrentDictionary(Size: 512)
    1.03 1.17 1.03
    1.17    		 System.Collections.TryGetValueTrue(Int32, Int32).ConcurrentDictionary(Size: 512)
    1.03 1.06 1.03
    1.06    		 System.IO.Tests.Perf_StreamWriter.WriteCharArray(writeLength: 100)
    1.02 1.16 1.02
    1.16    		 System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,no escaping required,16)
    1.01 1.35 1.01
    1.35    		 System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 1234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456
    1.01 1.11 1.01
    1.11    		 System.Collections.Perf_Frozen(ReferenceType).TryGetValue_True(Count: 4)
    1.01 0.74 1.01
    0.74    		 Microsoft.Extensions.Logging.LoggingOverhead.NoArguments_DefineMessage_FilteredByLevel
    1.01 1.14 1.01
    1.14    		 System.Memory.SequenceReader.TryReadTo
    1.00 0.93 1.00
    0.93    		 System.Text.Tests.Perf_Encoding.GetBytes(size: 16, encName: "utf-8")
    1.00 1.10 1.00
    1.10    		 System.Net.Primitives.Tests.IPAddressPerformanceTests.Ctor_Span(address: [16, 32, 48, 64, 80, ...])
    1.00 1.11 1.00
    1.11    		 System.Linq.Tests.Perf_Enumerable.ToArray(input: IEnumerable)
    0.99 1.09 0.99
    1.09    		 System.Globalization.Tests.Perf_DateTimeCultureInfo.ToString(culturestring: fr)
    0.98 0.89 0.98
    0.89    		 System.Collections.Perf_Frozen(ReferenceType).TryGetValue_True(Count: 512)
    0.98 1.07 0.98
    1.07    		 System.Collections.Perf_Frozen(ReferenceType).TryGetValue_True(Count: 64)
    0.96 0.90 0.96
    0.90    		 System.IO.Tests.UnmanagedMemoryStreamTests.ReadAsArrays
    0.95 1.18 0.95
    1.18    		 System.Collections.IterateForEachNonGeneric(Int32).ArrayList(Size: 512)
    0.94 0.94 0.94
    0.94    		 System.Collections.Perf_SingleCharFrozenDictionary.TryGetValue_False_FrozenDictionary(Count: 10000)
    0.94 1.12 0.94
    1.12    		 System.Collections.TryGetValueTrue(SmallClass, SmallClass).FrozenDictionaryOptimized(Size: 512)
    0.94 0.89 0.86
    0.73    		 1.03
    1.09    		 System.Memory.Span(Int32).IndexOfAnyFiveValues(Size: 33)
    0.92 0.94 0.92
    0.94    		 System.Collections.TryAddGiventSize(String).Dictionary(Count: 512)
    0.91 1.09 0.91
    1.09    		 System.Numerics.Tensors.Tests.Perf_NumberTensorPrimitives(Int32).Negate(BufferLength: 128)
    0.91 0.80 0.91
    0.80    		 System.Collections.IterateForEachNonGeneric(String).ArrayList(Size: 512)
    0.91 0.91 0.91
    0.91    		 System.Collections.AddGivenSize(String).Dictionary(Size: 512)
    0.90 0.80 0.90
    0.80    		 System.Tests.Perf_Int32.ParseSpan(value: "4")
    0.90 0.92 0.83
    0.79    		 0.98
    1.08    		 System.Memory.Span(Int32).IndexOfAnyFourValues(Size: 33)
    0.88 0.90 0.88
    0.90    		 System.Collections.TryGetValueTrue(SmallClass, SmallClass).ConcurrentDictionary(Size: 512)
    0.84 0.83 0.84
    0.83    		 System.Text.Json.Tests.Perf_Get.GetDateTime
    0.81 0.81 0.81
    0.81    		 System.Tests.Perf_Int64.ParseSpan(value: "9223372036854775807")
    0.79 0.81 0.79
    0.81    		 System.Linq.Tests.Perf_Enumerable.SelectToList(input: IEnumerable)
    0.79 0.79 0.79
    0.79    		 Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger: False, CaptureScopes: True)
    0.77 0.74 0.77
    0.74    		 Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger: True, CaptureScopes: True)
    0.77 0.76 0.77
    0.76    		 Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger: False, CaptureScopes: False)
    0.76 0.73 0.76
    0.73    		 Microsoft.Extensions.Logging.ScopesOverheadBenchmark.FilteredByLevel(HasISupportLoggingScopeLogger: True, CaptureScopes: False)
    0.71 0.77 0.71
    0.77    		 System.Memory.Span(Int32).IndexOfAnyFourValues(Size: 4)
    0.69 0.73 0.69
    0.73    		 Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Valid
    0.45 0.45 0.45
    0.45    		 System.Tests.Perf_UInt32.TryFormat(value: 0)

  15. AndyAyersMS commented on May 24, 2025

    @AndyAyersMS
    AndyAyersMS
    on May 24, 2025
    Member

    These largely seem resolved via #115119

  16. AndyAyersMS
    closed this as completedon May 24, 2025
  17. github-actions
    locked and limited conversation to collaborators on Jun 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

arch-arm64area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMIos-linuxLinux OS (any supported distro)runtime-coreclrspecific to the CoreCLR runtimetenet-performancePerformance related issuetenet-performance-benchmarksIssue from performance benchmark

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions