Skip to content

[Perf] Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark #70503

Description

@performanceautofiler

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline b09ae1ad36858f526b68a005c6f6b2cf8f7ab4b6
Compare 625a9cdd4437a414f1fda220f5bacba101f209c8
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOf - Duration of single invocation 2.75 ns 4.88 ns 1.78 0.23 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'
Details

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.87501562668358 > 3.1972914082750723.
IsChangePoint: Marked as a change because one of 5/20/2022 7:31:10 PM, 6/2/2022 10:27:31 PM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsRegressionStdDev: Marked as regression because -34.553061487659434 (T) = (0 -5.3353692196015245) / Math.Sqrt((0.008708534020578838 / (21)) + (0.104919283686032 / (25))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (25) - 2, .025) and -0.7850322745270233 = (2.9889483208449144 - 5.3353692196015245) / 2.9889483208449144 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline b09ae1ad36858f526b68a005c6f6b2cf8f7ab4b6
Compare 625a9cdd4437a414f1fda220f5bacba101f209c8
Diff Diff

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ProducerConsumer - Duration of single invocation 792.53 ns 1.02 μs 1.28 0.61 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Byte&gt;*'
Details

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Byte>.ProducerConsumer(RentalSize: 4096, ManipulateArray: False, Async: True, UseSharedPool: False)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.017125941 > 847.1907515531251.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 6/2/2022 10:27:31 PM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsRegressionStdDev: Marked as regression because -13.284601832092536 (T) = (0 -1208.7045763563606) / Math.Sqrt((1741.790102938255 / (21)) + (19307.131931795648 / (25))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (25) - 2, .025) and -0.47365972785309757 = (820.2060173804592 - 1208.7045763563606) / 820.2060173804592 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. changed the title [-][Perf] Changes at 6/3/2022 1:48:25 AM[/-] [+][Perf] Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark[/+] on Jun 9, 2022
  2. DrewScoggins commented on Jun 9, 2022

    @DrewScoggins
    Member

    Seems related to #70171

  3. added
    area-CodeGen-coreclrCLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI
    on Jun 10, 2022
  4. removed
    untriagedNew issue has not been triaged by the area owner
    on Jun 10, 2022
  5. added this to the 7.0.0 milestone on Jun 10, 2022
  6. tannergooding commented on Jun 16, 2022

    @tannergooding
    Member

    Second was resolved in 2d5c8dc

    First was mostly fixed but still has a small regression related to #70182

  7. ghost locked as resolved and limited conversation to collaborators on Jul 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions