Skip to content

Regressions in System.Buffers.Text.Tests.Utf8ParserTests #89261

Description

@performanceautofiler

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 4e48d2d85ed15140b95323e615a5433726b583b2
Compare 11fbd151912a3c5d1e72386fb037595b71e214d6
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
TryParseSingle - Duration of single invocation
📝 - Benchmark Source
37.60 ns 40.83 ns 1.09 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'
Details

Payloads

Baseline
Compare

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.828730634561595 > 39.29663517446423.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/15/2023 9:18:21 AM, 7/11/2023 7:34:35 PM, 7/18/2023 4:12:30 PM falls between 7/10/2023 3:14:43 AM and 7/18/2023 4:12:30 PM.
IsRegressionStdDev: Marked as regression because -15.351106528911764 (T) = (0 -40.50466641693999) / Math.Sqrt((0.1264184911309328 / (10)) + (0.21321768908095398 / (7))) is less than -2.131449545559758 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (10) + (7) - 2, .025) and -0.08540274483565771 = (37.31763772448618 - 40.50466641693999) / 37.31763772448618 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

  1. removed
    untriagedNew issue has not been triaged by the area owner
    on Jul 20, 2023
  2. ghost added
    needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area owners
    on Jul 20, 2023
  3. ghost added
    untriagedNew issue has not been triaged by the area owner
    on Jul 20, 2023
  4. changed the title [-][Perf] Linux/arm64: 1 Regression on 7/12/2023 6:09:40 AM[/-] [+]Regressions in System.Buffers.Text.Tests.Utf8ParserTests[/+] on Jul 20, 2023
  5. cincuranet commented on Jul 20, 2023

    @cincuranet
    Contributor

    Commit range is 14e8b82...d3d910e. Out best guess is #86875.

    cc @tannergooding

  6. tannergooding commented on Jul 20, 2023

    @tannergooding
    Member

    A small regression was expected since we had to generalize a little bit of code to share the algorithm between UTF-8 and UTF-16. A regression of 3ns is more than acceptable given the amount of code complexity we were able to avoid.

  7. ghost removed
    untriagedNew issue has not been triaged by the area owner
    on Jul 20, 2023
  8. ghost locked as resolved and limited conversation to collaborators on Aug 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    arch-arm64needs-area-labelAn area label is needed to ensure this gets routed to the appropriate area ownersos-linuxLinux OS (any supported distro)runtime-coreclrspecific to the CoreCLR runtime

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions