Repository navigation
[Perf] Windows/x64: 2 Regressions on 8/30/2022 9:25:07 PM #8310
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | c96f5047b20fa43d27ba0822f2e051b0c80e3fa2 |
| Compare | 4190ef878368d552380329348129a03008fc57cc |
| Diff | Diff |
Regressions in System.Collections.IndexerSet<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Array - Duration of single invocation | 246.09 ns | 327.06 ns | 1.33 | 0.01 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet<String>*'Details
Payloads
Histogram
System.Collections.IndexerSet<String>.Array(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 327.06472186813966 > 258.38845933346306.
IsChangePoint: Marked as a change because one of 7/6/2022 7:05:00 PM, 8/30/2022 6:17:03 PM, 9/5/2022 11:32:52 PM falls between 8/28/2022 2:29:57 AM and 9/5/2022 11:32:52 PM.
IsRegressionStdDev: Marked as regression because -587.155847199774 (T) = (0 -326.27243437616073) / Math.Sqrt((0.17492847660346614 / (30)) + (0.40596924951501634 / (32))) is less than -2.0002978220134566 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (32) - 2, .025) and -0.324303437725991 = (246.37286673242716 - 326.27243437616073) / 246.37286673242716 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | c96f5047b20fa43d27ba0822f2e051b0c80e3fa2 |
| Compare | 4190ef878368d552380329348129a03008fc57cc |
| Diff | Diff |
Regressions in System.Collections.IndexerSetReverse<String>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Array - Duration of single invocation | 247.11 ns | 326.00 ns | 1.32 | 0.00 | False |
_1.png)
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse<String>*'Details
Payloads
Histogram
System.Collections.IndexerSetReverse<String>.Array(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 325.99514228805117 > 259.5090287746064.
IsChangePoint: Marked as a change because one of 8/30/2022 6:17:03 PM, 9/5/2022 11:32:52 PM falls between 8/28/2022 2:29:57 AM and 9/5/2022 11:32:52 PM.
IsRegressionStdDev: Marked as regression because -858.022176110927 (T) = (0 -326.0701638596965) / Math.Sqrt((0.15194875524726445 / (31)) + (0.11270104593809152 / (32))) is less than -1.9996235849941724 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (32) - 2, .025) and -0.31840712561898316 = (247.32129971355303 - 326.0701638596965) / 247.32129971355303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Activity