Skip to content

[Perf] Windows/x64: 25 Regressions on 10/27/2022 4:20:56 AM #9386

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline fb36ccef4bab6bf69d890c30978daccdb62b5839
Compare e1ddbca5a008b4b15e8e6b309cbab12ba8b9b842
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MatchesSet - Duration of single invocation 140.32 μs 149.68 μs 1.07 0.04 False
MatchesBoundary - Duration of single invocation 125.80 μs 141.64 μs 1.13 0.01 False
IP_IsNotMatch - Duration of single invocation 135.75 ns 156.71 ns 1.15 0.00 False
OneNodeBacktracking - Duration of single invocation 78.46 ns 97.75 ns 1.25 0.10 False
Uri_IsNotMatch - Duration of single invocation 327.73 ns 347.45 ns 1.06 0.00 False
IP_IsNotMatch - Duration of single invocation 135.65 ns 156.52 ns 1.15 0.00 False
Backtracking - Duration of single invocation 116.74 ns 125.08 ns 1.07 0.03 False
IP_IsMatch - Duration of single invocation 135.05 ns 156.49 ns 1.16 0.00 False
IP_IsMatch - Duration of single invocation 135.54 ns 156.57 ns 1.16 0.01 False
Date_IsNotMatch - Duration of single invocation 380.71 ns 405.72 ns 1.07 0.01 False
Uri_IsMatch - Duration of single invocation 330.18 ns 351.42 ns 1.06 0.01 False
Email_IsMatch - Duration of single invocation 374.15 ns 395.66 ns 1.06 0.02 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: None)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 149.6773681972789 > 147.37894429301167.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -16.95378566519983 (T) = (0 -150103.2857415507) / Math.Sqrt((2179201.0225755284 / (30)) + (4834654.769243915 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.0662779534487438 = (140773.13073580884 - 150103.2857415507) / 140773.13073580884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 141.6434909119898 > 132.33088283188587.
IsChangePoint: Marked as a change because one of 9/9/2022 3:12:19 AM, 9/13/2022 12:23:36 PM, 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -12.60813826048895 (T) = (0 -135994.8813265555) / Math.Sqrt((5133239.581457509 / (30)) + (5050361.745308852 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.06324097055606338 = (127905.98283231286 - 135994.8813265555) / 127905.98283231286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 156.70914524329217 > 142.41545549872353.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -49.01071591084684 (T) = (0 -157.79025839502637) / Math.Sqrt((0.07727068529704825 / (29)) + (4.224832129999324 / (21))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (21) - 2, .025) and -0.16311140276083294 = (135.66220571863167 - 157.79025839502637) / 135.66220571863167 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 97.74729704554069 > 82.7314203966761.
IsChangePoint: Marked as a change because one of 8/17/2022 2:11:12 PM, 9/13/2022 12:23:36 PM, 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -14.057424855898388 (T) = (0 -99.39365223414833) / Math.Sqrt((2.9599169190657544 / (30)) + (40.86431245814375 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.253497019228293 = (79.29309021838709 - 99.39365223414833) / 79.29309021838709 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 347.4465555322682 > 342.26791580429375.
IsChangePoint: Marked as a change because one of 10/13/2022 1:52:29 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -22.208376038956498 (T) = (0 -347.40714038740873) / Math.Sqrt((4.64669863876642 / (21)) + (23.394050754442336 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.06833083228684746 = (325.1868521324579 - 347.40714038740873) / 325.1868521324579 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 156.51853024217021 > 142.06735607658766.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -63.77297491438204 (T) = (0 -157.42985744037549) / Math.Sqrt((0.11583000372110523 / (30)) + (2.408179036038734 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.16207204610667447 = (135.47340542939446 - 157.42985744037549) / 135.47340542939446 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 125.08248206012472 > 122.75261000676508.
IsChangePoint: Marked as a change because one of 8/17/2022 2:11:12 PM, 10/11/2022 8:42:28 PM, 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -16.416878211351246 (T) = (0 -127.14338593604865) / Math.Sqrt((2.7636293992044294 / (30)) + (4.368647414092228 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.07612550956706875 = (118.14921661619111 - 127.14338593604865) / 118.14921661619111 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 156.4941229425184 > 141.83105034631538.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -56.82170113980674 (T) = (0 -157.8546478310361) / Math.Sqrt((0.0863586493195018 / (30)) + (3.2465513193809636 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.16664987969206746 = (135.30593074993604 - 157.8546478310361) / 135.30593074993604 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 156.56892292005725 > 142.0595712306514.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -84.73893033426401 (T) = (0 -157.09503612349684) / Math.Sqrt((0.1456461086283489 / (30)) + (1.2587843899989375 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.15916312965448454 = (135.52452808805492 - 157.09503612349684) / 135.52452808805492 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 405.7160496685808 > 400.2365765712492.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -14.181375757313088 (T) = (0 -406.6855534247381) / Math.Sqrt((2.739113915165974 / (21)) + (88.49788668804165 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.06519107235442667 = (381.79587116312166 - 406.6855534247381) / 381.79587116312166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 351.42332669593156 > 346.76241906595715.
IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -36.73543486640055 (T) = (0 -348.655397260364) / Math.Sqrt((1.7862155281059624 / (21)) + (4.825545109386274 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.05512923979184741 = (330.4385700932197 - 348.655397260364) / 330.4385700932197 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 395.65830930938597 > 392.81345804644235.
IsChangePoint: Marked as a change because one of 10/2/2022 11:57:04 PM, 10/6/2022 2:13:23 PM, 10/26/2022 7:11:30 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -22.591750084223325 (T) = (0 -395.1405359216168) / Math.Sqrt((12.746870019817237 / (25)) + (6.022220069488994 / (26))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (26) - 2, .025) and -0.05178204847404808 = (375.686708567518 - 395.1405359216168) / 375.686708567518 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline fb36ccef4bab6bf69d890c30978daccdb62b5839
Compare e1ddbca5a008b4b15e8e6b309cbab12ba8b9b842
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 115.45 ns 123.65 ns 1.07 0.00 False
IsMatch - Duration of single invocation 104.88 ns 111.81 ns 1.07 0.00 False
IsMatch - Duration of single invocation 265.31 ns 295.25 ns 1.11 0.00 False
IsMatch - Duration of single invocation 130.88 ns 138.94 ns 1.06 0.00 False
IsMatch - Duration of single invocation 124.80 ns 136.72 ns 1.10 0.02 False
IsMatch - Duration of single invocation 129.06 ns 137.30 ns 1.06 0.00 False
IsMatch - Duration of single invocation 354.10 ns 375.66 ns 1.06 0.01 False
IsMatch - Duration of single invocation 129.99 ns 141.72 ns 1.09 0.01 False
IsMatch - Duration of single invocation 115.25 ns 123.68 ns 1.07 0.01 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: None)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 123.65256773932903 > 121.16671847194219.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -15.53004255619715 (T) = (0 -123.70815526189747) / Math.Sqrt((4.812008630688541 / (30)) + (0.7493681377738748 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.05886204005918687 = (116.83123068136676 - 123.70815526189747) / 116.83123068136676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 111.81415582187584 > 110.08925889543154.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -20.151739897943173 (T) = (0 -111.8313618019148) / Math.Sqrt((2.308125162632503 / (30)) + (0.27531700452196717 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.05716474277127702 = (105.78423331519492 - 111.8313618019148) / 105.78423331519492 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 295.254785251916 > 278.0895437133043.
IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -38.89712387636462 (T) = (0 -291.3419578699521) / Math.Sqrt((0.39609063374129044 / (21)) + (13.53596904777607 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.10075892270519281 = (264.67371906825764 - 291.3419578699521) / 264.67371906825764 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 138.94381713071377 > 137.1898055245005.
IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -18.53346879273005 (T) = (0 -138.30051426860425) / Math.Sqrt((0.17748373604913678 / (21)) + (5.251769740443632 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.060903071807088545 = (130.36112152359985 - 138.30051426860425) / 130.36112152359985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 136.72093659488166 > 131.24155782818664.
IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -7.871751131124267 (T) = (0 -136.1351844921133) / Math.Sqrt((31.630331189526476 / (25)) + (2.56956197806118 / (26))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (26) - 2, .025) and -0.0724238118384063 = (126.94159061867815 - 136.1351844921133) / 126.94159061867815 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 137.3042628640652 > 135.97735552010533.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -27.466397674631594 (T) = (0 -137.75589693474456) / Math.Sqrt((1.7882769718522573 / (30)) + (0.3488775490459466 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.05825359138569554 = (130.17286032014746 - 137.75589693474456) / 130.17286032014746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 375.65920684116657 > 371.8228106537809.
IsChangePoint: Marked as a change because one of 9/29/2022 7:42:19 AM, 10/1/2022 2:31:50 PM, 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -41.090342981757296 (T) = (0 -374.38419478534735) / Math.Sqrt((1.0301524391475554 / (21)) + (5.370450327466975 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.05531438577628663 = (354.76081803807807 - 374.38419478534735) / 354.76081803807807 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 141.71635866193859 > 136.49307257766338.
IsChangePoint: Marked as a change because one of 10/25/2022 2:34:53 PM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -23.96194575562641 (T) = (0 -140.73228351288716) / Math.Sqrt((0.1737417504489115 / (21)) + (5.422902612241273 / (30))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (21) + (30) - 2, .025) and -0.07994736540864977 = (130.3140208686322 - 140.73228351288716) / 130.3140208686322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 123.68102999118749 > 121.16961473280091.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -15.766000208294397 (T) = (0 -123.81766416672868) / Math.Sqrt((4.965037919127629 / (30)) + (0.9184300289719076 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.0618470695791649 = (116.60592915305642 - 123.81766416672868) / 116.60592915305642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline fb36ccef4bab6bf69d890c30978daccdb62b5839
Compare e1ddbca5a008b4b15e8e6b309cbab12ba8b9b842
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 408.73 ms 486.27 ms 1.19 0.01 False
Count - Duration of single invocation 65.64 ms 73.49 ms 1.12 0.02 False
Count - Duration of single invocation 424.37 ms 498.15 ms 1.17 0.00 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Related Issues

Regressions

Improvements

Details

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 486.2738692307692 > 427.76470450000005.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -62.28452812762535 (T) = (0 -489455323.4170591) / Math.Sqrt((3765741704446.335 / (30)) + (33076840907498.95 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.19896478592803762 = (408231608.7692307 - 489455323.4170591) / 408231608.7692307 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "Tom.{10,25}river|river.{10,25}Tom", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 73.490665 > 68.9077038125.
IsChangePoint: Marked as a change because one of 9/9/2022 1:01:45 PM, 9/29/2022 7:42:19 AM, 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -21.404867857202156 (T) = (0 -71558822.38393602) / Math.Sqrt((900507283926.2388 / (30)) + (772821706780.8362 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.08380021862323693 = (66025842.35943222 - 71558822.38393602) / 66025842.35943222 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 498.14676 > 445.325805.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 11/1/2022 6:58:35 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 6:58:35 AM.
IsRegressionStdDev: Marked as regression because -147.8056279620369 (T) = (0 -498076435.23809516) / Math.Sqrt((1690271903110.6167 / (30)) + (4027384203840.654 / (21))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (21) - 2, .025) and -0.17345863288105068 = (424451634.920635 - 498076435.23809516) / 424451634.920635 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions