-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | ee6de921299d1b86a02d9049b9c8dd164cab1401 |
| Compare | 7fa0d5b5942f138c362f2753398b4a8d4f71eb73 |
| Diff | Diff |
Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Count - Duration of single invocation | 1.83 ms | 1.69 ms | 0.93 | 0.02 | False | |||||
| Count - Duration of single invocation | 573.50 μs | 462.08 μs | 0.81 | 0.04 | False | |||||
| Count - Duration of single invocation | 59.60 μs | 41.91 μs | 0.70 | 0.03 | False | |||||
| Count - Duration of single invocation | 743.73 μs | 578.49 μs | 0.78 | 0.01 | False | |||||
| Count - Duration of single invocation | 94.96 μs | 72.95 μs | 0.77 | 0.00 | False | |||||
| Count - Duration of single invocation | 117.25 μs | 93.17 μs | 0.79 | 0.01 | False | |||||
| Count - Duration of single invocation | 740.27 μs | 580.83 μs | 0.78 | 0.01 | False | |||||
| Count - Duration of single invocation | 121.22 μs | 92.41 μs | 0.76 | 0.00 | False | |||||
| Count - Duration of single invocation | 67.48 μs | 48.66 μs | 0.72 | 0.03 | False | |||||
| Count - Duration of single invocation | 187.03 μs | 173.60 μs | 0.93 | 0.02 | False | |||||
| Count - Duration of single invocation | 90.99 μs | 73.02 μs | 0.80 | 0.02 | False | |||||
| Count - Duration of single invocation | 128.87 μs | 114.56 μs | 0.89 | 0.01 | False | |||||
| Count - Duration of single invocation | 3.26 ms | 1.72 ms | 0.53 | 0.05 | False | |||||
| Count - Duration of single invocation | 1.55 ms | 1.13 ms | 0.73 | 0.00 | False | |||||
| Count - Duration of single invocation | 2.44 ms | 277.89 μs | 0.11 | 0.03 | False | |||||
| Count - Duration of single invocation | 154.08 μs | 128.93 μs | 0.84 | 0.01 | False | |||||
| Count - Duration of single invocation | 124.05 μs | 101.21 μs | 0.82 | 0.01 | False | |||||
| Count - Duration of single invocation | 1.04 ms | 905.02 μs | 0.87 | 0.00 | False | |||||
| Count - Duration of single invocation | 212.89 μs | 198.73 μs | 0.93 | 0.01 | False | |||||
| Count - Duration of single invocation | 1.68 ms | 1.51 ms | 0.90 | 0.01 | False | |||||
| Count - Duration of single invocation | 88.51 μs | 71.61 μs | 0.81 | 0.01 | False | |||||
| Count - Duration of single invocation | 2.90 ms | 2.30 ms | 0.79 | 0.00 | False | |||||
| Count - Duration of single invocation | 177.58 μs | 164.45 μs | 0.93 | 0.01 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'Payloads
Histogram
System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: NonBacktracking)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.693843918918919 < 1.7328354134164534.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 29.26085067699087 (T) = (0 -1694744.5902636999) / Math.Sqrt((639421722.5817436 / (299)) + (719813279.9669814 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.07791416019614592 = (1837946.6608273755 - 1694744.5902636999) / 1837946.6608273755 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 462.08318627450984 < 558.9391769471152.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 145.89551296112984 (T) = (0 -468816.7794674998) / Math.Sqrt((88587001.06963713 / (299)) + (15201372.793566674 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.2130619243859896 = (595747.9933877953 - 468816.7794674998) / 595747.9933877953 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 41.91380473637176 < 57.229892581381065.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 82.16515117542504 (T) = (0 -43231.96474810726) / Math.Sqrt((691346.3419113484 / (299)) + (1588943.781803123 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.2991987678014325 = (61689.338947762815 - 43231.96474810726) / 61689.338947762815 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 578.4888888888888 < 698.4432172168109.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 177.76612451645101 (T) = (0 -584319.1257207167) / Math.Sqrt((66237019.36671966 / (299)) + (17933764.36581146 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.2101645265762856 = (739798.5344819444 - 584319.1257207167) / 739798.5344819444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 72.94723315787145 < 85.67225399100255.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 108.0581034608251 (T) = (0 -73837.37627545052) / Math.Sqrt((1345271.9878836863 / (299)) + (699365.526416478 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.19000445669636698 = (91157.75621961911 - 73837.37627545052) / 91157.75621961911 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 93.17234344181459 < 110.49573409716672.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 105.83644157860124 (T) = (0 -93021.00936739385) / Math.Sqrt((1610427.9006883134 / (299)) + (1332408.7399374843 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.19574866097788854 = (115661.6157835659 - 93021.00936739385) / 115661.6157835659 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 580.8335658914727 < 702.5169954063765.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 132.04337964971478 (T) = (0 -582779.1279290583) / Math.Sqrt((302781415.344062 / (299)) + (21212275.334034715 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22571910285054883 = (752671.4530535172 - 582779.1279290583) / 752671.4530535172 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 92.40703246124032 < 114.03117397026494.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 135.24686747386653 (T) = (0 -93022.61428499474) / Math.Sqrt((1898796.677487415 / (299)) + (1039944.3025711386 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.22052342382429033 = (119339.8456453752 - 93022.61428499474) / 119339.8456453752 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 48.659067982456136 < 62.86347395184992.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 93.25567403400107 (T) = (0 -46589.24304051709) / Math.Sqrt((689362.6433879746 / (299)) + (1476878.934846277 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.3027585002240229 = (66819.37758364377 - 46589.24304051709) / 66819.37758364377 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 173.5950806823421 < 177.12879186368556.
IsChangePoint: Marked as a change because one of 10/26/2022 7:11:30 AM, 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 41.790853696293475 (T) = (0 -172739.53094363294) / Math.Sqrt((6962713.629807717 / (299)) + (3695134.8001098535 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.08170646941662718 = (188109.27572787658 - 172739.53094363294) / 188109.27572787658 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 73.01580283064911 < 85.92356141162347.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 57.29838727093823 (T) = (0 -73311.23158074208) / Math.Sqrt((1754440.8020451448 / (299)) + (2569966.359308927 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.18445847267762322 = (89892.70211835916 - 73311.23158074208) / 89892.70211835916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 114.55663580246913 < 122.56867613636364.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 63.37891597285626 (T) = (0 -110513.73486523086) / Math.Sqrt((4344916.001973893 / (299)) + (2037056.037937892 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1367194286461899 = (128016.01070660201 - 110513.73486523086) / 128016.01070660201 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.7159723218390803 < 3.0813864523809515.
IsChangePoint: Marked as a change because one of 1/5/2023 12:24:13 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 324.7549063097047 (T) = (0 -1721693.1326587913) / Math.Sqrt((4247332207.8707232 / (299)) + (155584640.4001951 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.469331762575631 = (3244387.0034791133 - 1721693.1326587913) / 3244387.0034791133 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.1325757971938777 < 1.4860485140625.
IsChangePoint: Marked as a change because one of 1/5/2023 12:24:13 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 239.5432632729244 (T) = (0 -1143861.5691759877) / Math.Sqrt((26341856.469546553 / (299)) + (60224796.19074455 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.26936698821036364 = (1565576.0836403708 - 1143861.5691759877) / 1565576.0836403708 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 277.8927631578947 < 2.311905891369048.
IsChangePoint: Marked as a change because one of 1/5/2023 12:24:13 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 1005.4197834740946 (T) = (0 -278331.34369657247) / Math.Sqrt((1253203094.4905436 / (299)) + (6045342.69268273 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.8844919512480535 = (2409627.2658391856 - 278331.34369657247) / 2409627.2658391856 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 128.92785342261905 < 145.94663114005573.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 115.71473509636823 (T) = (0 -129921.0676101156) / Math.Sqrt((3394516.379122636 / (299)) + (1180135.7786901584 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.16200493493406765 = (155037.98653025908 - 129921.0676101156) / 155037.98653025908 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 101.21130472350227 < 117.51147890044614.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 63.86937721365162 (T) = (0 -99472.76909192477) / Math.Sqrt((3308877.395589397 / (299)) + (4087497.1871546367 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.19084276666114391 = (122933.79456235783 - 99472.76909192477) / 122933.79456235783 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 905.0174305555555 < 0.9842397688802084.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 178.14440743154486 (T) = (0 -903069.5966552738) / Math.Sqrt((47355284.488947794 / (299)) + (12426068.658104882 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.1260859549422157 = (1033362.0357315138 - 903069.5966552738) / 1033362.0357315138 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 198.72614039378746 < 202.14865106517976.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 47.898086352140105 (T) = (0 -198085.59752383124) / Math.Sqrt((7287540.646832671 / (299)) + (3229498.825961016 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.07795071637690346 = (214831.89786285016 - 198085.59752383124) / 214831.89786285016 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.513497102272727 < 1.5831085.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 47.58372779363429 (T) = (0 -1508276.3796935384) / Math.Sqrt((429067440.3099698 / (299)) + (334001040.19935834 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.09685932393574213 = (1670034.823662649 - 1508276.3796935384) / 1670034.823662649 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: None)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 71.61492104261795 < 83.75480239588654.
IsChangePoint: Marked as a change because one of 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 91.94403568749895 (T) = (0 -71735.29978637917) / Math.Sqrt((1398855.1969620679 / (299)) + (957038.9226665046 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.19042762406655492 = (88608.87787045309 - 71735.29978637917) / 88608.87787045309 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.2986441071428567 < 2.7496874878472215.
IsChangePoint: Marked as a change because one of 1/5/2023 12:24:13 PM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 229.78316705277726 (T) = (0 -2292073.24829932) / Math.Sqrt((40091735.88739634 / (299)) + (135691672.94079757 / (20))) is greater than 1.9674756580274855 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (20) - 2, .975) and 0.2086694800625716 = (2896480.2829550384 - 2292073.24829932) / 2896480.2829550384 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: NonBacktracking)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 164.45479308776711 < 170.99650606142185.
IsChangePoint: Marked as a change because one of 10/27/2022 12:34:03 AM, 1/3/2023 6:29:42 AM, 1/10/2023 1:28:02 AM falls between 1/1/2023 3:57:25 AM and 1/10/2023 1:28:02 AM.
IsImprovementStdDev: Marked as improvement because 55.50820131292505 (T) = (0 -166936.71606059402) / Math.Sqrt((6056815.002438731 / (299)) + (1796716.176210971 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.08330986577854094 = (182108.11901272685 - 166936.71606059402) / 182108.11901272685 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository






















