-
Notifications
You must be signed in to change notification settings - Fork 4
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.Collections.CtorFromCollection<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ImmutableDictionary - Duration of single invocation | 112.46 μs | 105.35 μs | 0.94 | 0.01 | False | |||||
| ConcurrentDictionary - Duration of single invocation | 50.45 μs | 21.37 μs | 0.42 | 0.01 | False | |||||
| SortedDictionary - Duration of single invocation | 51.44 μs | 47.75 μs | 0.93 | 0.01 | False | |||||
| ImmutableHashSet - Duration of single invocation | 101.30 μs | 92.31 μs | 0.91 | 0.02 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection<Int32>*'Payloads
Histogram
System.Collections.CtorFromCollection<Int32>.ImmutableDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 105.35255485977947 < 105.94128253126411.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 23.049601540146526 (T) = (0 -102770.1392218111) / Math.Sqrt((2691054.2202400323 / (176)) + (2903124.8310453584 / (22))) is greater than 1.9721412216619414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (22) - 2, .975) and 0.07924446037690715 = (111615.01049874714 - 102770.1392218111) / 111615.01049874714 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.CtorFromCollection<Int32>.ConcurrentDictionary(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 21.372497068533697 < 47.96721730939688.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 483.1631132228493 (T) = (0 -21410.01874746862) / Math.Sqrt((241772.504528089 / (177)) + (49935.27820447392 / (22))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (177) + (22) - 2, .975) and 0.5764025960629124 = (50543.31907720668 - 21410.01874746862) / 50543.31907720668 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.CtorFromCollection<Int32>.SortedDictionary(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 47.746682404731 < 48.784586067184144.
IsChangePoint: Marked as a change because one of 11/29/2022 11:47:49 AM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 24.141666482139122 (T) = (0 -48054.06240096575) / Math.Sqrt((4911339.670790407 / (176)) + (48134.29390526623 / (23))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (23) - 2, .975) and 0.08004782088544113 = (52235.391677877335 - 48054.06240096575) / 52235.391677877335 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.CtorFromCollection<Int32>.ImmutableHashSet(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 92.30963643604915 < 96.30810780518843.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 29.218827221705517 (T) = (0 -91775.68990772369) / Math.Sqrt((2139438.1716021453 / (176)) + (2341554.70736618 / (23))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (23) - 2, .975) and 0.09704687395480038 = (101639.48411108289 - 91775.68990772369) / 101639.48411108289 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.Globalization.Tests.StringSearch
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| IsPrefix_FirstHalf - Duration of single invocation | 130.65 ns | 115.38 ns | 0.88 | 0.01 | False | |||||
| IsSuffix_SecondHalf - Duration of single invocation | 155.76 ns | 130.98 ns | 0.84 | 0.05 | False | |||||
| IsPrefix_FirstHalf - Duration of single invocation | 130.58 ns | 115.34 ns | 0.88 | 0.01 | False | |||||
| IsPrefix_FirstHalf - Duration of single invocation | 130.65 ns | 115.41 ns | 0.88 | 0.01 | False | |||||
| IsSuffix_SecondHalf - Duration of single invocation | 155.72 ns | 130.91 ns | 0.84 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'Payloads
Histogram
System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreNonSpace, False))
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.38181117529587 < 124.06334117779082.
IsChangePoint: Marked as a change because one of 1/5/2023 9:29:47 PM, 1/18/2023 8:37:25 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 26.069166433606043 (T) = (0 -115.4424901116955) / Math.Sqrt((37.58226259418721 / (176)) + (0.005828155060240404 / (23))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (23) - 2, .975) and 0.09454153524984525 = (127.49617415476901 - 115.4424901116955) / 127.49617415476901 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, IgnoreCase, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 130.9839507948598 < 148.02393678828426.
IsChangePoint: Marked as a change because one of 12/1/2022 7:29:59 PM, 12/17/2022 10:44:50 PM, 1/5/2023 8:47:31 AM, 1/30/2023 11:52:26 AM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 17.37857359470476 (T) = (0 -131.64079832883016) / Math.Sqrt((137.3460828948471 / (299)) + (0.11259727424457044 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08252619779660327 = (143.48180625177832 - 131.64079832883016) / 143.48180625177832 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.33705423993571 < 124.06411666526401.
IsChangePoint: Marked as a change because one of 1/5/2023 9:29:47 PM, 1/18/2023 8:37:25 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 26.189534045291953 (T) = (0 -115.4297714482856) / Math.Sqrt((37.44526008945016 / (176)) + (0.0038139552224689508 / (23))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (23) - 2, .975) and 0.0947718302582303 = (127.5145596509814 - 115.4297714482856) / 127.5145596509814 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, None, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 115.40575862636128 < 124.10852671919976.
IsChangePoint: Marked as a change because one of 1/5/2023 9:29:47 PM, 1/18/2023 8:37:25 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 26.0336849574237 (T) = (0 -115.4397346515852) / Math.Sqrt((37.57377342353674 / (176)) + (0.03457601257491241 / (23))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (23) - 2, .975) and 0.09466697078028857 = (127.51079539324928 - 115.4397346515852) / 127.51079539324928 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (, IgnoreCase, False))
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 130.90982142857143 < 148.0047240456937.
IsChangePoint: Marked as a change because one of 11/29/2022 11:47:49 AM, 1/30/2023 11:52:26 AM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 12.804066081387074 (T) = (0 -131.63897266332842) / Math.Sqrt((134.9905495204924 / (176)) + (0.10234313040382935 / (23))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (23) - 2, .975) and 0.07870688690607695 = (142.88500672848102 - 131.63897266332842) / 142.88500672848102 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.Collections.ContainsFalse<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ImmutableSortedSet - Duration of single invocation | 28.42 μs | 24.12 μs | 0.85 | 0.01 | False | |||||
| SortedSet - Duration of single invocation | 27.31 μs | 23.67 μs | 0.87 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse<Int32>*'Payloads
Histogram
System.Collections.ContainsFalse<Int32>.ImmutableSortedSet(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 24.119118621794875 < 27.043979271687164.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 150.23338181539486 (T) = (0 -23669.60922981652) / Math.Sqrt((42861.80967792578 / (299)) + (20353.831751889342 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.1691130610945708 = (28487.160071378345 - 23669.60922981652) / 28487.160071378345 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.ContainsFalse<Int32>.SortedSet(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.67047927681268 < 25.888289303453377.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 138.2687016781119 (T) = (0 -23939.910188114336) / Math.Sqrt((46289.16333201491 / (176)) + (8678.622311545772 / (23))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (23) - 2, .975) and 0.1275156928629827 = (27438.785995671493 - 23939.910188114336) / 27438.785995671493 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in ByteMark
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| BenchStringSort - Duration of single invocation | 1.15 secs | 1.05 secs | 0.91 | 0.00 | False | |||||
| BenchAssignJagged - Duration of single invocation | 776.98 ms | 733.18 ms | 0.94 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'Payloads
Histogram
ByteMark.BenchStringSort
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.0505456563571427 < 1.0904002357460163.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 59.02071878061766 (T) = (0 -1053637435.0474602) / Math.Sqrt((158720197384879.47 / (299)) + (43928839982943.29 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.08047171842101832 = (1145845599.4830208 - 1053637435.0474602) / 1145845599.4830208 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### ByteMark.BenchAssignJagged
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 733.1787160666668 < 740.87619277.
IsChangePoint: Marked as a change because one of 2/9/2023 4:10:34 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 31.657745798622308 (T) = (0 -732962557.0306461) / Math.Sqrt((106355059489392.9 / (178)) + (28430009807236.355 / (21))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (178) + (21) - 2, .975) and 0.05690084612783028 = (777185043.6099467 - 732962557.0306461) / 777185043.6099467 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.Tests.Perf_Double
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Parse - Duration of single invocation | 106.03 ns | 86.42 ns | 0.82 | 0.02 | False | |||||
| TryParse - Duration of single invocation | 105.05 ns | 99.42 ns | 0.95 | 0.02 | False | |||||
| TryParse - Duration of single invocation | 101.26 ns | 94.66 ns | 0.93 | 0.03 | False | |||||
| Parse - Duration of single invocation | 107.78 ns | 88.63 ns | 0.82 | 0.01 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'Payloads
Histogram
System.Tests.Perf_Double.Parse(value: "1.7976931348623157e+308")
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 86.415774728392 < 100.66060473063416.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 25.185048579495312 (T) = (0 -92.1525876252544) / Math.Sqrt((1.8287100435895554 / (184)) + (5.617528475182294 / (23))) is greater than 1.9716034992643237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (184) + (23) - 2, .975) and 0.12110010331213447 = (104.849924288911 - 92.1525876252544) / 104.849924288911 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Double.TryParse(value: "-1.7976931348623157e+308")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 99.42237991518 < 99.76880559625542.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 27.088340860480123 (T) = (0 -91.34426223663898) / Math.Sqrt((2.6105219491728575 / (278)) + (6.650202025013274 / (23))) is greater than 1.9679296690655894 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (278) + (23) - 2, .975) and 0.1394368511430958 = (106.14475225667356 - 91.34426223663898) / 106.14475225667356 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Double.TryParse(value: "1.7976931348623157e+308")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 94.65690724420142 < 96.45732403271971.
IsChangePoint: Marked as a change because one of 1/13/2023 4:37:08 PM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 28.350379122285045 (T) = (0 -89.5921318060243) / Math.Sqrt((3.244979833550474 / (299)) + (5.374637199448164 / (23))) is greater than 1.9674049737480528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (23) - 2, .975) and 0.13530675994757752 = (103.61146318270364 - 89.5921318060243) / 103.61146318270364 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Double.Parse(value: "-1.7976931348623157e+308")
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 88.62537096593823 < 102.39567916584159.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 26.024538683934907 (T) = (0 -94.48285144214587) / Math.Sqrt((1.2180688987601382 / (200)) + (5.264791393410541 / (23))) is greater than 1.9707562704893595 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (200) + (23) - 2, .975) and 0.11779533200071511 = (107.09856212438729 - 94.48285144214587) / 107.09856212438729 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | x64 |
|---|---|
| OS | alpine 3.15 |
| Baseline | 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4 |
| Compare | fe1607679b977b453d43c4de276968dd9ae6d5ec |
| Diff | Diff |
Improvements in System.Collections.TryGetValueFalse<Int32, Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| SortedDictionary - Duration of single invocation | 39.89 μs | 35.79 μs | 0.90 | 0.02 | False | |||||
| ImmutableSortedDictionary - Duration of single invocation | 27.00 μs | 24.00 μs | 0.89 | 0.02 | False | |||||
| ConcurrentDictionary - Duration of single invocation | 2.26 μs | 1.59 μs | 0.70 | 0.12 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse<Int32, Int32>*'Payloads
Histogram
System.Collections.TryGetValueFalse<Int32, Int32>.SortedDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 35.78999876952489 < 37.792382288119015.
IsChangePoint: Marked as a change because one of 11/29/2022 11:47:49 AM, 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 25.769584041891722 (T) = (0 -34990.88178213833) / Math.Sqrt((7980060.287465244 / (176)) + (107521.1874714204 / (23))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (23) - 2, .975) and 0.1414136556603103 = (40754.062783340276 - 34990.88178213833) / 40754.062783340276 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.TryGetValueFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 23.996671505376344 < 25.66422042298272.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 40.86584341027516 (T) = (0 -23041.352222932466) / Math.Sqrt((193039.6375189821 / (176)) + (211028.39886229963 / (23))) is greater than 1.9720790337782532 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (23) - 2, .975) and 0.15236586564212892 = (27183.1339595444 - 23041.352222932466) / 27183.1339595444 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Collections.TryGetValueFalse<Int32, Int32>.ConcurrentDictionary(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 1.5858755983225379 < 2.1346860723141075.
IsChangePoint: Marked as a change because one of 2/9/2023 12:40:40 PM, 2/14/2023 7:27:15 AM falls between 2/5/2023 7:19:43 PM and 2/14/2023 7:27:15 AM.
IsImprovementStdDev: Marked as improvement because 44.87680973636697 (T) = (0 -1600.6065238941605) / Math.Sqrt((9993.937580192412 / (176)) + (3525.7777294950224 / (22))) is greater than 1.9721412216619414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (176) + (22) - 2, .975) and 0.2923162077775273 = (2261.7538249215436 - 1600.6065238941605) / 2261.7538249215436 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
_1.png)
_2.png)
_3.png)
_4.png)





_1.png)
_2.png)






_1.png)
_2.png)
_3.png)