-
Notifications
You must be signed in to change notification settings - Fork 4
Open
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 9fd407ae7a1b6a846a60fc0269b60d984b0e3c0b |
| Compare | 7be37908e5a1cbb83b1062768c1649827eeaceaa |
| Diff | Diff |
Improvements in System.Tests.Perf_Array
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Reverse - Duration of single invocation | 105.75 ns | 86.83 ns | 0.82 | 0.59 | False | |||||
| IndexOfShort - Duration of single invocation | 26.56 ns | 16.82 ns | 0.63 | 0.68 | False | |||||
| IndexOfChar - Duration of single invocation | 22.27 ns | 13.16 ns | 0.59 | 0.72 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Array*'Payloads
Histogram
System.Tests.Perf_Array.Reverse
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 6/22/2022 6:55:40 AM, 8/10/2022 5:14:51 PM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM.
IsImprovementStdDev: Marked as improvement because 31.386959615774234 (T) = (0 -68.59596554323386) / Math.Sqrt((1643.6627725194826 / (299)) + (65.56959697639218 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.5523060072810145 = (153.22065218393737 - 68.59596554323386) / 153.22065218393737 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Array.IndexOfShort
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/31/2022 5:02:26 PM, 6/6/2022 8:15:49 PM, 6/17/2022 2:19:07 PM, 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM.
IsImprovementStdDev: Marked as improvement because 11.941384025455461 (T) = (0 -16.20729493479791) / Math.Sqrt((7.2856275579830045 / (299)) + (1.4515482441687102 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.3143653543424668 = (23.638383849834323 - 16.20729493479791) / 23.638383849834323 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Tests.Perf_Array.IndexOfChar
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 5/31/2022 10:47:41 AM, 6/6/2022 8:15:49 PM, 6/30/2022 10:26:14 PM, 8/11/2022 10:02:44 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM.
IsImprovementStdDev: Marked as improvement because 4.888432564741097 (T) = (0 -18.322311755593066) / Math.Sqrt((9.464402570143697 / (299)) + (8.505378133252316 / (34))) is greater than 1.9671567996108528 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (34) - 2, .975) and 0.124062865710557 = (20.917382125209315 - 18.322311755593066) / 20.917382125209315 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 9fd407ae7a1b6a846a60fc0269b60d984b0e3c0b |
| Compare | 7be37908e5a1cbb83b1062768c1649827eeaceaa |
| Diff | Diff |
Improvements in System.Memory.Span<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Reverse - Duration of single invocation | 235.75 ns | 163.58 ns | 0.69 | 0.63 | False | |||||
| LastIndexOfValue - Duration of single invocation | 115.81 ns | 58.08 ns | 0.50 | 0.66 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span<Int32>*'Payloads
Histogram
System.Memory.Span<Int32>.Reverse(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 6/22/2022 6:55:40 AM, 8/10/2022 5:14:51 PM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM.
IsImprovementStdDev: Marked as improvement because 29.220096576861827 (T) = (0 -137.68064416364183) / Math.Sqrt((6847.434229540765 / (299)) + (340.75373276634394 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.5459981203497988 = (303.260075199957 - 137.68064416364183) / 303.260075199957 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Memory.Span<Int32>.LastIndexOfValue(Size: 512)
```log
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/17/2022 10:14:53 AM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM.
IsImprovementStdDev: Marked as improvement because 13.875710502574806 (T) = (0 -69.99694582425289) / Math.Sqrt((111.45597203297228 / (299)) + (66.30161479916354 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.44936617689685776 = (127.12067963747548 - 69.99694582425289) / 127.12067963747548 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 9fd407ae7a1b6a846a60fc0269b60d984b0e3c0b |
| Compare | 7be37908e5a1cbb83b1062768c1649827eeaceaa |
| Diff | Diff |
Improvements in System.Memory.Span<Byte>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| Reverse - Duration of single invocation | 246.23 ns | 43.96 ns | 0.18 | 0.68 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span<Byte>*'Payloads
Histogram
System.Memory.Span<Byte>.Reverse(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 6/22/2022 6:55:40 AM, 8/10/2022 5:14:51 PM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM.
IsImprovementStdDev: Marked as improvement because 82.8473279479604 (T) = (0 -33.996704204738755) / Math.Sqrt((2343.3931044860055 / (299)) + (19.158088121052103 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.8756840704307884 = (273.4702167497486 - 33.996704204738755) / 273.4702167497486 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 9fd407ae7a1b6a846a60fc0269b60d984b0e3c0b |
| Compare | 7be37908e5a1cbb83b1062768c1649827eeaceaa |
| Diff | Diff |
Improvements in System.Collections.IterateForEach<Int32>
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConcurrentDictionary - Duration of single invocation | 28.57 μs | 23.07 μs | 0.81 | 0.65 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach<Int32>*'Payloads
Histogram
System.Collections.IterateForEach<Int32>.ConcurrentDictionary(Size: 512)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/10/2022 5:14:51 PM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM.
IsImprovementStdDev: Marked as improvement because 18.403811107525176 (T) = (0 -21388.95300485393) / Math.Sqrt((7315062.335286484 / (299)) + (2779745.331549286 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.21355190974424765 = (27196.90373702638 - 21388.95300485393) / 27196.90373702638 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 9fd407ae7a1b6a846a60fc0269b60d984b0e3c0b |
| Compare | 7be37908e5a1cbb83b1062768c1649827eeaceaa |
| Diff | Diff |
Improvements in BenchmarksGame.BinaryTrees_2
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| RunBench - Duration of single invocation | 302.28 ms | 252.44 ms | 0.84 | 0.54 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.BinaryTrees_2*'Payloads
Histogram
BenchmarksGame.BinaryTrees_2.RunBench
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 6/22/2022 11:10:17 PM, 8/10/2022 5:14:51 PM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM.
IsImprovementStdDev: Marked as improvement because 18.275465793291815 (T) = (0 -247854770.25983882) / Math.Sqrt((689560499765791.8 / (299)) + (147520533866402.12 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.1560991589880753 = (293701295.47760046 - 247854770.25983882) / 293701295.47760046 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 9fd407ae7a1b6a846a60fc0269b60d984b0e3c0b |
| Compare | 7be37908e5a1cbb83b1062768c1649827eeaceaa |
| Diff | Diff |
Improvements in BenchmarksGame.FannkuchRedux_9
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| RunBench - Duration of single invocation | 128.82 ms | 62.10 ms | 0.48 | 0.66 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.FannkuchRedux_9*'Payloads
Histogram
BenchmarksGame.FannkuchRedux_9.RunBench(n: 11, expectedSum: 556355)
Description of detection logic
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because we could not find enough baseline builds for window checking.
IsChangePoint: Marked as a change because one of 8/10/2022 5:14:51 PM, 8/18/2022 11:13:04 AM falls between 8/9/2022 9:33:37 PM and 8/18/2022 11:13:04 AM.
IsImprovementStdDev: Marked as improvement because 9.326437938067002 (T) = (0 -75839386.09669577) / Math.Sqrt((658128230302220.8 / (299)) + (287732473443652.06 / (37))) is greater than 1.9670919629190173 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (37) - 2, .975) and 0.2797724204288068 = (105299197.43124636 - 75839386.09669577) / 105299197.43124636 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository



_1.png)
_2.png)
_1.png)
_1.png)

