[Perf] Windows/arm64: 5 Improvements on 5/9/2023 6:35:22 PM #17665
Description
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | aaa1de18a7f97774fd44a1728dafc0d86894800e |
| Compare | 3747edb1812106f0b88a42b724c3e44b69dc79c4 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Tests.Perf_Enum
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| InterpolateIntoStringBuilder_NonFlags - Duration of single invocation | 127.42 ns | 117.85 ns | 0.92 | 0.00 | False | |||||
| InterpolateIntoStringBuilder_Flags - Duration of single invocation | 113.37 ns | 106.83 ns | 0.94 | 0.00 | False | |||||
| InterpolateIntoStringBuilder_Flags - Duration of single invocation | 118.70 ns | 109.27 ns | 0.92 | 0.00 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'Payloads
Histogram
System.Tests.Perf_Enum.InterpolateIntoStringBuilder_NonFlags(value: 42)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 117.84864251658131 < 120.74858292926051.
IsChangePoint: Marked as a change because one of 3/20/2023 7:45:43 PM, 4/12/2023 8:39:35 PM, 5/9/2023 1:22:51 PM, 5/10/2023 10:36:48 PM falls between 5/2/2023 4:25:00 AM and 5/10/2023 10:36:48 PM.
IsImprovementStdDev: Marked as improvement because 43.10419597589514 (T) = (0 -118.09610845408913) / Math.Sqrt((4.059321136798374 / (299)) + (0.19100080708740286 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.08289480240241906 = (128.77051483673833 - 118.09610845408913) / 128.77051483673833 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: Red)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 106.82916527808635 < 108.02813039918908.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 4/12/2023 8:39:35 PM, 5/9/2023 1:22:51 PM, 5/10/2023 10:36:48 PM falls between 5/2/2023 4:25:00 AM and 5/10/2023 10:36:48 PM.
IsImprovementStdDev: Marked as improvement because 43.05392407723469 (T) = (0 -106.82560231245947) / Math.Sqrt((2.5672561147599766 / (299)) + (0.16206118597047145 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.08198497740408771 = (116.36585424319519 - 106.82560231245947) / 116.36585424319519 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: 32)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 109.26854201937013 < 112.53579742828478.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 3/20/2023 8:31:51 AM, 4/12/2023 8:39:35 PM, 5/9/2023 1:22:51 PM, 5/10/2023 10:36:48 PM falls between 5/2/2023 4:25:00 AM and 5/10/2023 10:36:48 PM.
IsImprovementStdDev: Marked as improvement because 27.816500640792256 (T) = (0 -109.30258703078658) / Math.Sqrt((5.457922050050556 / (299)) + (0.6407376727850235 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.09706701135431912 = (121.05282275125504 - 109.30258703078658) / 121.05282275125504 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Run Information
| Name | Value |
|---|---|
| Architecture | arm64 |
| OS | Windows 10.0.19041 |
| Queue | SurfaceWindows |
| Baseline | aaa1de18a7f97774fd44a1728dafc0d86894800e |
| Compare | 3747edb1812106f0b88a42b724c3e44b69dc79c4 |
| Diff | Diff |
| Configs | CompilationMode:tiered, RunKind:micro |
Improvements in System.Text.Tests.Perf_StringBuilder
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| AppendLine_Strings - Duration of single invocation | 290.45 ns | 258.97 ns | 0.89 | 0.18 | False | |||||
| Append_Strings - Duration of single invocation | 103.05 μs | 86.32 μs | 0.84 | 0.03 | False |
Repro
General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md
Payloads
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'Payloads
Histogram
System.Text.Tests.Perf_StringBuilder.AppendLine_Strings
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 258.9687309690895 < 275.53522504628836.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/9/2023 1:22:51 PM, 5/10/2023 10:36:48 PM falls between 5/2/2023 4:25:00 AM and 5/10/2023 10:36:48 PM.
IsImprovementStdDev: Marked as improvement because 60.66970985962225 (T) = (0 -259.49284752046424) / Math.Sqrt((144.15690377914453 / (299)) + (0.5280010737925435 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.15484982032294473 = (307.0375582474826 - 259.49284752046424) / 307.0375582474826 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
System.Text.Tests.Perf_StringBuilder.Append_Strings(repeat: 1000)
Description of detection logic
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 86.31852914389799 < 97.8977422676282.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/2/2023 12:51:17 PM, 5/9/2023 1:22:51 PM, 5/10/2023 10:36:48 PM falls between 5/2/2023 4:25:00 AM and 5/10/2023 10:36:48 PM.
IsImprovementStdDev: Marked as improvement because 84.50907079960709 (T) = (0 -86630.2625909634) / Math.Sqrt((10314708.560047647 / (299)) + (135635.65994215125 / (4))) is greater than 1.9678765312856872 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (4) - 2, .975) and 0.20327728213283175 = (108733.26522290359 - 86630.2625909634) / 108733.26522290359 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
JIT Disasms
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository