KEMBAR78
[Perf] Windows/x64: 5 Improvements on 11/24/2022 9:42:44 PM · Issue #10226 · dotnet/perf-autofiling-issues · GitHub
Skip to content

[Perf] Windows/x64: 5 Improvements on 11/24/2022 9:42:44 PM #10226

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline ac03fbd184b182a6632a50bbe70bc733e487264c
Compare 4832236b4820ca83bbc9ce96003c2029b85c6ce2
Diff Diff

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToUpperInvariant - Duration of single invocation 48.78 ns 27.70 ns 0.57 0.01 False
Concat_str_str - Duration of single invocation 29.97 ns 27.73 ns 0.93 0.05 False
ToLowerInvariant - Duration of single invocation 48.28 ns 27.65 ns 0.57 0.03 False
ToLower - Duration of single invocation 54.87 ns 34.09 ns 0.62 0.04 False
ToUpper - Duration of single invocation 54.25 ns 33.29 ns 0.61 0.04 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.ToUpperInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.70413843674101 < 45.865418126467624.
IsChangePoint: Marked as a change because one of 11/24/2022 8:21:24 PM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsImprovementStdDev: Marked as improvement because 241.29521787786197 (T) = (0 -27.244834913532998) / Math.Sqrt((1.782055651107921 / (299)) + (0.061557660885357475 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.4704735836591698 = (51.45132343311997 - 27.244834913532998) / 51.45132343311997 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Concat_str_str(size: 100)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.72660231768547 < 28.508668179425687.
IsChangePoint: Marked as a change because one of 9/13/2022 12:23:36 PM, 9/23/2022 1:16:35 AM, 11/21/2022 9:59:44 PM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsImprovementStdDev: Marked as improvement because 14.494934712396955 (T) = (0 -28.06160042430037) / Math.Sqrt((0.6494612449954009 / (299)) + (0.4854504050157415 / (33))) is greater than 1.9671786747533526 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (33) - 2, .975) and 0.06289463252159187 = (29.944978865940534 - 28.06160042430037) / 29.944978865940534 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLowerInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.64643342074679 < 45.75298337236028.
IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/16/2022 6:51:59 PM, 11/24/2022 8:21:24 PM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsImprovementStdDev: Marked as improvement because 153.2733072724317 (T) = (0 -27.24888821563144) / Math.Sqrt((6.463296630011602 / (299)) + (0.0708161262318981 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.4772228412073368 = (52.123333541507165 - 27.24888821563144) / 52.123333541507165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToLower(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.08580737527149 < 52.30128524165019.
IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/16/2022 11:33:39 PM, 11/24/2022 8:21:24 PM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsImprovementStdDev: Marked as improvement because 94.94878090412887 (T) = (0 -33.99525752045803) / Math.Sqrt((17.900331925980527 / (299)) + (0.15693150926913657 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.4255169308525942 = (59.17538626666687 - 33.99525752045803) / 59.17538626666687 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.ToUpper(s: "This is a much longer piece of text that might benefit more from vectorization.")

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.288464192246515 < 51.761065998791636.
IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/16/2022 11:33:39 PM, 11/24/2022 8:21:24 PM, 11/29/2022 1:43:29 AM falls between 11/20/2022 12:28:28 PM and 11/29/2022 1:43:29 AM.
IsImprovementStdDev: Marked as improvement because 91.62803508287547 (T) = (0 -33.96511062163668) / Math.Sqrt((20.14602569694683 / (299)) + (0.12731826921053743 / (15))) is greater than 1.9675964973877862 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (15) - 2, .975) and 0.4262933300524889 = (59.20292093648515 - 33.96511062163668) / 59.20292093648515 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions