KEMBAR78
[Perf] Windows/x64: 60 Improvements on 2/9/2023 12:40:40 PM · Issue #12985 · dotnet/perf-autofiling-issues · GitHub
Skip to content

[Perf] Windows/x64: 60 Improvements on 2/9/2023 12:40:40 PM #12985

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.Collections.CreateAddAndClear<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 51.94 μs 45.68 μs 0.88 0.01 False
ImmutableSortedSet - Duration of single invocation 179.85 μs 164.94 μs 0.92 0.01 False
ImmutableList - Duration of single invocation 101.71 μs 89.08 μs 0.88 0.02 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<Int32>.SortedSet(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 45.67613022941969 < 50.054110001727324.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 77.79091002341295 (T) = (0 -45967.448897016846) / Math.Sqrt((636384.9539038617 / (299)) + (92727.96894886105 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.11877436088007147 = (52163.086111434626 - 45967.448897016846) / 52163.086111434626 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 164.9414298245614 < 168.83490861742422.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 28.2660621643388 (T) = (0 -166082.7369351791) / Math.Sqrt((4395956.838199926 / (299)) + (2980543.748657064 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.0618742612189658 = (177036.75538311191 - 166082.7369351791) / 177036.75538311191 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CreateAddAndClear&lt;Int32&gt;.ImmutableList(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 89.08496428571429 < 96.82860696067956.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 61.842070840397945 (T) = (0 -89169.62147396228) / Math.Sqrt((2307053.8824507645 / (299)) + (872604.1082623651 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.13116046066080025 = (102630.71307940323 - 89169.62147396228) / 102630.71307940323 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.Reflection.Attributes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsDefinedClassHitInherit - Duration of single invocation 475.86 ns 438.57 ns 0.92 0.07 False Trace Trace
IsDefinedMethodBaseHitInherit - Duration of single invocation 505.33 ns 464.21 ns 0.92 0.06 False
IsDefinedMethodOverrideMissInherit - Duration of single invocation 501.27 ns 469.46 ns 0.94 0.08 False
IsDefinedClassHit - Duration of single invocation 487.33 ns 440.26 ns 0.90 0.12 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Reflection.Attributes*'

Payloads

Baseline
Compare

Histogram

System.Reflection.Attributes.IsDefinedClassHitInherit


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 438.57259661841033 < 456.7786079876879.
IsChangePoint: Marked as a change because one of 1/9/2023 8:29:54 AM, 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 39.06867646681837 (T) = (0 -438.84170280776834) / Math.Sqrt((207.86481978433716 / (299)) + (15.803539921155485 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09571512165522761 = (485.2914311815499 - 438.84170280776834) / 485.2914311815499 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```### Baseline Jit Disasm

```assembly
; System.Reflection.Attributes.IsDefinedClassHitInherit()
       mov       rcx,22ECAF79ED8
       mov       rdx,22ECAF79F00
       mov       r8d,1
       jmp       qword ptr [7FFA756193C8]; System.RuntimeType.IsDefined(System.Type, Boolean)
; Total bytes of code 32
; System.RuntimeType.IsDefined(System.Type, Boolean)
       push      rdi
       push      rsi
       sub       rsp,28
       mov       rsi,rcx
       mov       edi,r8d
       test      rdx,rdx
       je        short M01_L02
       mov       rcx,offset MT_System.RuntimeType
       cmp       [rdx],rcx
       jne       short M01_L01
M01_L00:
       test      rdx,rdx
       je        short M01_L03
       mov       r8,offset MT_System.RuntimeType
       cmp       [rdx],r8
       jne       short M01_L03
       movzx     r8d,dil
       mov       rcx,rsi
       add       rsp,28
       pop       rsi
       pop       rdi
       jmp       qword ptr [7FFA757EF000]; System.Reflection.CustomAttribute.IsDefined(System.RuntimeType, System.RuntimeType, Boolean)
M01_L01:
       mov       rcx,rdx
       mov       rax,[rdx]
       mov       rax,[rax+58]
       call      qword ptr [rax]
       mov       rdx,rax
       jmp       short M01_L00
M01_L02:
       mov       ecx,0E9
       mov       rdx,7FFA754F4000
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FFA757E5BA0]
       int       3
M01_L03:
       mov       rcx,offset MT_System.ArgumentException
       call      CORINFO_HELP_NEWSFAST
       mov       rsi,rax
       call      qword ptr [7FFA75785648]
       mov       rdi,rax
       mov       ecx,0E9
       mov       rdx,7FFA754F4000
       call      CORINFO_HELP_STRCNS
       mov       r8,rax
       mov       rdx,rdi
       mov       rcx,rsi
       call      qword ptr [7FFA756BDBD0]
       mov       rcx,rsi
       call      CORINFO_HELP_THROW
       int       3
; Total bytes of code 189

Compare Jit Disasm

; System.Reflection.Attributes.IsDefinedClassHitInherit()
       mov       rcx,21CB106B4F8
       mov       rdx,21CB106B520
       mov       r8d,1
       jmp       qword ptr [7FF8D19B93C8]; System.RuntimeType.IsDefined(System.Type, Boolean)
; Total bytes of code 32
; System.RuntimeType.IsDefined(System.Type, Boolean)
       push      rdi
       push      rsi
       sub       rsp,28
       mov       rsi,rcx
       mov       edi,r8d
       test      rdx,rdx
       je        short M01_L02
       mov       rcx,offset MT_System.RuntimeType
       cmp       [rdx],rcx
       jne       short M01_L01
M01_L00:
       test      rdx,rdx
       je        short M01_L03
       mov       r8,offset MT_System.RuntimeType
       cmp       [rdx],r8
       jne       short M01_L03
       movzx     r8d,dil
       mov       rcx,rsi
       add       rsp,28
       pop       rsi
       pop       rdi
       jmp       qword ptr [7FF8D1B8F000]; System.Reflection.CustomAttribute.IsDefined(System.RuntimeType, System.RuntimeType, Boolean)
M01_L01:
       mov       rcx,rdx
       mov       rax,[rdx]
       mov       rax,[rax+58]
       call      qword ptr [rax]
       mov       rdx,rax
       jmp       short M01_L00
M01_L02:
       mov       ecx,0E9
       mov       rdx,7FF8D1894000
       call      CORINFO_HELP_STRCNS
       mov       rcx,rax
       call      qword ptr [7FF8D1B85BA0]
       int       3
M01_L03:
       mov       rcx,offset MT_System.ArgumentException
       call      CORINFO_HELP_NEWSFAST
       mov       rsi,rax
       call      qword ptr [7FF8D1B25648]
       mov       rdi,rax
       mov       ecx,0E9
       mov       rdx,7FF8D1894000
       call      CORINFO_HELP_STRCNS
       mov       r8,rax
       mov       rdx,rdi
       mov       rcx,rsi
       call      qword ptr [7FF8D1A5DBD0]
       mov       rcx,rsi
       call      CORINFO_HELP_THROW
       int       3
; Total bytes of code 189

System.Reflection.Attributes.IsDefinedMethodBaseHitInherit


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 464.21334104088754 < 479.90053645031685.
IsChangePoint: Marked as a change because one of 1/9/2023 3:58:12 PM, 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 43.203907879421294 (T) = (0 -461.2785938687879) / Math.Sqrt((223.54645906472078 / (299)) + (18.74129844932296 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.10590987142709027 = (515.919569098757 - 461.2785938687879) / 515.919569098757 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.IsDefinedMethodOverrideMissInherit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 469.46392847473214 < 478.48772646293617.
IsChangePoint: Marked as a change because one of 1/9/2023 3:58:12 PM, 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 15.080878984945256 (T) = (0 -469.3222084596509) / Math.Sqrt((213.01727900607807 / (299)) + (182.41604652610516 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.08794220685143164 = (514.5750762563807 - 469.3222084596509) / 514.5750762563807 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Reflection.Attributes.IsDefinedClassHit

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 440.26146786817515 < 461.80375843954914.
IsChangePoint: Marked as a change because one of 1/8/2023 4:08:48 AM, 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 37.403131333265854 (T) = (0 -439.511091456915) / Math.Sqrt((216.90719751509855 / (299)) + (17.828124148878537 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09540303567479137 = (485.8639911363972 - 439.511091456915) / 485.8639911363972 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array_Comparison - Duration of single invocation 29.66 μs 25.23 μs 0.85 0.01 False
Array_ComparerClass - Duration of single invocation 29.64 μs 25.04 μs 0.84 0.00 False
Array_ComparerStruct - Duration of single invocation 35.00 μs 31.33 μs 0.90 0.01 False
LinqQuery - Duration of single invocation 38.17 μs 33.75 μs 0.88 0.01 False
LinqOrderByExtension - Duration of single invocation 38.33 μs 34.67 μs 0.90 0.01 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.Array_Comparison(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.229884 < 28.35812238571429.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 173.16249586197134 (T) = (0 -25112.597280885777) / Math.Sqrt((56485.88749501119 / (299)) + (11228.252987832124 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.1542223824374112 = (29691.72600388353 - 25112.597280885777) / 29691.72600388353 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.Array_ComparerClass(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 25.040995999999996 < 28.205588033333335.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 149.07867568162393 (T) = (0 -25081.66954845155) / Math.Sqrt((73843.08487620296 / (299)) + (16899.548970046642 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.15922131745241758 = (29831.47654559153 - 25081.66954845155) / 29831.47654559153 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.Array_ComparerStruct(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 31.33234133333334 < 33.27548114358975.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 55.59867456333919 (T) = (0 -31305.1419062604) / Math.Sqrt((177432.9135300424 / (299)) + (103669.80595089601 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.11454730078167981 = (35354.95677397184 - 31305.1419062604) / 35354.95677397184 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.LinqQuery(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.74641384615384 < 36.69364468571428.
IsChangePoint: Marked as a change because one of 1/6/2023 2:04:15 PM, 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 32.981985973214094 (T) = (0 -34905.49433166833) / Math.Sqrt((294584.7940115831 / (299)) + (178904.21963234092 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.08275603027196897 = (38054.75476935328 - 34905.49433166833) / 38054.75476935328 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;Int32&gt;.LinqOrderByExtension(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 34.67483428571428 < 36.412250436813196.
IsChangePoint: Marked as a change because one of 1/6/2023 5:32:27 AM, 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 40.58332838879311 (T) = (0 -34934.77786147186) / Math.Sqrt((263286.858829293 / (299)) + (106125.97764062008 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.0806625813621342 = (37999.952088573635 - 34934.77786147186) / 37999.952088573635 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.Collections.ContainsTrueComparer<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 37.64 μs 30.41 μs 0.81 0.00 False
ImmutableSortedSet - Duration of single invocation 37.72 μs 33.34 μs 0.88 0.00 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrueComparer&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrueComparer<Int32>.SortedSet(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 30.410661003236243 < 35.26340905629143.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 58.92581861955677 (T) = (0 -30533.26449607834) / Math.Sqrt((197245.08827584476 / (299)) + (255394.99499278617 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.1761147263967729 = (37060.09255699208 - 30533.26449607834) / 37060.09255699208 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrueComparer&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 33.33773809523809 < 35.60359647509266.
IsChangePoint: Marked as a change because one of 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 72.5578858344467 (T) = (0 -32742.939104785375) / Math.Sqrt((90310.0255664583 / (299)) + (84680.61765431883 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.12493697532826514 = (37417.80669691573 - 32742.939104785375) / 37417.80669691573 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 765e3dbb2d3aa2b114d8c07e44ae2ca926df16b4
Compare fe1607679b977b453d43c4de276968dd9ae6d5ec
Diff Diff

Improvements in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 41.03 μs 37.16 μs 0.91 0.01 False
ImmutableSortedSet - Duration of single invocation 42.15 μs 36.49 μs 0.87 0.01 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.SortedSet(Size: 512)


Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 37.15679056754596 < 39.07662562882765.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 41.36812197885888 (T) = (0 -36869.76000015153) / Math.Sqrt((752198.9934455368 / (299)) + (192308.76841529066 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.10638003772565663 = (41258.881355240395 - 36869.76000015153) / 41258.881355240395 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 36.49460145578721 < 39.655365546997814.
IsChangePoint: Marked as a change because one of 12/14/2022 5:20:21 PM, 2/9/2023 4:27:10 AM, 2/14/2023 2:39:38 AM falls between 2/4/2023 10:21:07 PM and 2/14/2023 2:39:38 AM.
IsImprovementStdDev: Marked as improvement because 40.72026905086423 (T) = (0 -37684.4274264379) / Math.Sqrt((673584.2143392202 / (299)) + (161763.53758866794 / (22))) is greater than 1.9674283869023508 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (22) - 2, .975) and 0.09576245422236819 = (41675.36241157716 - 37684.4274264379) / 41675.36241157716 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions