[Perf] Windows/arm64: 61 Regressions on 5/19/2023 1:23:34 PM #18096

performanceautofiler · 2023-05-25T12:18:35Z

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Type

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
op_Equality - Duration of single invocation	0.00 ns	1.00 ns	∞	0.64	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Type*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Type.op_Equality

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0005790361523093 > 0.03471310710829168.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/2/2023 12:51:17 PM, 5/8/2023 9:53:14 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -11.19939252455023 (T) = (0 -0.8831170193591498) / Math.Sqrt((0.023006911612154502 / (15)) + (0.04734369882703656 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -10.40786606573086 = (0.07741298979762976 - 0.8831170193591498) / 0.07741298979762976 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchF.Whetsto

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	745.42 ms	828.78 ms	1.11	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Whetsto*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.Whetsto.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 828.7836857142858 > 782.6540472500001.
IsChangePoint: Marked as a change because one of 2/13/2023 5:42:59 PM, 3/26/2023 5:20:11 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -183.54175350085512 (T) = (0 -829800654.4716818) / Math.Sqrt((2521125404719.4097 / (15)) + (558459473542.9811 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.11310241432451396 = (745484551.8192917 - 829800654.4716818) / 745484551.8192917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentBag - Duration of single invocation	317.08 ms	379.61 ms	1.20	0.09	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.AddRemoveFromDifferentThreads&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>.ConcurrentBag(Size: 2000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 379.61281875 > 333.9694095592106.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -31.801378720969538 (T) = (0 -400280190.6730769) / Math.Sqrt((21300487468634.418 / (15)) + (65075487410089.23 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.25218210924119316 = (319666115.43079925 - 400280190.6730769) / 319666115.43079925 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOfAnyFiveValues - Duration of single invocation	986.79 ns	1.07 μs	1.08	0.00	True
SequenceCompareToDifferent - Duration of single invocation	1.60 ns	2.82 ns	1.76	0.49	False
IndexOfAnyFourValues - Duration of single invocation	824.49 ns	905.91 ns	1.10	0.00	True
LastIndexOfValue - Duration of single invocation	51.62 ns	58.76 ns	1.14	0.03	False
IndexOfAnyFourValues - Duration of single invocation	50.34 ns	54.73 ns	1.09	0.01	False
IndexOfAnyFiveValues - Duration of single invocation	60.73 ns	64.61 ns	1.06	0.00	False
EndsWith - Duration of single invocation	0.59 ns	2.00 ns	3.39	0.53	False
SequenceEqual - Duration of single invocation	0.90 ns	2.73 ns	3.04	0.55	False
IndexOfValue - Duration of single invocation	48.50 ns	59.22 ns	1.22	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.068541409058643 > 1035.9277728160678.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -375.51808303094015 (T) = (0 -1068.3102897453318) / Math.Sqrt((0.3119408441887276 / (15)) + (0.3465075560237005 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.08291810747718642 = (986.5106902996704 - 1068.3102897453318) / 986.5106902996704 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareToDifferent(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.8163210121141704 > 1.7079414529872565.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/18/2023 3:09:06 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -11.01816589090237 (T) = (0 -2.3759710253369715) / Math.Sqrt((0.006230711863395296 / (12)) + (0.06987800733847346 / (16))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (16) - 2, .025) and -0.47966009283561617 = (1.6057546167807146 - 2.3759710253369715) / 1.6057546167807146 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 905.9054088997818 > 864.0978952504881.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -307.4127588822405 (T) = (0 -905.319364601452) / Math.Sqrt((0.3963435004488222 / (15)) + (0.5855788938336356 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.09983977248595163 = (823.1375035248743 - 905.319364601452) / 823.1375035248743 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.LastIndexOfValue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.757690314797 > 53.389530604068995.
IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 4/27/2023 10:53:57 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -23.569963255153336 (T) = (0 -58.98605658887015) / Math.Sqrt((1.3636223106005971 / (15)) + (0.3369453108497769 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.1581823999692975 = (50.92985059212938 - 58.98605658887015) / 50.92985059212938 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 54.72537195659344 > 53.082851289449415.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -15.842351803952933 (T) = (0 -54.88965048537154) / Math.Sqrt((0.3077117642100171 / (14)) + (0.5291312620675351 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.07788859144409872 = (50.92330591590478 - 54.88965048537154) / 50.92330591590478 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.6143824301757 > 63.542388463843444.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/27/2023 10:53:57 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -22.524792409013713 (T) = (0 -64.60638662998963) / Math.Sqrt((0.3925827652594718 / (15)) + (0.2293356326654387 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.0787224253083712 = (59.89157647438431 - 64.60638662998963) / 59.89157647438431 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.EndsWith(Size: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0034407880341054 > 1.0907502589124334.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -2.577405395211302 (T) = (0 -2.005657779159591) / Math.Sqrt((1.4283936040282295 / (15)) + (0.11339795861276358 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.7074183973001169 = (1.1746727002186867 - 2.005657779159591) / 1.1746727002186867 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceEqual(Size: 4)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7332540059713883 > 0.9277042687502376.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 6:15:03 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -7.645860683478148 (T) = (0 -3.520153954652793) / Math.Sqrt((1.090179572500375 / (14)) + (0.17579878812382685 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -1.9123149157990094 = (1.2087133625406783 - 3.520153954652793) / 1.2087133625406783 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfValue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.22448784500751 > 50.529368574943184.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -52.653427885088135 (T) = (0 -59.22791786039896) / Math.Sqrt((0.30085783693481505 / (15)) + (0.3329945868891131 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.23454946290617779 = (47.97532998068309 - 59.22791786039896) / 47.97532998068309 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LastIndexOfValue - Duration of single invocation	26.40 ns	38.49 ns	1.46	0.08	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.LastIndexOfValue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.49296762268035 > 27.670639036713258.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -35.39576488347328 (T) = (0 -38.542747369997365) / Math.Sqrt((0.7679091986169674 / (15)) + (0.7957653827968025 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.44487048109971983 = (26.67557256804202 - 38.542747369997365) / 26.67557256804202 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IterateGetPositionMemory - Duration of single invocation	30.15 ns	32.64 ns	1.08	0.15	False
IterateGetPositionArray - Duration of single invocation	17.54 ns	20.49 ns	1.17	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionMemory

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.6444869739794 > 31.428880608270028.
IsChangePoint: Marked as a change because one of 3/10/2023 4:14:40 PM, 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -15.963782217252101 (T) = (0 -32.71080371042895) / Math.Sqrt((0.02959711074520723 / (15)) + (0.35853511864771487 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.09157967129534747 = (29.966483043433687 - 32.71080371042895) / 29.966483043433687 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.IterateGetPositionArray

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.48507549071174 > 18.409303701225344.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/14/2023 2:39:38 AM, 3/10/2023 4:14:40 PM, 3/16/2023 2:30:29 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -7.67953149199861 (T) = (0 -19.003303326030796) / Math.Sqrt((0.37409297603085684 / (14)) + (0.4376994392566526 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.11025866307899433 = (17.11610452408487 - 19.003303326030796) / 17.11610452408487 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Subtract - Duration of single invocation	8.42 ns	10.05 ns	1.19	0.18	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 16,16 bits)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.051523330341292 > 9.86780459001272.
IsChangePoint: Marked as a change because one of 4/18/2023 7:42:00 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -4.8613421156230885 (T) = (0 -9.99934115258754) / Math.Sqrt((0.2223413834321661 / (15)) + (0.19762645763700856 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.09199066179063953 = (9.156984123097429 - 9.99934115258754) / 9.156984123097429 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParseHex - Duration of single invocation	8.59 ns	11.06 ns	1.29	0.10	False
TryParseHex - Duration of single invocation	27.37 ns	29.06 ns	1.06	0.01	False
TryParseHex - Duration of single invocation	3.26 ns	4.72 ns	1.45	0.53	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParseHex(value: "3039")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.055968072706959 > 8.733066646736265.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -49.6792558419427 (T) = (0 -10.976828952937948) / Math.Sqrt((0.02116850131060033 / (15)) + (0.019609575133217726 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.32371121191335106 = (8.292465043845592 - 10.976828952937948) / 8.292465043845592 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParseHex(value: "FFFFFFFFFFFFFFFF")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.056049760164036 > 28.706357587957555.
IsChangePoint: Marked as a change because one of 3/17/2023 1:20:12 AM, 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -20.664435885750475 (T) = (0 -28.9977067750792) / Math.Sqrt((0.045192480149480026 / (15)) + (0.04019545277610622 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.05896249332416025 = (27.383129202294306 - 28.9977067750792) / 27.383129202294306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParseHex(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.718036110601231 > 3.4085684091733075.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/25/2023 1:02:54 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -21.50927122939714 (T) = (0 -4.690145550564664) / Math.Sqrt((0.03276422541397964 / (15)) + (0.035328532693879526 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.472938214095065 = (3.184210651664142 - 4.690145550564664) / 3.184210651664142 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-05-25T12:18:39Z

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentBag - Duration of single invocation	12.56 μs	17.53 μs	1.40	0.03	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.ConcurrentBag(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.527475587789155 > 13.142785289205257.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -52.65268666418597 (T) = (0 -17481.794045901195) / Math.Sqrt((95723.16488075127 / (15)) + (25991.964184139637 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.3806963604295452 = (12661.577553852952 - 17481.794045901195) / 12661.577553852952 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
TryParseHex - Duration of single invocation	2.94 ns	4.84 ns	1.64	0.48	False
TryParseHex - Duration of single invocation	7.81 ns	10.00 ns	1.28	0.18	False
TryParseHex - Duration of single invocation	13.89 ns	16.36 ns	1.18	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.TryParseHex(value: "0")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.836506323145075 > 3.1958598901445914.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/9/2023 11:04:55 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -29.810223556616176 (T) = (0 -4.87997245384831) / Math.Sqrt((0.04069253454477362 / (14)) + (0.0026912185816376903 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.5171375473337941 = (3.216565605685744 - 4.87997245384831) / 3.216565605685744 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParseHex(value: "3039")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.997209598588658 > 8.26003861146036.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -27.978737609444646 (T) = (0 -9.83968557295194) / Math.Sqrt((0.016089880954092188 / (15)) + (0.05143518519976314 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.2525824448201704 = (7.855519302256064 - 9.83968557295194) / 7.855519302256064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParseHex(value: "FFFFFFFF")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.364604893826034 > 14.602836018094564.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 4/10/2023 1:33:42 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -39.368813919958235 (T) = (0 -16.19901702800669) / Math.Sqrt((0.01821961659248186 / (14)) + (0.028461801114050084 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.16766507528355776 = (13.872999519209644 - 16.19901702800669) / 13.872999519209644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.AddRemoveFromDifferentThreads<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentBag - Duration of single invocation	314.67 ms	359.35 ms	1.14	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.AddRemoveFromDifferentThreads&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.AddRemoveFromDifferentThreads<Int32>.ConcurrentBag(Size: 2000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 359.350015 > 315.3152415.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -21.50972158796152 (T) = (0 -351255328.84615386) / Math.Sqrt((41627610006387.09 / (14)) + (23852274257588.14 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.15510433611896465 = (304089698.10150373 - 351255328.84615386) / 304089698.10150373 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf2.CastingPerf

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FooObjIsDescendant - Duration of single invocation	638.53 μs	677.40 μs	1.06	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf2.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf2.CastingPerf.FooObjIsDescendant

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 677.4010677083334 > 670.137703125.
IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -77.44686547493879 (T) = (0 -675845.4655681439) / Math.Sqrt((65349.208880442624 / (15)) + (3008573.957223231 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.05892219662716906 = (638239.0205067154 - 675845.4655681439) / 638239.0205067154 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SerializeToWriter - Duration of single invocation	9.32 μs	12.07 μs	1.30	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.073158063687725 > 9.77394264549404.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -43.18368570868108 (T) = (0 -11827.074799889484) / Math.Sqrt((1352.7067485904022 / (15)) + (42933.74881105932 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.2701263223514647 = (9311.731118203485 - 11827.074799889484) / 9311.731118203485 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IndexerSet<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Span - Duration of single invocation	168.74 ns	188.40 ns	1.12	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.Span(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 188.40308595299481 > 177.0636367727448.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -85.08575252414792 (T) = (0 -188.0745991113909) / Math.Sqrt((0.04924637816618577 / (15)) + (0.6332435009099192 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.11502388426107539 = (168.67315737907055 - 188.0745991113909) / 168.67315737907055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Sort<IntStruct>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
List - Duration of single invocation	6.24 μs	9.00 μs	1.44	0.07	True
Array - Duration of single invocation	6.05 μs	8.67 μs	1.43	0.08	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<IntStruct>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.99972 > 6.45655842631579.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -49.99512289509321 (T) = (0 -8891.957052631578) / Math.Sqrt((10512.875819531226 / (15)) + (29975.5435376297 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.4457128661854628 = (6150.569217864923 - 8891.957052631578) / 6150.569217864923 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<IntStruct>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.666797999999998 > 6.383701053947369.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -67.3721681903976 (T) = (0 -8692.469201641927) / Math.Sqrt((7343.19527331745 / (14)) + (12540.951155098433 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.42673400829530167 = (6092.564662440416 - 8692.469201641927) / 6092.564662440416 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Devirtualization.EqualityComparer

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ValueTupleCompareWrapped - Duration of single invocation	4.73 ns	6.42 ns	1.36	0.54	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Devirtualization.EqualityComparer*'

Payloads

Baseline
Compare

Histogram

Devirtualization.EqualityComparer.ValueTupleCompareWrapped

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.415116400403436 > 4.7786524741766465.
IsChangePoint: Marked as a change because one of 2/6/2023 2:52:08 AM, 5/3/2023 1:18:01 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -11.690710961700512 (T) = (0 -6.9228505003460015) / Math.Sqrt((0.3658975777847552 / (14)) + (0.1353134090333497 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.4767206006813164 = (4.687989384824724 - 6.9228505003460015) / 4.687989384824724 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	4.77 μs	5.68 μs	1.19	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.684063710927409 > 5.109702898461829.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -6.644484089818056 (T) = (0 -5646.435024018703) / Math.Sqrt((128283.36001965683 / (15)) + (203198.93637829894 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.2239839797891586 = (4613.160888749008 - 5646.435024018703) / 4613.160888749008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Add_Remove_SteadyState<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentBag - Duration of single invocation	77.16 ns	86.50 ns	1.12	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Add_Remove_SteadyState<String>.ConcurrentBag(Count: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 86.50056325630786 > 81.05373295121936.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -11.213595672647394 (T) = (0 -87.50408914980348) / Math.Sqrt((0.8740681197917398 / (14)) + (9.875359978231062 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.13146624770636992 = (77.33689743479817 - 87.50408914980348) / 77.33689743479817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Utf8Encoding

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetBytes - Duration of single invocation	123.53 μs	135.12 μs	1.09	0.03	False
GetBytes - Duration of single invocation	150.90 μs	170.01 μs	1.13	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetBytes(Input: Cyrillic)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 135.1240070812808 > 129.92150010545558.
IsChangePoint: Marked as a change because one of 3/6/2023 6:03:21 AM, 3/14/2023 8:42:09 PM, 3/21/2023 2:53:21 PM, 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -113.66252806912455 (T) = (0 -135103.90978238347) / Math.Sqrt((83752.3499005488 / (15)) + (54995.58361395787 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.09092094015693324 = (123843.90546481604 - 135103.90978238347) / 123843.90546481604 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Utf8Encoding.GetBytes(Input: Chinese)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 170.01000864055302 > 158.3389490685096.
IsChangePoint: Marked as a change because one of 5/6/2023 11:12:53 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -212.40675508938006 (T) = (0 -169956.95100984428) / Math.Sqrt((71379.45857475641 / (14)) + (40480.1741342697 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.1277221920282206 = (150708.17282062603 - 169956.95100984428) / 150708.17282062603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-25T12:18:41Z

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark	Baseline	Test	Test/Base	Edge Detector
GetInt64 - Duration of single invocation	744.85 ns	828.98 ns	1.11	False
GetInt16 - Duration of single invocation	765.16 ns	1.14 μs	1.49	True
GetUInt16 - Duration of single invocation	711.56 ns	1.15 μs	1.62	True
GetByte - Duration of single invocation	723.09 ns	1.14 μs	1.58	True
GetSByte - Duration of single invocation	836.65 ns	1.22 μs	1.46	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetInt64

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 828.9770943246559 > 784.840645358469.
IsChangePoint: Marked as a change because one of 4/8/2023 3:10:58 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -49.15853514242204 (T) = (0 -832.2301093302132) / Math.Sqrt((26.10756527989579 / (15)) + (14.638781907698615 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.11112144993511537 = (749.0001289947303 - 832.2301093302132) / 749.0001289947303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetInt16

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.139920536168402 > 804.9754070919571.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -243.03513610453797 (T) = (0 -1135.5659330610165) / Math.Sqrt((13.138938798106079 / (15)) + (18.34893080718718 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.4786071635981079 = (767.9970454746616 - 1135.5659330610165) / 767.9970454746616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt16

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.150145283906367 > 746.3128378670154.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -179.8703279523672 (T) = (0 -1143.09538086351) / Math.Sqrt((13.16890157060995 / (15)) + (63.20646272128278 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.6051145835380723 = (712.1581179231722 - 1143.09538086351) / 712.1581179231722 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetByte

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1415545398520954 > 760.1987923049178.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -174.08640671399212 (T) = (0 -1142.8973163171313) / Math.Sqrt((13.956611913422536 / (15)) + (62.71826181608239 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.5758142541863701 = (725.2741325831172 - 1142.8973163171313) / 725.2741325831172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetSByte

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2211502906976746 > 878.8824314769156.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -179.69957435185384 (T) = (0 -1216.3432168198285) / Math.Sqrt((1.4722611663456278 / (15)) + (56.49379873329378 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.4522988807428388 = (837.5295422645228 - 1216.3432168198285) / 837.5295422645228 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Base64DecodeInPlace - Duration of single invocation	263.11 ms	335.25 ms	1.27	0.01	False
Base64EncodeInPlace - Duration of single invocation	172.81 ms	345.98 ms	2.00	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests.Base64DecodeInPlace(NumberOfBytes: 200000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 335.25035789473685 > 276.25752925.
IsChangePoint: Marked as a change because one of 5/9/2023 7:27:03 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -9.566694624361837 (T) = (0 -344103903.1488485) / Math.Sqrt((1383495725304418.8 / (14)) + (7325550780607.305 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.38343368669842065 = (248731765.35845107 - 344103903.1488485) / 248731765.35845107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests.Base64EncodeInPlace(NumberOfBytes: 200000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 345.977925 > 181.38983625000003.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -57.00863987285303 (T) = (0 -335804909.8172089) / Math.Sqrt((29554631641.099335 / (15)) + (106294142529495.39 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.9436333900038406 = (172771733.36507937 - 335804909.8172089) / 172771733.36507937 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Span.Sorting

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
QuickSortArray - Duration of single invocation	12.42 μs	14.09 μs	1.13	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Span.Sorting*'

Payloads

Baseline
Compare

Histogram

Span.Sorting.QuickSortArray(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.087914285714286 > 13.03435818111455.
IsChangePoint: Marked as a change because one of 3/23/2023 6:11:55 PM, 3/24/2023 12:04:20 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -2.097315790281355 (T) = (0 -14025.160121698476) / Math.Sqrt((4368571.219599407 / (15)) + (9385.616750958163 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.08790392298719392 = (12891.910604741492 - 14025.160121698476) / 12891.910604741492 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IterateGetPositionArray - Duration of single invocation	17.56 ns	19.56 ns	1.11	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateGetPositionArray

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.559522806353137 > 18.060435146449407.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 2/14/2023 11:21:41 AM, 3/10/2023 4:14:40 PM, 3/16/2023 2:30:29 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -12.190619998808915 (T) = (0 -19.016222867001943) / Math.Sqrt((0.04317526601383314 / (15)) + (0.22726305726743015 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.10068245480001123 = (17.276756601391543 - 19.016222867001943) / 17.276756601391543 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Edge Detector
List - Duration of single invocation	30.30 μs	35.80 μs	1.18	True
Span - Duration of single invocation	27.93 μs	33.77 μs	1.21	True
Queue - Duration of single invocation	30.69 μs	35.07 μs	1.14	True
ImmutableArray - Duration of single invocation	32.09 μs	37.34 μs	1.16	True
ICollection - Duration of single invocation	30.60 μs	35.89 μs	1.17	True
Array - Duration of single invocation	30.72 μs	37.03 μs	1.21	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.80255849753694 > 31.66621266570032.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -94.30695482430592 (T) = (0 -35748.0912820327) / Math.Sqrt((15641.692420121632 / (15)) + (31125.1624489645 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.18295799449677264 = (30219.239777182323 - 35748.0912820327) / 30219.239777182323 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.Span(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.76769954507858 > 29.24699802289804.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -132.28926867274916 (T) = (0 -33758.22578844788) / Math.Sqrt((23662.137192731607 / (15)) + (5310.258384268991 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.21158438575042585 = (27862.876235020856 - 33758.22578844788) / 27862.876235020856 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.06652078302863 > 32.22154535402042.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -97.7971766986943 (T) = (0 -35228.97683067751) / Math.Sqrt((14080.958237710836 / (14)) + (13944.543435357182 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.14489778245905902 = (30770.412320138526 - 35228.97683067751) / 30770.412320138526 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.341423585429 > 33.7565703330105.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -76.55686305377539 (T) = (0 -37415.716912386815) / Math.Sqrt((21320.68932758415 / (14)) + (40539.09378645629 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.16197438940528572 = (32200.12183877537 - 37415.716912386815) / 32200.12183877537 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.ICollection(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.88788972286373 > 32.14185606559537.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -92.44328873980245 (T) = (0 -35846.62128500339) / Math.Sqrt((30856.442977565894 / (15)) + (13744.073125065484 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.16810993171120614 = (30687.712099571305 - 35846.62128500339) / 30687.712099571305 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.03434650455927 > 32.222999712234696.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -137.01186618025287 (T) = (0 -36853.11517391593) / Math.Sqrt((6201.114787105004 / (15)) + (20858.143967074848 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.20048861710971946 = (30698.42949668528 - 36853.11517391593) / 30698.42949668528 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.BubbleSort2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	31.11 ms	36.53 ms	1.17	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BubbleSort2*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.BubbleSort2.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.529563125 > 32.775285000000004.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -121.88925807650058 (T) = (0 -36304693.37912088) / Math.Sqrt((5601020344.379959 / (14)) + (17340439966.12823 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.16252649327237 = (31229132.057823133 - 36304693.37912088) / 31229132.057823133 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	18.17 ms	19.32 ms	1.06	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.320673076923075 > 19.066956125000004.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -13.75524811678317 (T) = (0 -19776090.785053212) / Math.Sqrt((4152663999.156616 / (15)) + (171403418635.2054 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.08778523146468803 = (18180142.746031746 - 19776090.785053212) / 18180142.746031746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CtorFromCollection<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentBag - Duration of single invocation	20.01 μs	25.93 μs	1.30	0.10	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<String>.ConcurrentBag(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.93299641077361 > 20.973688123884227.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -60.230062959167896 (T) = (0 -25714.776900962115) / Math.Sqrt((10810.005559867926 / (14)) + (108420.67918606369 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.28797048827787997 = (19965.346360804306 - 25714.776900962115) / 19965.346360804306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-25T12:18:44Z

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParseUInt64 - Duration of single invocation	27.18 ns	29.58 ns	1.09	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.580081412024764 > 28.62751970875355.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -26.060039389017664 (T) = (0 -29.430656986647527) / Math.Sqrt((0.04243993968475235 / (14)) + (0.05010385934355748 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.07930275198348902 = (27.268212679492688 - 29.430656986647527) / 27.268212679492688 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentBag - Duration of single invocation	27.43 μs	31.92 μs	1.16	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.ConcurrentBag(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.91607269783604 > 28.832344901315793.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -18.05180283819103 (T) = (0 -31925.457435666074) / Math.Sqrt((19335.586618672045 / (15)) + (791116.7294470404 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.16408476602765434 = (27425.371731827683 - 31925.457435666074) / 27425.371731827683 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	3.49 ms	3.76 ms	1.08	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "[a-z]shing", Options: Compiled)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.762358076923077 > 3.6580668031249997.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -12.878452166011758 (T) = (0 -3735823.27930403) / Math.Sqrt((32344365.53383092 / (14)) + (5138660173.965988 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.07381151988683438 = (3479030.7331567244 - 3735823.27930403) / 3479030.7331567244 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Add_Remove_SteadyState<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentBag - Duration of single invocation	74.59 ns	101.42 ns	1.36	0.10	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Add_Remove_SteadyState&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Add_Remove_SteadyState<Int32>.ConcurrentBag(Count: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 101.41991762788895 > 78.43705724956281.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -4.636036573940384 (T) = (0 -84.02575806000678) / Math.Sqrt((6.062610696259798 / (15)) + (38.63275412131941 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.1128112107513422 = (75.50764878013288 - 84.02575806000678) / 75.50764878013288 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.MDBenchI.MDMulMatrix

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	894.73 ms	996.70 ms	1.11	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDMulMatrix*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDMulMatrix.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 996.69768 > 938.8553682692308.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -268.1247238040669 (T) = (0 -995512324.7111862) / Math.Sqrt((374988467531.73816 / (15)) + (1523662605780.3245 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.11304719148644189 = (894402620.4151404 - 995512324.7111862) / 894402620.4151404 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchF.Simpsn

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	231.19 ms	247.97 ms	1.07	0.00	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchF.Simpsn*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.Simpsn.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 247.97247857142855 > 243.09051374999999.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/3/2023 1:18:01 AM, 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -21.524574559174937 (T) = (0 -245420188.09523806) / Math.Sqrt((2968659744117.3975 / (15)) + (2214329346132.698 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.05621391397332138 = (232358412.29548228 - 245420188.09523806) / 232358412.29548228 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Span - Duration of single invocation	49.91 μs	53.41 μs	1.07	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.Span(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 53.41280579539508 > 52.40438476084027.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -175.37686529967186 (T) = (0 -53268.98871337523) / Math.Sqrt((2387.691948193168 / (15)) + (2640.449944847907 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.0668546922665804 = (49930.87540366241 - 53268.98871337523) / 49930.87540366241 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.AddRemoveFromSameThreads<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ConcurrentBag - Duration of single invocation	367.00 ms	411.16 ms	1.12	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.AddRemoveFromSameThreads&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.AddRemoveFromSameThreads<String>.ConcurrentBag(Size: 2000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 411.1560214285714 > 384.74906137500005.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -81.82189860489659 (T) = (0 -411959350.3578473) / Math.Sqrt((3276970702048.7134 / (14)) + (962630409630.9618 / (13))) is less than -2.0595385527466985 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (14) + (13) - 2, .025) and -0.12390960910754259 = (366541354.4110276 - 411959350.3578473) / 366541354.4110276 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	arm64
OS	Windows 10.0.19041
Queue	SurfaceWindows
Baseline	f73356c1f4b9b292dc4fb02f1dd47e4a61e423f5
Compare	1ec5ff876baf614275dcdc1d6ba435e94d78e4b1
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableHashSet - Duration of single invocation	16.73 μs	19.14 μs	1.14	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.13646549289144 > 17.5542864225262.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/24/2023 5:44:05 PM falls between 5/16/2023 12:30:38 AM and 5/24/2023 5:44:05 PM.
IsRegressionStdDev: Marked as regression because -23.637040426983493 (T) = (0 -18984.57950466439) / Math.Sqrt((4468.8165855324705 / (15)) + (116555.67307541407 / (13))) is less than -2.055529438636913 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (13) - 2, .025) and -0.13615120071808556 = (16709.553704353348 - 18984.57950466439) / 16709.553704353348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler bot assigned EgorBo May 25, 2023

performanceautofiler bot added arch-arm64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr untriaged labels May 25, 2023

AndyAyersMS mentioned this issue Jun 6, 2023

Dynamic PGO Microbenchmark Regressions dotnet/runtime#87194

Closed

[Perf] Windows/arm64: 61 Regressions on 5/19/2023 1:23:34 PM #18096

[Perf] Windows/arm64: 61 Regressions on 5/19/2023 1:23:34 PM #18096

Comments

performanceautofiler bot commented May 25, 2023 • edited Loading

Run Information

Regressions in System.Tests.Perf_Type

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_Type.op_Equality

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in Benchstone.BenchF.Whetsto

Repro

Payloads

Payloads

Histogram

Benchstone.BenchF.Whetsto.Test

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>

Repro

Payloads

Payloads

Histogram

System.Collections.Concurrent.AddRemoveFromDifferentThreads<String>.ConcurrentBag(Size: 2000000)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Memory.Span<Int32>

Repro

Payloads

Payloads

Histogram

System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 512)

Description of detection logic

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareToDifferent(Size: 512)

Description of detection logic

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 512)

Description of detection logic

JIT Disasms

System.Memory.Span<Int32>.LastIndexOfValue(Size: 512)

Description of detection logic

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 33)

Description of detection logic

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 33)

Description of detection logic

JIT Disasms

System.Memory.Span<Int32>.EndsWith(Size: 4)

Description of detection logic

JIT Disasms

System.Memory.Span<Int32>.SequenceEqual(Size: 4)

Description of detection logic

JIT Disasms

System.Memory.Span<Int32>.IndexOfValue(Size: 512)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Memory.Span<Char>

Repro

Payloads

Payloads

Histogram

System.Memory.Span<Char>.LastIndexOfValue(Size: 512)

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

performanceautofiler bot commented May 25, 2023 •

edited

Loading

performanceautofiler bot commented May 25, 2023 •

edited

Loading