[Perf] Linux/x64: 100 Regressions on 5/19/2023 3:32:16 PM #18139

performanceautofiler · 2023-05-25T22:46:29Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
GetInt32 - Duration of single invocation	528.52 ns	684.22 ns	1.29	0.36	True
GetUInt32 - Duration of single invocation	456.51 ns	675.84 ns	1.48	0.01	True
GetInt64 - Duration of single invocation	514.30 ns	540.62 ns	1.05	0.26	False
GetByte - Duration of single invocation	493.34 ns	1.06 μs	2.14	0.00	True
GetUInt16 - Duration of single invocation	471.69 ns	1.05 μs	2.22	0.03	True
GetSByte - Duration of single invocation	522.56 ns	1.13 μs	2.17	0.08	True
GetInt16 - Duration of single invocation	536.83 ns	1.07 μs	1.99	0.29	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetInt32

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 684.2210193876656 > 553.781476182016.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -50.92924043707628 (T) = (0 -683.0352528030054) / Math.Sqrt((358.38885873614237 / (32)) + (0.8323082192689876 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.3336773631032697 = (512.1442949392824 - 683.0352528030054) / 512.1442949392824 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt32

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 675.8424866895881 > 479.02069640083187.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -146.80195842020757 (T) = (0 -672.1220112802578) / Math.Sqrt((8.749990851607988 / (32)) + (29.671688695607084 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.4875114602159394 = (451.8432491153292 - 672.1220112802578) / 451.8432491153292 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetInt64

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 540.6189658411663 > 540.0672794547761.
IsChangePoint: Marked as a change because one of 3/13/2023 6:11:08 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -10.158627822064688 (T) = (0 -545.5062037935229) / Math.Sqrt((451.63858261068384 / (32)) + (105.65679614975146 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.09489498487924088 = (498.2269636148606 - 545.5062037935229) / 498.2269636148606 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetByte

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0579500254168361 > 518.0150960806756.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -455.2042181220963 (T) = (0 -1058.0884413416322) / Math.Sqrt((24.13813527743634 / (32)) + (10.906918740095097 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -1.140103703545185 = (494.4098921883354 - 1058.0884413416322) / 494.4098921883354 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt16

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.049005106263982 > 521.9436954810417.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -241.94651113069088 (T) = (0 -1047.8604380046556) / Math.Sqrt((172.3454986598846 / (32)) + (0.4805710940061378 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -1.1618748600781839 = (484.699858143852 - 1047.8604380046556) / 484.699858143852 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetSByte

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1330918764525384 > 520.5240461137341.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -164.51829540344696 (T) = (0 -1133.4305453533163) / Math.Sqrt((447.29127602327736 / (32)) + (0.37313407413326466 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -1.1889372060315475 = (517.799479231375 - 1133.4305453533163) / 517.799479231375 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetInt16

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0698795481059566 > 563.3255394268948.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -146.05441157237894 (T) = (0 -1069.3865388826698) / Math.Sqrt((460.1931006169656 / (32)) + (0.9326417834666758 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -1.0792318627013697 = (514.3180797033893 - 1069.3865388826698) / 514.3180797033893 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ToUpperInvariant - Duration of single invocation	19.42 ns	23.52 ns	1.21	0.05	False
Remove_IntInt - Duration of single invocation	11.80 ns	13.46 ns	1.14	0.04	False
ToUpper - Duration of single invocation	26.23 ns	30.30 ns	1.16	0.09	False
ToUpper - Duration of single invocation	26.22 ns	29.77 ns	1.14	0.08	False
Remove_IntInt - Duration of single invocation	12.46 ns	14.11 ns	1.13	0.03	False
ToLower - Duration of single invocation	26.54 ns	29.24 ns	1.10	0.06	False
ToLower - Duration of single invocation	26.56 ns	29.16 ns	1.10	0.08	False
ToUpperInvariant - Duration of single invocation	19.49 ns	22.93 ns	1.18	0.04	False
ToUpperInvariant - Duration of single invocation	32.69 ns	36.67 ns	1.12	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.ToUpperInvariant(s: "test")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.518855367488367 > 20.3986998372811.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -14.579131643840784 (T) = (0 -23.14086457308742) / Math.Sqrt((0.3941456895427794 / (31)) + (0.4414940006611672 / (15))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (15) - 2, .025) and -0.14855560226914444 = (20.147796525800892 - 23.14086457308742) / 20.147796525800892 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.45643936027862 > 12.404909983163334.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -8.200472154498273 (T) = (0 -13.38715956591609) / Math.Sqrt((0.33385005199418955 / (32)) + (0.11084923503706078 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.08906157501582379 = (12.292380773531173 - 13.38715956591609) / 12.292380773531173 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToUpper(s: "test")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.295188486394704 > 27.39628531037615.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -11.088062442849822 (T) = (0 -29.201956360271677) / Math.Sqrt((0.7215054397237539 / (32)) + (0.3574243570571762 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.08905025524732843 = (26.814149502806714 - 29.201956360271677) / 26.814149502806714 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToUpper(s: "TeSt")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.76527193813556 > 27.466186426238288.
IsChangePoint: Marked as a change because one of 4/25/2023 8:51:41 AM, 5/19/2023 3:32:16 PM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -10.68561086005034 (T) = (0 -29.13972941398914) / Math.Sqrt((0.7935600545224485 / (33)) + (0.258534461520616 / (14))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (14) - 2, .025) and -0.08179462253033612 = (26.936470941065334 - 29.13972941398914) / 26.936470941065334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.10674745421446 > 13.189408798189588.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -7.750535090284235 (T) = (0 -13.883872130974202) / Math.Sqrt((0.2943162531219381 / (32)) + (0.07589590470884754 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.07141590063860678 = (12.95843390293056 - 13.883872130974202) / 12.95843390293056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToLower(s: "TEST")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.236664870252564 > 27.382825068394617.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -22.285611734369343 (T) = (0 -29.144923366945342) / Math.Sqrt((0.18302506977850783 / (30)) + (0.1077712863615697 / (15))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (15) - 2, .025) and -0.09665444635164959 = (26.57621410637114 - 29.144923366945342) / 26.57621410637114 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToLower(s: "TeSt")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.158188095360405 > 27.90956439265966.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -8.983150076439566 (T) = (0 -29.542511591662947) / Math.Sqrt((0.15184835433405036 / (32)) + (1.5408823772507472 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.11072138788056454 = (26.59758956117233 - 29.542511591662947) / 26.59758956117233 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToUpperInvariant(s: "TeSt")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.931878407779745 > 20.45927318793372.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -15.887616090884968 (T) = (0 -22.844576824000317) / Math.Sqrt((0.43863699552179947 / (32)) + (0.2525493480901699 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.13836284949914704 = (20.06792195832058 - 22.844576824000317) / 20.06792195832058 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.ToUpperInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.66739532048517 > 34.76059718847477.
IsChangePoint: Marked as a change because one of 4/25/2023 8:51:41 AM, 5/19/2023 8:24:48 PM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -11.128481274132096 (T) = (0 -36.10919219364063) / Math.Sqrt((0.5629305107798042 / (34)) + (0.2666398992874171 / (13))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (13) - 2, .025) and -0.0630783446213597 = (33.96663319908164 - 36.10919219364063) / 33.96663319908164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
EncodeUtf8 - Duration of single invocation	28.43 ns	38.40 ns	1.35	0.05	True
EncodeUtf16 - Duration of single invocation	102.07 ns	109.73 ns	1.08	0.02	False
EncodeUtf8 - Duration of single invocation	57.33 ns	62.90 ns	1.10	0.03	False
EncodeUtf16 - Duration of single invocation	24.37 ns	27.17 ns	1.11	0.08	False
EncodeUtf8 - Duration of single invocation	39.76 ns	59.74 ns	1.50	0.02	True
EncodeUtf8 - Duration of single invocation	69.32 ns	95.60 ns	1.38	0.09	True
EncodeUtf16 - Duration of single invocation	44.40 ns	52.33 ns	1.18	0.29	False
EncodeUtf16 - Duration of single invocation	51.68 ns	67.25 ns	1.30	0.34	False
EncodeUtf8 - Duration of single invocation	21.24 ns	24.53 ns	1.15	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,16)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 38.397999835933 > 29.815554840812116.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -9.921544351013834 (T) = (0 -35.63684615660033) / Math.Sqrt((0.047444760561155416 / (32)) + (7.410628920751283 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.25445132938552 = (28.40831311809974 - 35.63684615660033) / 28.40831311809974 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.72776542840451 > 107.22843478911417.
IsChangePoint: Marked as a change because one of 3/17/2023 1:30:52 PM, 5/19/2023 4:22:12 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -16.03797430476544 (T) = (0 -108.74186882098536) / Math.Sqrt((2.6584413284029726 / (31)) + (1.0790365328055906 / (16))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (16) - 2, .025) and -0.061263169684068225 = (102.46456480097879 - 108.74186882098536) / 102.46456480097879 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.90095793087328 > 60.202160273395464.
IsChangePoint: Marked as a change because one of 3/17/2023 5:15:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -8.16906496190465 (T) = (0 -63.20492876609608) / Math.Sqrt((0.5928227061057829 / (31)) + (5.87311777111458 / (14))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (14) - 2, .025) and -0.09361274645339838 = (57.7946160293674 - 63.20492876609608) / 57.7946160293674 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,hello "there",16)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.167371453822273 > 25.437456882201886.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -19.17319435395543 (T) = (0 -27.134139322037036) / Math.Sqrt((0.5856924740847169 / (32)) + (0.02095144943939456 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.11009507434504447 = (24.443076948202986 - 27.134139322037036) / 24.443076948202986 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,16)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.7394758847701 > 41.13940423814788.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -17.036773345353865 (T) = (0 -57.03252256704288) / Math.Sqrt((0.10472912835194814 / (32)) + (16.067127737880188 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.44851013383673094 = (39.37322993797661 - 57.03252256704288) / 39.37322993797661 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: Url,&lorem ipsum=dolor sit amet,512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 95.59642649694898 > 72.53340238060454.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -35.89729937271127 (T) = (0 -92.1573046597354) / Math.Sqrt((0.538137018015087 / (31)) + (6.211918535794639 / (15))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (15) - 2, .025) and -0.3438475724267784 = (68.57720068155776 - 92.1573046597354) / 68.57720068155776 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: JavaScript,&Hello+<World>!,16)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.327846103226264 > 46.47994741223262.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -28.483456043078974 (T) = (0 -51.7821227647119) / Math.Sqrt((2.377215812879235 / (30)) + (0.204090593631642 / (15))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (15) - 2, .025) and -0.20135761765959198 = (43.10300447055101 - 51.7821227647119) / 43.10300447055101 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,16)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.25485953234566 > 56.75409534707578.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -37.73324507212552 (T) = (0 -67.10366428656677) / Math.Sqrt((4.222298741396266 / (32)) + (0.09893949029008854 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.2647026828417326 = (53.05884552706706 - 67.10366428656677) / 53.05884552706706 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: UnsafeRelaxed,hello "there",16)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.527560220665638 > 22.099157023108674.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -11.609579754017666 (T) = (0 -24.225719341500213) / Math.Sqrt((0.07986035504680097 / (32)) + (1.0400714988812905 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.14736939720966255 = (21.114141095636498 - 24.225719341500213) / 21.114141095636498 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

performanceautofiler · 2023-05-25T22:46:32Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Document.Tests.Perf_EnumerateArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EnumerateUsingIndexer - Duration of single invocation	732.12 ns	932.16 ns	1.27	0.09	False
EnumerateUsingIndexer - Duration of single invocation	790.53 ns	935.72 ns	1.18	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Document.Tests.Perf_EnumerateArray*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfNumbers)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 932.1589371403538 > 829.724958592464.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -19.215364039178187 (T) = (0 -1035.0104503652065) / Math.Sqrt((2506.2576626970786 / (32)) + (1722.462821656205 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.3477446273035746 = (767.9573929639373 - 1035.0104503652065) / 767.9573929639373 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfStrings)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 935.7195952388095 > 808.3064192864825.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -12.300857104639139 (T) = (0 -947.9945656842369) / Math.Sqrt((1970.532236106345 / (32)) + (1722.2414014449319 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.20821798257698443 = (784.6221289160735 - 947.9945656842369) / 784.6221289160735 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IndexOfValue - Duration of single invocation	5.61 ns	6.91 ns	1.23	0.21	False
IndexOfAnyTwoValues - Duration of single invocation	6.97 ns	10.33 ns	1.48	0.16	False
SequenceCompareTo - Duration of single invocation	3.25 ns	4.42 ns	1.36	0.26	False
SequenceCompareToDifferent - Duration of single invocation	2.82 ns	4.01 ns	1.42	0.21	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfValue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.912150135471782 > 5.66298169658995.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -25.761478771312333 (T) = (0 -7.091923056203857) / Math.Sqrt((0.04982492698293707 / (32)) + (0.035550264330661725 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.2947253148253048 = (5.477550315111247 - 7.091923056203857) / 5.477550315111247 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.332048462826327 > 7.310659291344664.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 10:30:38 AM, 5/19/2023 10:55:27 PM, 5/21/2023 4:09:07 PM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -2.970970574367623 (T) = (0 -9.586840246361552) / Math.Sqrt((0.7760633010839786 / (41)) + (2.189207196029238 / (4))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (4) - 2, .025) and -0.3041126369870303 = (7.351236369053677 - 9.586840246361552) / 7.351236369053677 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.SequenceCompareTo(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.4202634984667535 > 3.4174248901974806.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -4.336356677737806 (T) = (0 -3.98527740887128) / Math.Sqrt((0.08685663957818379 / (32)) + (0.27289058172060127 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.19352526323156274 = (3.3390808989504173 - 3.98527740887128) / 3.3390808989504173 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.SequenceCompareToDifferent(Size: 33)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.012063555223925 > 3.09478131473116.
IsChangePoint: Marked as a change because one of 4/17/2023 1:54:35 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -8.154632283299502 (T) = (0 -3.520464544838268) / Math.Sqrt((0.0777922530651722 / (32)) + (0.020351137153277406 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.1687312855497496 = (3.0122104100108062 - 3.520464544838268) / 3.0122104100108062 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in ByteMark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
BenchNumericSortJagged - Duration of single invocation	945.45 ms	1.14 secs	1.21	0.18	False
BenchAssignRectangular - Duration of single invocation	1.07 secs	1.23 secs	1.15	0.00	False
BenchLUDecomp - Duration of single invocation	933.59 ms	1.01 secs	1.08	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

Histogram

ByteMark.BenchNumericSortJagged

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1393727013125 > 996.3059215211539.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -36.49168269459199 (T) = (0 -1142496609.963115) / Math.Sqrt((693280948753178.8 / (32)) + (7835313573945.154 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.17709367225365077 = (970608063.6519805 - 1142496609.963115) / 970608063.6519805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

ByteMark.BenchAssignRectangular

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2262207512142855 > 1.1170735931250002.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/1/2023 3:42:23 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -64.6438090222839 (T) = (0 -1226366033.5654945) / Math.Sqrt((11150362657774.084 / (32)) + (89146962185193.55 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.15236157077349505 = (1064219828.8010644 - 1226366033.5654945) / 1064219828.8010644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

ByteMark.BenchLUDecomp

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0102128405 > 984.6962241350002.
IsChangePoint: Marked as a change because one of 3/15/2023 10:57:55 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -34.32066347635161 (T) = (0 -1002087326.3029426) / Math.Sqrt((43032034999738.266 / (32)) + (25228335954874.367 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.06335932538152461 = (942378838.8213946 - 1002087326.3029426) / 942378838.8213946 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Segment

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ReadMultiSegmentSequence - Duration of single invocation	5.48 μs	6.44 μs	1.18	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Segment*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Segment.ReadMultiSegmentSequence(segmentSize: 4096, TestCase: Json4KB)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.438370551253045 > 5.748192680279682.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -61.48869532159136 (T) = (0 -6399.237056481155) / Math.Sqrt((1291.8709349006735 / (32)) + (2794.278776704741 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.16912683784395396 = (5473.518226886582 - 6399.237056481155) / 5473.518226886582 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Tests.Perf_Strings

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
WriteStringsUtf16 - Duration of single invocation	50.47 ms	59.03 ms	1.17	0.12	False
WriteStringsUtf16 - Duration of single invocation	51.87 ms	55.78 ms	1.08	0.14	False
WriteStringsUtf16 - Duration of single invocation	50.30 ms	55.39 ms	1.10	0.14	False
WriteStringsUtf16 - Duration of single invocation	48.10 ms	56.09 ms	1.17	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: False, Escaped: AllEscaped)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.025586821428575 > 51.51849613375.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -17.429436387742932 (T) = (0 -55903253.21961538) / Math.Sqrt((2037697253420.633 / (32)) + (801641356953.6874 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.11944440152484007 = (49938391.88750001 - 55903253.21961538) / 49938391.88750001 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: False, SkipValidation: True, Escaped: AllEscaped)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.77566753333333 > 54.46854068500001.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -14.94639974088398 (T) = (0 -55820250.02591575) / Math.Sqrt((7301469433532.394 / (32)) + (27434459931.928978 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.1473322515845095 = (48652210.332992785 - 55820250.02591575) / 48652210.332992785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: True, Escaped: AllEscaped)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.385806233333334 > 52.879722586250004.
IsChangePoint: Marked as a change because one of 5/11/2023 5:02:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -19.700795818910233 (T) = (0 -56066018.38037851) / Math.Sqrt((2850938798086.266 / (32)) + (63792642950.09811 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.12026821188704866 = (50046959.991783984 - 56066018.38037851) / 50046959.991783984 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: AllEscaped)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.094196571428576 > 50.68195695500001.
IsChangePoint: Marked as a change because one of 3/4/2023 3:36:11 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -24.52685726583575 (T) = (0 -56051031.24663918) / Math.Sqrt((2817721231468.5317 / (32)) + (32130734692.331856 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.15130044950295587 = (48684972.95457303 - 56051031.24663918) / 48684972.95457303 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in BenchmarksGame.BinaryTrees_5

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	118.82 ms	132.25 ms	1.11	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.BinaryTrees_5*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.BinaryTrees_5.RunBench

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 132.246642225 > 124.39691988295456.
IsChangePoint: Marked as a change because one of 3/24/2023 3:46:52 AM, 5/11/2023 1:19:27 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -6.788687429945671 (T) = (0 -132194966.82742856) / Math.Sqrt((14634991831604.56 / (32)) + (35539496301383.695 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.09449770180032778 = (120781401.92526896 - 132194966.82742856) / 120781401.92526896 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
SerializeToUtf8Bytes - Duration of single invocation	8.77 μs	12.14 μs	1.38	0.02	True
SerializeToWriter - Duration of single invocation	8.02 μs	11.71 μs	1.46	0.03	True
SerializeToStream - Duration of single invocation	8.38 μs	11.87 μs	1.42	0.04	True
SerializeToString - Duration of single invocation	8.99 μs	13.09 μs	1.46	0.02	True
SerializeObjectProperty - Duration of single invocation	9.21 μs	12.65 μs	1.37	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToUtf8Bytes(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.144443298718945 > 9.035683627947865.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -41.70841111764362 (T) = (0 -12177.877434501945) / Math.Sqrt((22375.590835120773 / (32)) + (98307.58767977667 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.41180001080264056 = (8625.780805582048 - 12177.877434501945) / 8625.780805582048 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.70657620696108 > 8.40836561211699.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -62.862602074506015 (T) = (0 -11315.548219183798) / Math.Sqrt((28424.598403162898 / (30)) + (24770.904230781987 / (15))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (15) - 2, .025) and -0.3951104022695506 = (8110.862194687808 - 11315.548219183798) / 8110.862194687808 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToStream(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.870151656215922 > 8.779162766372478.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -57.34996375076426 (T) = (0 -11910.902130248613) / Math.Sqrt((16369.207607031496 / (32)) + (46902.28348603282 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.4093037619395698 = (8451.621610557615 - 11910.902130248613) / 8451.621610557615 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeToString(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.09254799385465 > 9.443945393969278.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -51.10574605181022 (T) = (0 -12413.618167769628) / Math.Sqrt((20908.20166900863 / (31)) + (55531.8178211353 / (15))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (15) - 2, .025) and -0.374300557312446 = (9032.680734733489 - 12413.618167769628) / 9032.680734733489 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Json.Serialization.Tests.WriteJson<ImmutableDictionary<String, String>>.SerializeObjectProperty(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.64732333635509 > 9.718611488740912.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -65.12962638668972 (T) = (0 -12667.137349291825) / Math.Sqrt((17728.474358316387 / (32)) + (32383.444116630213 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.365755478136619 = (9274.820824130502 - 12667.137349291825) / 9274.820824130502 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-25T22:46:34Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
InterpolateIntoSpan_Flags - Duration of single invocation	84.34 ns	92.50 ns	1.10	0.03	False
InterpolateIntoStringBuilder_Flags - Duration of single invocation	70.51 ns	79.34 ns	1.13	0.02	False
InterpolateIntoStringBuilder_NonFlags - Duration of single invocation	83.23 ns	92.57 ns	1.11	0.03	False
InterpolateIntoSpan_Flags - Duration of single invocation	69.49 ns	75.06 ns	1.08	0.03	False
InterpolateIntoSpan_NonFlags - Duration of single invocation	81.67 ns	90.30 ns	1.11	0.03	False
InterpolateIntoStringBuilder_Flags - Duration of single invocation	85.55 ns	96.64 ns	1.13	0.02	False
InterpolateIntoString - Duration of single invocation	122.75 ns	131.13 ns	1.07	0.07	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: Red, Green)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 92.50026872397687 > 88.50420297521076.
IsChangePoint: Marked as a change because one of 3/22/2023 7:51:01 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -31.403984206774272 (T) = (0 -92.260446124747) / Math.Sqrt((1.9423415972888105 / (32)) + (0.28512127268273113 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.10631491127279108 = (83.394380013014 - 92.260446124747) / 83.394380013014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: 32)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.33655025355415 > 74.07242541292561.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 4/12/2023 4:44:56 PM, 5/8/2023 3:52:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -52.99933688380903 (T) = (0 -79.16568734051553) / Math.Sqrt((0.3682048799498162 / (31)) + (0.18195132899659244 / (15))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (15) - 2, .025) and -0.11573641366956723 = (70.95375428336695 - 79.16568734051553) / 70.95375428336695 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoStringBuilder_NonFlags(value: Saturday)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 92.56945723349205 > 87.45751581688549.
IsChangePoint: Marked as a change because one of 5/8/2023 3:52:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -22.322388119125204 (T) = (0 -94.46956125837055) / Math.Sqrt((1.5755724489982479 / (32)) + (2.2353967490728017 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.11758449508266747 = (84.53012875002588 - 94.46956125837055) / 84.53012875002588 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoSpan_Flags(value: 32)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.06224136535461 > 72.49351103184085.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -20.56760873515259 (T) = (0 -76.85941374577219) / Math.Sqrt((2.807679861543463 / (32)) + (0.8893753597452173 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.11434344311588202 = (68.97282361249509 - 76.85941374577219) / 68.97282361249509 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoSpan_NonFlags(value: Saturday)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.29796950108437 > 85.8325962055281.
IsChangePoint: Marked as a change because one of 3/22/2023 7:51:01 PM, 4/30/2023 4:58:26 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -41.03253980155933 (T) = (0 -89.8194385075101) / Math.Sqrt((0.8845194126850143 / (32)) + (0.1576320152258002 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.0979707149724196 = (81.80494915091275 - 89.8194385075101) / 81.80494915091275 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoStringBuilder_Flags(value: Red, Green)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 96.63845235848069 > 90.3485339945098.
IsChangePoint: Marked as a change because one of 5/8/2023 3:52:55 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -23.43072626750555 (T) = (0 -95.92573980520687) / Math.Sqrt((2.503294354305362 / (32)) + (1.0465956193739419 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.1037145456614762 = (86.9117292893126 - 95.92573980520687) / 86.9117292893126 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.InterpolateIntoString(value: Red, Green)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 131.12517645729278 > 128.5598934314891.
IsChangePoint: Marked as a change because one of 3/16/2023 9:46:36 PM, 4/25/2023 8:51:41 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -12.69057375733904 (T) = (0 -134.263268316937) / Math.Sqrt((9.02812070462331 / (32)) + (5.095167386471736 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.08053617849980306 = (124.25615262910095 - 134.263268316937) / 124.25615262910095 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
ImmutableList - Duration of single invocation	443.02 μs	488.38 μs	1.10	0.09	False
Queue - Duration of single invocation	12.49 μs	16.89 μs	1.35	0.20	False
ICollection - Duration of single invocation	12.62 μs	15.26 μs	1.21	0.19	False
Array - Duration of single invocation	12.37 μs	16.11 μs	1.30	0.18	False
ImmutableArray - Duration of single invocation	11.77 μs	15.33 μs	1.30	0.35	False
List - Duration of single invocation	13.15 μs	14.83 μs	1.13	0.14	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.ImmutableList(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 488.38454715401787 > 465.4592736458333.
IsChangePoint: Marked as a change because one of 3/2/2023 7:59:08 PM, 4/17/2023 6:40:18 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -7.831007618101359 (T) = (0 -502939.5444651228) / Math.Sqrt((120858451.73991457 / (32)) + (730024733.9960274 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.12709056497486204 = (446228.15601010737 - 502939.5444651228) / 446228.15601010737 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.88842591169629 > 13.17273910187025.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -15.414827732277706 (T) = (0 -15142.5621222753) / Math.Sqrt((307061.6874822864 / (32)) + (332378.11606028967 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.22159956124785327 = (12395.683988955681 - 15142.5621222753) / 12395.683988955681 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.ICollection(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.263898057491287 > 13.484988967794143.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -20.247531359354173 (T) = (0 -14827.420802614019) / Math.Sqrt((276593.5258164804 / (30)) + (25717.11379553936 / (15))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (15) - 2, .025) and -0.16657677989055375 = (12710.197098218518 - 14827.420802614019) / 12710.197098218518 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.106638832989333 > 14.007165103365386.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -11.928092091313843 (T) = (0 -15480.935493218147) / Math.Sqrt((635088.1023777969 / (32)) + (321481.0376892327 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.18559814333442903 = (13057.489656384645 - 15480.935493218147) / 13057.489656384645 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.325836388305325 > 12.51131611780021.
IsChangePoint: Marked as a change because one of 3/24/2023 7:28:10 PM, 4/12/2023 4:44:56 PM, 5/4/2023 10:11:14 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -29.748401351694632 (T) = (0 -15447.687727483115) / Math.Sqrt((322071.8912551385 / (32)) + (27611.585796448326 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.26602046055310996 = (12201.767829829698 - 15447.687727483115) / 12201.767829829698 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.ContainsTrue<Int32>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.83192461419753 > 13.258284103255596.
IsChangePoint: Marked as a change because one of 4/17/2023 11:29:15 PM, 4/19/2023 3:30:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -29.606323191703485 (T) = (0 -15004.323732627408) / Math.Sqrt((142262.28204998648 / (30)) + (32940.484394806575 / (15))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (15) - 2, .025) and -0.19668348051425516 = (12538.255918916459 - 15004.323732627408) / 12538.255918916459 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	3.69 ms	4.41 ms	1.20	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.409487060267857 > 3.9952717351250002.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -13.588490547261962 (T) = (0 -4316833.72678228) / Math.Sqrt((5418971181.107147 / (32)) + (22942368603.851246 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.1490848362652976 = (3756758.0656730738 - 4316833.72678228) / 3756758.0656730738 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.BenchE

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	276.17 ms	356.12 ms	1.29	0.14	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BenchE*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.BenchE.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 356.11750953333336 > 282.80785699903845.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -65.84730873131082 (T) = (0 -356925425.13663006) / Math.Sqrt((44664693234602.34 / (31)) + (4734029789611.39 / (14))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (14) - 2, .025) and -0.3263658547260003 = (269100281.6944224 - 356925425.13663006) / 269100281.6944224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.TreeInsert

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	47.18 μs	55.78 μs	1.18	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.TreeInsert*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.TreeInsert.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 55.77618174673784 > 49.52638376835467.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -14.017488642523466 (T) = (0 -52964.0330299298) / Math.Sqrt((300527.36089596956 / (32)) + (2513883.456873353 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.12529125419237 = (47066.95518392035 - 52964.0330299298) / 47066.95518392035 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
BitArrayRightShift - Duration of single invocation	155.18 ns	201.32 ns	1.30	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 201.32030660959882 > 162.92866380746491.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -17.72549250733693 (T) = (0 -201.04986891588587) / Math.Sqrt((183.87258544941128 / (32)) + (0.07364191415045032 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.2681157833656458 = (158.54220218148296 - 201.04986891588587) / 158.54220218148296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.MDBenchI.MDGeneralArray

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test2 - Duration of single invocation	11.26 ms	12.27 ms	1.09	0.23	False
Test - Duration of single invocation	11.26 ms	12.26 ms	1.09	0.20	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.MDBenchI.MDGeneralArray*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDGeneralArray.Test2

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.273855234375 > 11.828288721757815.
IsChangePoint: Marked as a change because one of 4/3/2023 7:47:24 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -255.39227642026646 (T) = (0 -12259729.110750915) / Math.Sqrt((182440930.71159327 / (32)) + (142294797.30617592 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.08835741133902182 = (11264432.973050272 - 12259729.110750915) / 11264432.973050272 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Benchstone.MDBenchI.MDGeneralArray.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.262719195913462 > 11.815038631009617.
IsChangePoint: Marked as a change because one of 4/5/2023 11:58:06 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -287.5810023755632 (T) = (0 -12255881.493839668) / Math.Sqrt((117914289.17915422 / (32)) + (124102565.47536221 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.0883085765523448 = (11261403.022904683 - 12255881.493839668) / 11261403.022904683 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-25T22:46:36Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
FirstSpanSingleSegment - Duration of single invocation	4.34 ns	6.26 ns	1.44	0.03	True
FirstSpanTenSegments - Duration of single invocation	4.03 ns	6.52 ns	1.62	0.03	True
FirstSingleSegment - Duration of single invocation	3.34 ns	6.04 ns	1.81	0.02	True
FirstTenSegments - Duration of single invocation	2.81 ns	5.54 ns	1.97	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanSingleSegment

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.257967343520055 > 4.551177644840227.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -23.29822672172461 (T) = (0 -6.828581755049161) / Math.Sqrt((0.0010039601110026556 / (32)) + (0.17262246955339045 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.5785584466196626 = (4.325833971920355 - 6.828581755049161) / 4.325833971920355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanTenSegments

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.5155562819862824 > 4.355378091174689.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -23.20425327080238 (T) = (0 -6.161632148424472) / Math.Sqrt((0.003428236039969372 / (32)) + (0.11854679717466177 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.508410429454905 = (4.084851197065181 - 6.161632148424472) / 4.084851197065181 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSingleSegment

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.036784303556567 > 3.500131367834638.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -33.85963845727289 (T) = (0 -5.40548475673721) / Math.Sqrt((0.0063583130325200864 / (30)) + (0.055107188333568966 / (15))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (15) - 2, .025) and -0.6406029611412395 = (3.294815921200725 - 5.40548475673721) / 3.294815921200725 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstTenSegments

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.542014260700952 > 2.993749579877033.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -75.69863956622872 (T) = (0 -5.207696839945459) / Math.Sqrt((0.00336090805231913 / (32)) + (0.013280896630340303 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.8431794967816505 = (2.8253877872657247 - 5.207696839945459) / 2.8253877872657247 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Sort<IntStruct>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Array - Duration of single invocation	3.21 μs	4.75 μs	1.48	0.38	False
List - Duration of single invocation	3.00 μs	5.12 μs	1.71	0.43	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<IntStruct>.Array(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.7461702 > 3.2062747500000004.
IsChangePoint: Marked as a change because one of 3/20/2023 10:25:17 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -6.9826507015412185 (T) = (0 -4494.832469604909) / Math.Sqrt((15318.891233420429 / (32)) + (623600.2297515443 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.4674951995781481 = (3062.928226884157 - 4494.832469604909) / 3062.928226884157 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Sort<IntStruct>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.11715022857143 > 3.135964027.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 3/31/2023 2:00:12 PM, 4/19/2023 3:30:01 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -31.38367593478496 (T) = (0 -4987.054083904762) / Math.Sqrt((16525.220016810934 / (32)) + (46186.358009102216 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.606028050706562 = (3105.2098260119046 - 4987.054083904762) / 3105.2098260119046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.CreateAddAndClear<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	13.39 μs	14.60 μs	1.09	0.03	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndClear&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndClear<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.600854552571962 > 14.163912247433075.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -7.057575661148285 (T) = (0 -14504.049163231255) / Math.Sqrt((30293.881393344065 / (32)) + (210440.33733958917 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.06331811578526837 = (13640.366836522771 - 14504.049163231255) / 13640.366836522771 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int32

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ParseHex - Duration of single invocation	15.92 ns	18.88 ns	1.19	0.07	False
ParseHex - Duration of single invocation	15.78 ns	18.83 ns	1.19	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int32.ParseHex(value: "80000000")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.881141851097915 > 16.860607978899843.
IsChangePoint: Marked as a change because one of 4/21/2023 8:59:32 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -22.659405443427232 (T) = (0 -18.476744717675405) / Math.Sqrt((0.10153054991846215 / (32)) + (0.09621411234920361 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.13646555464840268 = (16.258077195653943 - 18.476744717675405) / 16.258077195653943 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.ParseHex(value: "7FFFFFFF")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.829295794947726 > 16.732068902308505.
IsChangePoint: Marked as a change because one of 4/22/2023 7:41:29 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -20.528069930407067 (T) = (0 -18.561875175403454) / Math.Sqrt((0.08542750921294981 / (32)) + (0.17201700624971558 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.15140463552605735 = (16.121070388884483 - 18.561875175403454) / 16.121070388884483 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Burgers

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test1 - Duration of single invocation	166.65 ms	239.90 ms	1.44	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Burgers*'

Payloads

Baseline
Compare

Histogram

Burgers.Test1

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 239.90123033333333 > 175.0093932375.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 4/9/2023 11:04:55 PM, 4/25/2023 3:46:39 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -356.85051875041233 (T) = (0 -239817870.32400492) / Math.Sqrt((1022530052749.824 / (32)) + (143780313790.74448 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.4352841956243646 = (167087376.18314084 - 239817870.32400492) / 167087376.18314084 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_SByte

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParse - Duration of single invocation	11.83 ns	13.23 ns	1.12	0.18	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_SByte.TryParse(value: "-128")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.231644219423384 > 12.47754329183238.
IsChangePoint: Marked as a change because one of 3/3/2023 4:59:02 PM, 3/13/2023 2:14:21 PM, 4/23/2023 12:49:47 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -18.376846367801566 (T) = (0 -13.487978951844491) / Math.Sqrt((0.11722076851424434 / (32)) + (0.01632906073551305 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.10365345008147293 = (12.221208524151121 - 13.487978951844491) / 12.221208524151121 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FirstTenSegments - Duration of single invocation	2.81 ns	5.98 ns	2.13	0.04	True
FirstSingleSegment - Duration of single invocation	3.22 ns	6.02 ns	1.87	0.03	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstTenSegments

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.981036573954714 > 2.965884758289749.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -14.111080040417942 (T) = (0 -4.661164818808123) / Math.Sqrt((0.003321071701732053 / (30)) + (0.2557153344220198 / (15))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (15) - 2, .025) and -0.6571534826075528 = (2.812753838270743 - 4.661164818808123) / 2.812753838270743 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSingleSegment

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.019236958316021 > 3.4213643840316474.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -11.056642829315152 (T) = (0 -5.4643110799510355) / Math.Sqrt((0.006937431683866711 / (32)) + (0.5961915305928626 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.6792553986246904 = (3.254008344666514 - 5.4643110799510355) / 3.254008344666514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ImmutableHashSet - Duration of single invocation	12.76 μs	14.71 μs	1.15	0.04	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<Int32>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.713516086889301 > 13.392746060623569.
IsChangePoint: Marked as a change because one of 3/17/2023 10:06:19 AM, 5/2/2023 4:25:00 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -15.540189308722645 (T) = (0 -14470.287027504093) / Math.Sqrt((17111.112017396914 / (32)) + (165904.26236220822 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.13573954198701496 = (12740.84989784527 - 14470.287027504093) / 12740.84989784527 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Enumerate - Duration of single invocation	139.90 ns	160.72 ns	1.15	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_PriorityQueue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_PriorityQueue<Int32, Int32>.Enumerate(Size: 100)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 160.71512922512008 > 146.41831522668988.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -33.057644789081856 (T) = (0 -161.08638233058952) / Math.Sqrt((12.252411676860424 / (32)) + (0.09488700180673496 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.14682665265536898 = (140.46271244011393 - 161.08638233058952) / 140.46271244011393 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Divide - Duration of single invocation	3.74 ms	4.37 ms	1.17	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 65536,32768 bits)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.372020636245111 > 3.928756260187499.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -78.21399102626782 (T) = (0 -4430884.234022863) / Math.Sqrt((1462620646.1418786 / (31)) + (336255026.1182713 / (14))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (14) - 2, .025) and -0.17501534921911008 = (3770916.0454521156 - 4430884.234022863) / 3770916.0454521156 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-25T22:46:37Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IsPrefix_DifferentFirstChar - Duration of single invocation	8.38 ns	9.56 ns	1.14	0.28	False
IndexOf_Word_NotFound - Duration of single invocation	369.23 ns	405.42 ns	1.10	0.09	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreCase, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.556868822711182 > 8.790993141829459.
IsChangePoint: Marked as a change because one of 4/25/2023 9:59:52 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -5.6555447117787425 (T) = (0 -9.03349814284436) / Math.Sqrt((0.25324989395564723 / (31)) + (0.21699989319032206 / (14))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (14) - 2, .025) and -0.10658507572927467 = (8.163401387725203 - 9.03349814284436) / 8.163401387725203 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, None, False))

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 405.4187013407365 > 388.32959534981467.
IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 4/25/2023 3:46:39 AM, 5/15/2023 9:18:21 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -6.570912231424944 (T) = (0 -409.8893414734199) / Math.Sqrt((410.95012693885724 / (13)) + (24.358233089109156 / (32))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (32) - 2, .025) and -0.10036583152316089 = (372.5027892823955 - 409.8893414734199) / 372.5027892823955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Tests.Perf_Dictionary

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ContainsValue - Duration of single invocation	2.64 ms	2.96 ms	1.12	0.01	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.Perf_Dictionary*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_Dictionary.ContainsValue(Items: 3000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.9603684672268904 > 2.7762231546874996.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -67.00787103380394 (T) = (0 -2964442.881964602) / Math.Sqrt((448982507.3104287 / (32)) + (119348039.36016098 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.1185566956565099 = (2650239.2712643803 - 2964442.881964602) / 2650239.2712643803 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOfValue - Duration of single invocation	17.68 ns	24.54 ns	1.39	0.32	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Int32>.IndexOfValue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.54267828642568 > 17.84434254086989.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/25/2023 6:46:56 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -27.073451361725375 (T) = (0 -23.75775388443502) / Math.Sqrt((1.3543615505518474 / (32)) + (0.3602802455237186 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.41546336483958446 = (16.78443573643992 - 23.75775388443502) / 16.78443573643992 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Random

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
NextSingle - Duration of single invocation	4.64 ns	7.56 ns	1.63	0.14	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Random.NextSingle

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.563236999782454 > 4.8780758981969266.
IsChangePoint: Marked as a change because one of 4/25/2023 1:57:48 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -69.10719096789336 (T) = (0 -7.382932564586813) / Math.Sqrt((0.005940462383931967 / (31)) + (0.02030308119861151 / (15))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (15) - 2, .025) and -0.582138852497114 = (4.666425170543166 - 7.382932564586813) / 4.666425170543166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SerializeToWriter - Duration of single invocation	5.64 μs	6.96 μs	1.23	0.05	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<ImmutableSortedDictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.959818016226536 > 5.890379978352993.
IsChangePoint: Marked as a change because one of 5/5/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -54.805811473483764 (T) = (0 -6892.943712104004) / Math.Sqrt((8997.102876876697 / (32)) + (3532.434820315598 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.22059386610077106 = (5647.204941414091 - 6892.943712104004) / 5647.204941414091 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Stack - Duration of single invocation	666.85 μs	723.33 μs	1.08	0.24	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.Stack(Size: 512)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 723.3311824269481 > 700.120620299479.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 4/25/2023 6:46:56 PM, 5/2/2023 3:18:41 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -11.727423514498032 (T) = (0 -721034.1290377778) / Math.Sqrt((1167206272.941396 / (31)) + (2253334.336399496 / (14))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (14) - 2, .025) and -0.11112988054263832 = (648919.7542645951 - 721034.1290377778) / 648919.7542645951 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in LinqBenchmarks

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count00ForX - Duration of single invocation	92.32 ms	121.42 ms	1.32	0.27	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Count00ForX

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 121.41808490000001 > 97.36639463250002.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -23.019016831818075 (T) = (0 -115118935.35880955) / Math.Sqrt((5167500078283.699 / (32)) + (10688309031811.809 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.22992348615506777 = (93598452.79375 - 115118935.35880955) / 93598452.79375 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Base64DecodeInPlace - Duration of single invocation	144.49 ms	553.60 ms	3.83	0.01	True
Base64EncodeInPlace - Duration of single invocation	124.02 ms	587.03 ms	4.73	0.02	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests.Base64DecodeInPlace(NumberOfBytes: 200000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 553.5966587222222 > 151.76443832307692.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -220.50363428936274 (T) = (0 -555513057.1434386) / Math.Sqrt((52341051027198.18 / (32)) + (28354436673616.625 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -2.9269459333043155 = (141461855.2377175 - 555513057.1434386) / 141461855.2377175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Base64EncodeDecodeInPlaceTests.Base64EncodeInPlace(NumberOfBytes: 200000000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 587.0331278125 > 130.20227150807693.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -33.567150708650054 (T) = (0 -576488214.0460254) / Math.Sqrt((129256097802.05917 / (32)) + (2723223214815685 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -3.641613184247466 = (124199969.0974874 - 576488214.0460254) / 124199969.0974874 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt64

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
TryParseHex - Duration of single invocation	8.76 ns	9.80 ns	1.12	0.12	False
ToString - Duration of single invocation	18.72 ns	20.03 ns	1.07	0.06	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParseHex(value: "3039")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.799966438883787 > 9.412155540010277.
IsChangePoint: Marked as a change because one of 4/17/2023 6:40:18 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -10.490307150144876 (T) = (0 -9.817315911838422) / Math.Sqrt((0.006346745153158221 / (32)) + (0.09118039967162564 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.09248888316241094 = (8.986192960994154 - 9.817315911838422) / 8.986192960994154 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.ToString(value: 18446744073709551615)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.026071811240758 > 19.54080268665248.
IsChangePoint: Marked as a change because one of 4/12/2023 4:44:56 PM, 5/19/2023 8:24:48 PM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -3.806734572940788 (T) = (0 -20.323946216594834) / Math.Sqrt((0.17591638554288413 / (34)) + (1.3826521536684533 / (13))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (13) - 2, .025) and -0.06672630147793913 = (19.052634390317554 - 20.323946216594834) / 19.052634390317554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	23.56 ms	28.33 ms	1.20	0.01	False
Count - Duration of single invocation	23.61 ms	28.32 ms	1.20	0.01	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.32500133333333 > 24.791136171428573.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/5/2023 9:43:39 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -19.919062471821835 (T) = (0 -28318017.9029932) / Math.Sqrt((1454270945706.6711 / (30)) + (305793468.635734 / (15))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (15) - 2, .025) and -0.18329589413828312 = (23931476.51679748 - 28318017.9029932) / 23931476.51679748 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: "\p{Sm}", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.32353164285714 > 24.785936247142857.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/5/2023 9:43:39 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -21.400967156418353 (T) = (0 -28325612.247889407) / Math.Sqrt((1365770730350.9285 / (32)) + (978376178.037646 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.18512433840669246 = (23900962.396883175 - 28325612.247889407) / 23900962.396883175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.MulMatrix

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	282.02 ms	307.04 ms	1.09	0.08	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.MulMatrix*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.MulMatrix.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 307.03509160000004 > 298.08831430000004.
IsChangePoint: Marked as a change because one of 3/3/2023 4:59:02 PM, 3/13/2023 6:11:08 PM, 3/25/2023 12:18:49 AM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -51.61551302217098 (T) = (0 -307175496.744591) / Math.Sqrt((6217688321186.443 / (32)) + (92184117871.76604 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.08135074308245727 = (284066477.698964 - 307175496.744591) / 284066477.698964 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler · 2023-05-25T22:46:39Z

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	14.01 ms	18.68 ms	1.33	0.02	False
Count - Duration of single invocation	21.67 ms	25.36 ms	1.17	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: NonBacktracking)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.68161915 > 14.574792549861113.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/5/2023 9:43:39 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -14.478376090479163 (T) = (0 -19277799.678217337) / Math.Sqrt((90446246969.36028 / (32)) + (1975185860378.7458 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.38015448361976834 = (13967856.42985192 - 19277799.678217337) / 13967856.42985192 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Mariomkas.Count(Pattern: "(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9])", Options: None)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.361554899999998 > 22.634955700000003.
IsChangePoint: Marked as a change because one of 5/1/2023 3:42:23 PM, 5/5/2023 9:43:39 PM, 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -11.792880639406405 (T) = (0 -25723600.430023547) / Math.Sqrt((123795503599.91458 / (32)) + (1693003194100.8884 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.19325152539747803 = (21557567.606255427 - 25723600.430023547) / 21557567.606255427 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.RentReturnArrayPoolTests<Object>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SingleParallel - Duration of single invocation	4.11 μs	4.66 μs	1.13	0.26	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.RentReturnArrayPoolTests&lt;Object&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.RentReturnArrayPoolTests<Object>.SingleParallel(RentalSize: 4096, ManipulateArray: True, Async: True, UseSharedPool: True)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.6594305055000005 > 4.349729606307693.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -13.451365315920684 (T) = (0 -4775.904454377821) / Math.Sqrt((19189.4337495073 / (32)) + (16009.541798080354 / (14))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (14) - 2, .025) and -0.13326527062947752 = (4214.286432447561 - 4775.904454377821) / 4214.286432447561 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in Benchstone.BenchI.BubbleSort2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	19.64 ms	26.39 ms	1.34	0.41	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Benchstone.BenchI.BubbleSort2*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.BubbleSort2.Test

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.389335367521365 > 23.340142073471664.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -21.56859468306719 (T) = (0 -27082543.303337503) / Math.Sqrt((1657420174755.7085 / (32)) + (217453829069.1141 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.257941427268 = (21529256.22471583 - 27082543.303337503) / 21529256.22471583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name	Value
Architecture	x64
OS	ubuntu 18.04
Queue	TigerUbuntu
Baseline	2f49fcff6df15a200ef01eea16b3ce7930f75c5c
Compare	558345d16cf76525d0c7fdbafbfd3a2457142b39
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_StringBuilder

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Append_Char - Duration of single invocation	117.36 μs	139.29 μs	1.19	0.00	True

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.Append_Char(length: 100000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 139.29125292968746 > 123.17791124650186.
IsChangePoint: Marked as a change because one of 5/19/2023 10:30:38 AM, 5/22/2023 9:02:11 PM falls between 5/14/2023 5:56:01 AM and 5/22/2023 9:02:11 PM.
IsRegressionStdDev: Marked as regression because -183.61459461813268 (T) = (0 -139220.69659352934) / Math.Sqrt((280376.2359117038 / (32)) + (72503.09514777736 / (15))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (15) - 2, .025) and -0.18172474885323506 = (117811.44190187383 - 139220.69659352934) / 117811.44190187383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

performanceautofiler bot assigned EgorBo May 25, 2023

performanceautofiler bot added arch-x64 branch-refs/heads/main kind-micro os-linux perf-regression runtime-coreclr untriaged labels May 25, 2023

AndyAyersMS mentioned this issue Jun 6, 2023

Dynamic PGO Microbenchmark Regressions dotnet/runtime#87194

Closed

[Perf] Linux/x64: 100 Regressions on 5/19/2023 3:32:16 PM #18139

[Perf] Linux/x64: 100 Regressions on 5/19/2023 3:32:16 PM #18139

Comments

performanceautofiler bot commented May 25, 2023 • edited Loading

Run Information

Regressions in System.Text.Json.Tests.Perf_Get

Repro

Payloads

Payloads

Histogram

System.Text.Json.Tests.Perf_Get.GetInt32

Description of detection logic

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt32

Description of detection logic

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetInt64

Description of detection logic

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetByte

Description of detection logic

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetUInt16

Description of detection logic

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetSByte

Description of detection logic

JIT Disasms

System.Text.Json.Tests.Perf_Get.GetInt16

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Tests.Perf_String

Repro

Payloads

Payloads

Histogram

System.Tests.Perf_String.ToUpperInvariant(s: "test")

Description of detection logic

JIT Disasms

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

Description of detection logic

JIT Disasms

System.Tests.Perf_String.ToUpper(s: "test")

Description of detection logic

JIT Disasms

System.Tests.Perf_String.ToUpper(s: "TeSt")

Description of detection logic

JIT Disasms

System.Tests.Perf_String.Remove_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 10, i2: 1)

Description of detection logic

JIT Disasms

System.Tests.Perf_String.ToLower(s: "TEST")

Description of detection logic

JIT Disasms

System.Tests.Perf_String.ToLower(s: "TeSt")

Description of detection logic

JIT Disasms

System.Tests.Perf_String.ToUpperInvariant(s: "TeSt")

Description of detection logic

JIT Disasms

System.Tests.Perf_String.ToUpperInvariant(s: "This is a much longer piece of text that might benefit more from vectorization.")

Description of detection logic

JIT Disasms

Docs

Run Information

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Repro

Payloads

Payloads

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,16)

Description of detection logic

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: Url,&lorem ipsum=dolor sit amet,512)

Description of detection logic

JIT Disasms

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf8(arguments: JavaScript,&Hello+<World>!,512)

Description of detection logic

performanceautofiler bot commented May 25, 2023 •

edited

Loading

performanceautofiler bot commented May 25, 2023 •

edited

Loading