Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Changes at 7/3/2022 3:44:27 AM #6588

Closed
performanceautofiler bot opened this issue Jul 5, 2022 · 13 comments
Closed

[Perf] Changes at 7/3/2022 3:44:27 AM #6588

performanceautofiler bot opened this issue Jul 5, 2022 · 13 comments
Assignees
@kunalspathak
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jul 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in MicroBenchmarks.Serializers.Binary_ToStream<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 3.75 μs 5.32 μs 1.42 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_ToStream<Location>.BinaryFormatter_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.315176657500022 > 3.948569878304789.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -94.47961773813896 (T) = (0 -5341.22386164912) / Math.Sqrt((1468.1952581805351 / (40)) + (1504.993780541855 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.42846303048520934 = (3739.14042412064 - 5341.22386164912) / 3739.14042412064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Formats.Cbor.Tests.Perf_CborReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadCoseKey - Duration of single invocation 632.91 ns 1.09 μs 1.72 0.00 False
SkipValue - Duration of single invocation 312.97 ns 466.54 ns 1.49 0.01 False
ReadCoseKey - Duration of single invocation 623.22 ns 1.07 μs 1.72 0.00 False
SkipValue - Duration of single invocation 661.33 ns 1.14 μs 1.72 0.01 False
ReadCoseKey - Duration of single invocation 627.87 ns 1.08 μs 1.72 0.00 False
SkipValue - Duration of single invocation 67.20 ns 113.63 ns 1.69 0.02 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Formats.Cbor.Tests.Perf_CborReader*'

Payloads

Baseline
Compare

Histogram

System.Formats.Cbor.Tests.Perf_CborReader.ReadCoseKey(publicKey: ECDSA_P521)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0889535326566284 > 665.8465183206365.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -132.65737882987992 (T) = (0 -1089.7650045921278) / Math.Sqrt((11.765806719294897 / (39)) + (57.92724479707406 / (5))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (5) - 2, .025) and -0.7232414501549762 = (632.3925207893078 - 1089.7650045921278) / 632.3925207893078 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Array, Strict))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 466.5359706500204 > 321.4682308014141.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -78.24703498105899 (T) = (0 -468.7833667147567) / Math.Sqrt((134.8017741914312 / (43)) + (14.717190381446807 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.6180985817877265 = (289.71248846706857 - 468.7833667147567) / 289.71248846706857 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Formats.Cbor.Tests.Perf_CborReader.ReadCoseKey(publicKey: ECDSA_P256)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0742104200857643 > 656.1564959513361.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -292.3271941940449 (T) = (0 -1072.9914800495617) / Math.Sqrt((8.77878480050338 / (40)) + (12.955831325923697 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.7247141453579946 = (622.1271408583731 - 1072.9914800495617) / 622.1271408583731 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Map, Lax))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1406195758298983 > 694.0804510452089.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -132.54684029027663 (T) = (0 -1130.6685319504734) / Math.Sqrt((52.57923981153919 / (45)) + (66.9646442401212 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -0.6995958176719755 = (665.2573042331962 - 1130.6685319504734) / 665.2573042331962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Formats.Cbor.Tests.Perf_CborReader.ReadCoseKey(publicKey: ECDSA_P384)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0790472179309614 > 658.7048030096287.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -144.89878168817228 (T) = (0 -1082.704421003379) / Math.Sqrt((9.721020861654216 / (40)) + (48.19687920822504 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.7262604331417445 = (627.1964532216543 - 1082.704421003379) / 627.1964532216543 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Text String, Strict))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 113.63470406274077 > 70.74300668029872.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -113.7939491327966 (T) = (0 -112.60415466580845) / Math.Sqrt((0.4298536103626667 / (44)) + (1.0373865357526186 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.6712647679930397 = (67.37661011129352 - 112.60415466580845) / 67.37661011129352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryFormat - Duration of single invocation 2.37 ns 16.92 ns 7.15 0.02 False
TryParse - Duration of single invocation 16.27 ns 27.01 ns 1.66 0.10 False
TryParseHex - Duration of single invocation 10.23 ns 78.85 ns 7.71 0.01 False
TryFormat - Duration of single invocation 14.76 ns 25.85 ns 1.75 0.02 False
ParseSpan - Duration of single invocation 19.49 ns 34.34 ns 1.76 0.08 False
TryParse - Duration of single invocation 10.34 ns 22.89 ns 2.21 0.22 False
ParseSpan - Duration of single invocation 10.26 ns 24.20 ns 2.36 0.24 False
Parse - Duration of single invocation 10.17 ns 24.44 ns 2.40 0.19 False
TryFormat - Duration of single invocation 8.59 ns 20.95 ns 2.44 0.02 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.TryFormat(value: 0)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.91966495575429 > 2.4776547487598863.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -390.6845353118678 (T) = (0 -16.861869582184724) / Math.Sqrt((5.086078003709308E-06 / (40)) + (0.008267305653136816 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -6.147756066738653 = (2.359043792869443 - 16.861869582184724) / 2.359043792869443 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParse(value: "12345")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.010873390959024 > 17.29571142811355.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -49.42967381524521 (T) = (0 -26.072299540229515) / Math.Sqrt((0.12654280588133346 / (44)) + (0.28440900872015357 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.6540993910206176 = (15.762232718157463 - 26.072299540229515) / 15.762232718157463 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParseHex(value: "3039")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 78.8535215333743 > 10.729882202014581.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -143.11602249285013 (T) = (0 -79.47503623962695) / Math.Sqrt((0.04751503916877666 / (38)) + (1.167333953315415 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -6.838756717349125 = (10.138729789091279 - 79.47503623962695) / 10.138729789091279 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.847544485776115 > 15.524049989489571.
IsChangePoint: Marked as a change because one of 5/5/2022 12:13:02 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -77.16966225120265 (T) = (0 -25.742489729917178) / Math.Sqrt((0.28994006014829105 / (48)) + (0.12089953596548741 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.844008031707353 = (13.960074624014736 - 25.742489729917178) / 13.960074624014736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.ParseSpan(value: "4294967295")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.34486911494328 > 20.68407522808782.
IsChangePoint: Marked as a change because one of 6/2/2022 5:58:32 PM, 6/6/2022 7:24:37 AM, 6/27/2022 8:54:23 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -31.036527637527342 (T) = (0 -34.65600785426642) / Math.Sqrt((3.637291612581206 / (43)) + (0.5028555419762483 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.5484644325968664 = (22.38088723558619 - 34.65600785426642) / 22.38088723558619 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParse(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.891623737833722 > 10.836116959349829.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -66.79254435565488 (T) = (0 -22.564025451643634) / Math.Sqrt((0.31640947388295937 / (46)) + (0.19340964427110405 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -1.2216672530626114 = (10.156347860166138 - 22.564025451643634) / 10.156347860166138 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.ParseSpan(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.19515921370389 > 10.668977919001438.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -111.67145405317714 (T) = (0 -24.39245559487208) / Math.Sqrt((0.1706682760915452 / (44)) + (0.08652266570762958 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -1.400404255307521 = (10.16181151193098 - 24.39245559487208) / 10.16181151193098 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.Parse(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.43579370204875 > 10.680173444642229.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -57.80076226331446 (T) = (0 -23.993993093375718) / Math.Sqrt((0.1796345195499972 / (45)) + (0.3197362745316433 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -1.3615563519986056 = (10.160245836636248 - 23.993993093375718) / 10.160245836636248 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryFormat(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.94880127893083 > 9.009087211079217.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -80.60169656550127 (T) = (0 -20.625262976970596) / Math.Sqrt((0.013012744476194951 / (40)) + (0.11315158760370954 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -1.4515345124969241 = (8.413205227922107 - 20.625262976970596) / 8.413205227922107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Collections.ContainsTrue<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 247.63 μs 336.41 μs 1.36 0.29 False
ImmutableSortedSet - Duration of single invocation 255.28 μs 340.12 μs 1.33 0.31 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<String>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 336.40730496453904 > 257.54052164713545.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -39.954045031818815 (T) = (0 -336429.1420656028) / Math.Sqrt((70418269.09835629 / (38)) + (14256887.769381447 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.3469586081930806 = (249769.4732556898 - 336429.1420656028) / 249769.4732556898 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrue&lt;String&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 340.1188768115942 > 265.13153186603944.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -23.980130684752044 (T) = (0 -341099.542012666) / Math.Sqrt((92840617.62369713 / (38)) + (70405047.00872786 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -0.33079061196403614 = (256313.45678735824 - 341099.542012666) / 256313.45678735824 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfAnyThreeValues - Duration of single invocation 19.76 ns 28.69 ns 1.45 0.12 False
SequenceCompareTo - Duration of single invocation 28.08 ns 35.95 ns 1.28 0.27 False
Reverse - Duration of single invocation 23.61 ns 35.45 ns 1.50 0.32 False
LastIndexOfValue - Duration of single invocation 18.44 ns 27.42 ns 1.49 0.06 False
BinarySearch - Duration of single invocation 16.43 ns 42.15 ns 2.57 0.04 False
StartsWith - Duration of single invocation 13.04 ns 35.55 ns 2.73 0.05 False
EndsWith - Duration of single invocation 15.26 ns 34.66 ns 2.27 0.09 False
ToArray - Duration of single invocation 67.58 ns 75.03 ns 1.11 0.09 False
IndexOfAnyTwoValues - Duration of single invocation 14.67 ns 23.94 ns 1.63 0.13 False
SequenceEqual - Duration of single invocation 22.46 ns 35.98 ns 1.60 0.03 False
IndexOfValue - Duration of single invocation 14.12 ns 24.48 ns 1.73 0.01 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.688199755565066 > 20.881419863218152.
IsChangePoint: Marked as a change because one of 6/2/2022 5:58:32 PM, 6/22/2022 6:53:40 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -61.840519359574714 (T) = (0 -28.92039876521295) / Math.Sqrt((0.2102934346962426 / (39)) + (0.07606614489248643 / (5))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (5) - 2, .025) and -0.44288469037372724 = (20.04345805188505 - 28.92039876521295) / 20.04345805188505 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.SequenceCompareTo(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.94957439079211 > 27.283663864899022.
IsChangePoint: Marked as a change because one of 5/14/2022 12:06:14 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -23.63027623188907 (T) = (0 -37.06179492517028) / Math.Sqrt((1.9438269212479462 / (44)) + (1.154602137471969 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.4115699078717053 = (26.25572755447175 - 37.06179492517028) / 26.25572755447175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.Reverse(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.45366397056056 > 24.902559226898344.
IsChangePoint: Marked as a change because one of 5/9/2022 11:56:12 AM, 5/15/2022 3:27:46 PM, 5/23/2022 3:43:39 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -57.798294138277036 (T) = (0 -35.12827262178842) / Math.Sqrt((1.0094447962348283 / (38)) + (0.051563324883213225 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -0.4348795253488782 = (24.48168783595075 - 35.12827262178842) / 24.48168783595075 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.424838786660242 > 19.08521061539475.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/30/2022 12:57:05 PM, 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -42.38605528710897 (T) = (0 -27.376771727291928) / Math.Sqrt((0.5422337090065182 / (44)) + (0.2621858953392226 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.5277244699303845 = (17.919966765040712 - 27.376771727291928) / 17.919966765040712 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.BinarySearch(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 42.147145556289466 > 17.33429564624381.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -26.78557048888195 (T) = (0 -41.10283010583119) / Math.Sqrt((0.33277990073474195 / (43)) + (5.933545774743582 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -1.5170747680557564 = (16.32960237314679 - 41.10283010583119) / 16.32960237314679 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.StartsWith(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.548161584243466 > 13.69105706274028.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -256.6054961316823 (T) = (0 -35.3957907321004) / Math.Sqrt((0.13984740365975956 / (41)) + (0.03172485308650502 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -1.8257497299470427 = (12.526159113448363 - 35.3957907321004) / 12.526159113448363 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.EndsWith(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.65650488850461 > 15.953771230657068.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -185.98468751018183 (T) = (0 -34.280666749996044) / Math.Sqrt((0.06813649858180242 / (39)) + (0.07274194036029447 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -1.298099881061807 = (14.916961195854164 - 34.280666749996044) / 14.916961195854164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.ToArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 75.03455441467871 > 70.56792828674426.
IsChangePoint: Marked as a change because one of 5/8/2022 4:51:58 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -27.7089656954111 (T) = (0 -75.08183837034562) / Math.Sqrt((1.357968449079958 / (42)) + (0.1875182826406961 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.10806665897596596 = (67.75931552685961 - 75.08183837034562) / 67.75931552685961 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfAnyTwoValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.937264113312427 > 15.40971711384181.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -35.27357695463411 (T) = (0 -24.704095141077833) / Math.Sqrt((0.055350590857485255 / (40)) + (0.3844932580622353 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.66526842051968 = (14.834902792048641 - 24.704095141077833) / 14.834902792048641 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.SequenceEqual(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.98198477683873 > 24.450833187772165.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -24.8303363237684 (T) = (0 -34.25218027743353) / Math.Sqrt((0.12552476449011143 / (42)) + (1.2670586669103716 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.5048493227443056 = (22.761202573404383 - 34.25218027743353) / 22.761202573404383 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.IndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.476405071106043 > 14.833098213627494.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -44.70459340967441 (T) = (0 -23.759796295045543) / Math.Sqrt((0.11684973097288513 / (40)) + (0.2532712291078975 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.6658923449721285 = (14.262504036803808 - 23.759796295045543) / 14.262504036803808 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Xml.Linq.Perf_XName

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CreateElementWithNamespaceImplicitOperator - Duration of single invocation 58.12 ns 99.28 ns 1.71 0.02 False
CreateElement - Duration of single invocation 20.45 ns 37.44 ns 1.83 0.01 False
CreateElementWithNamespace - Duration of single invocation 51.52 ns 89.42 ns 1.74 0.01 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Xml.Linq.Perf_XName*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XName.CreateElementWithNamespaceImplicitOperator

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.27567712829044 > 60.998940596927966.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -383.01054324611323 (T) = (0 -99.36634214077976) / Math.Sqrt((0.07361181889527699 / (40)) + (0.05849939511359572 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.7093143045760825 = (58.132282561938226 - 99.36634214077976) / 58.132282561938226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Linq.Perf_XName.CreateElement

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.44108459214234 > 21.492448760449673.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -123.6517680827732 (T) = (0 -37.39192858859461) / Math.Sqrt((0.009984595662408353 / (41)) + (0.12937162015185238 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.8265429734274903 = (20.47142012674852 - 37.39192858859461) / 20.47142012674852 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Linq.Perf_XName.CreateElementWithNamespace

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 89.42279202744089 > 54.194765484552036.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -337.8469158672187 (T) = (0 -89.56011935222922) / Math.Sqrt((0.05252336712161745 / (39)) + (0.09023396388077683 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.7357354874949835 = (51.597792404119446 - 89.56011935222922) / 51.597792404119446 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToString - Duration of single invocation 20.00 μs 34.18 μs 1.71 0.01 False
SerializeToUtf8Bytes - Duration of single invocation 18.35 μs 32.01 μs 1.74 0.01 False
SerializeObjectProperty - Duration of single invocation 20.57 μs 34.53 μs 1.68 0.01 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<IndexViewModel>.SerializeToString

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.18288706864904 > 21.097055932868155.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -65.8014915036211 (T) = (0 -34452.032363045946) / Math.Sqrt((26096.558198934952 / (48)) + (335515.9172223032 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.7256591756974364 = (19964.56359878939 - 34452.032363045946) / 19964.56359878939 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.01247054491899 > 19.254513099408427.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -44.610765728849536 (T) = (0 -32818.07578749502) / Math.Sqrt((24370.314201037698 / (44)) + (749444.2319447572 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.8048294000205486 = (18183.478054558163 - 32818.07578749502) / 18183.478054558163 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;IndexViewModel&gt;.SerializeObjectProperty

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.527219179826794 > 21.36413070864745.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -117.37486038845883 (T) = (0 -34884.17630678242) / Math.Sqrt((54673.88898992577 / (43)) + (100117.23904581244 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.7238410290409056 = (20236.307013872938 - 34884.17630678242) / 20236.307013872938 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Collections.Concurrent.IsEmpty<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 2.59 ns 24.95 ns 9.64 0.05 False
Dictionary - Duration of single invocation 237.11 ns 290.61 ns 1.23 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.IsEmpty<Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.95106758864744 > 2.6935506571504333.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -110.19051742837695 (T) = (0 -25.34406051566115) / Math.Sqrt((0.00013683028088666514 / (44)) + (0.29866096728889135 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -8.81341709921623 = (2.582592817509541 - 25.34406051566115) / 2.582592817509541 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Concurrent.IsEmpty&lt;Int32&gt;.Dictionary(Size: 0)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 290.6119722409902 > 252.80389630010782.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -65.8122899140353 (T) = (0 -290.3395295031798) / Math.Sqrt((20.646494909582923 / (42)) + (1.0976733963026288 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.228752267269419 = (236.28809259362228 - 290.3395295031798) / 236.28809259362228 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Numerics.Tests.Constructor

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SpanCastBenchmark_UInt32 - Duration of single invocation 0.68 ns 18.30 ns 26.83 0.02 False
SpanCastBenchmark_Double - Duration of single invocation 0.68 ns 17.85 ns 26.14 0.01 False
ConstructorBenchmark_UInt64 - Duration of single invocation 0.66 ns 23.86 ns 36.00 0.03 False
ConstructorBenchmark_UInt32 - Duration of single invocation 0.37 ns 22.51 ns 60.79 0.03 False
ConstructorBenchmark_Int64 - Duration of single invocation 0.37 ns 23.91 ns 64.58 0.03 False
ConstructorBenchmark_Double - Duration of single invocation 0.37 ns 22.04 ns 59.53 0.03 False
SpanCastBenchmark_SByte - Duration of single invocation 0.62 ns 18.11 ns 29.04 0.03 False
ConstructorBenchmark_Byte - Duration of single invocation 0.37 ns 22.04 ns 59.49 0.02 False
ConstructorBenchmark_Int32 - Duration of single invocation 0.37 ns 24.05 ns 64.94 0.01 False
SpanCastBenchmark_Byte - Duration of single invocation 0.51 ns 18.62 ns 36.75 0.03 False
SpanCastBenchmark_Int64 - Duration of single invocation 0.68 ns 18.61 ns 27.28 0.02 False
SpanCastBenchmark_Single - Duration of single invocation 0.68 ns 18.30 ns 26.78 0.02 False
ConstructorBenchmark_Single - Duration of single invocation 0.37 ns 22.31 ns 60.27 0.04 False
ConstructorBenchmark_SByte - Duration of single invocation 0.37 ns 22.04 ns 59.51 0.04 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Constructor*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Constructor.SpanCastBenchmark_UInt32

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.299575669055766 > 0.7161789647409107.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -130.11944953243696 (T) = (0 -18.400038358943757) / Math.Sqrt((8.987683193854009E-09 / (38)) + (0.09270647846326723 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -25.974078544727114 = (0.6821377912292241 - 18.400038358943757) / 0.6821377912292241 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_Double

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.846400344095663 > 0.7171099374060972.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -148.2089477594267 (T) = (0 -18.245252476815164) / Math.Sqrt((4.606606233972117E-07 / (40)) + (0.11233177982201993 / (8))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (8) - 2, .025) and -25.714526922657406 = (0.6829711987652983 - 18.245252476815164) / 0.6829711987652983 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_UInt64

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.859347060502518 > 0.6957744959674189.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -431.44907700187196 (T) = (0 -24.05631186147837) / Math.Sqrt((1.337946294975776E-08 / (42)) + (0.014699654918329585 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -35.302979686865136 = (0.6626539217711167 - 24.05631186147837) / 0.6626539217711167 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_UInt32

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.508916497928826 > 0.38879122729663396.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -206.79937281241592 (T) = (0 -22.47645828914357) / Math.Sqrt((3.594853648815965E-09 / (41)) + (0.07998839371720727 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -59.70140208236112 = (0.3702790630543751 - 22.47645828914357) / 0.3702790630543751 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_Int64

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 23.911488404970658 > 0.3888187337377808.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -212.66399345679218 (T) = (0 -23.869689999981603) / Math.Sqrt((3.744461625927745E-09 / (42)) + (0.07326172406074846 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -63.46342172563216 = (0.3702827023606545 - 23.869689999981603) / 0.3702827023606545 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_Double

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.040321177867 > 0.38879158545711123.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -157.3699084328954 (T) = (0 -22.22770787424717) / Math.Sqrt((5.6117008646240104E-09 / (41)) + (0.13503686140867438 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -59.02772342832912 = (0.37029070244161816 - 22.22770787424717) / 0.37029070244161816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_SByte

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.10975605135779 > 0.654811863811089.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -492.35565572892415 (T) = (0 -18.14953264774622) / Math.Sqrt((3.088117801537681E-08 / (44)) + (0.008869468551557728 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -28.100100006319416 = (0.6236931365804531 - 18.14953264774622) / 0.6236931365804531 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_Byte

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.038845263203545 > 0.3888197519603179.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -210.3055085372432 (T) = (0 -22.158715194180015) / Math.Sqrt((6.385254880745471E-08 / (41)) + (0.07513571137964052 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -58.833566681977196 = (0.3703391995993875 - 22.158715194180015) / 0.3703391995993875 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_Int32

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.051651165885755 > 0.38887889237816337.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -131.14242766136292 (T) = (0 -23.65364925048487) / Math.Sqrt((8.699959811390598E-08 / (38)) + (0.22064683565317392 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -62.85492951976786 = (0.37042792824886456 - 23.65364925048487) / 0.37042792824886456 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_Byte

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.619786617815215 > 0.5320100797707281.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -155.98387804686683 (T) = (0 -18.4234362028521) / Math.Sqrt((5.453547415179928E-09 / (46)) + (0.09235425774286747 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -35.360414322987054 = (0.5066893913583598 - 18.4234362028521) / 0.5066893913583598 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_Int64

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.61124153227941 > 0.7161968584479997.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -182.4753809011709 (T) = (0 -18.52170943854598) / Math.Sqrt((4.44388496208191E-09 / (43)) + (0.07646310518847664 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -26.15380466725356 = (0.6821036560258698 - 18.52170943854598) / 0.6821036560258698 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.SpanCastBenchmark_Single

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.299662239813028 > 0.7176424982141907.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -178.43359363128027 (T) = (0 -18.443435379540478) / Math.Sqrt((2.986391135627896E-07 / (41)) + (0.069351245947204 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -26.005648262391656 = (0.6829473301414873 - 18.443435379540478) / 0.6829473301414873 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_Single

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.307693980345572 > 0.3887300597796308.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -148.76377369677783 (T) = (0 -22.238583603414373) / Math.Sqrt((4.461699402072339E-07 / (44)) + (0.15126181856810725 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -59.04113301214944 = (0.3703891397071796 - 22.238583603414373) / 0.3703891397071796 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Constructor.ConstructorBenchmark_SByte

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.041869219442237 > 0.38883877348076085.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -103.28499781679461 (T) = (0 -22.228110830800947) / Math.Sqrt((9.995664244626928E-09 / (38)) + (0.22392840314266974 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -59.028710257107186 = (0.370291327859559 - 22.228110830800947) / 0.370291327859559 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDeepUtf8 - Duration of single invocation 24.70 ms 35.81 ms 1.45 0.05 False
WriteDeepUtf8 - Duration of single invocation 24.62 ms 35.45 ms 1.44 0.04 False
WriteDeepUtf16 - Duration of single invocation 24.89 ms 35.63 ms 1.43 0.03 False
WriteDeepUtf16 - Duration of single invocation 24.67 ms 35.86 ms 1.45 0.04 False
WriteDeepUtf8 - Duration of single invocation 4.62 ms 13.53 ms 2.93 0.01 False
WriteDeepUtf8 - Duration of single invocation 4.40 ms 13.38 ms 3.04 0.00 False
WriteDeepUtf16 - Duration of single invocation 4.33 ms 13.35 ms 3.09 0.00 False
WriteDeepUtf16 - Duration of single invocation 4.67 ms 13.63 ms 2.92 0.00 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.811912222222226 > 25.890691249999996.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -153.52198837128333 (T) = (0 -35747930.79365079) / Math.Sqrt((38596331266.54888 / (44)) + (30003661687.84557 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.44633000468320755 = (24716303.110562053 - 35747930.79365079) / 24716303.110562053 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.45276666666666 > 25.918068249999997.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -65.55319346248578 (T) = (0 -35555067.92989418) / Math.Sqrt((1061458752110.4937 / (42)) + (8658237474.321285 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.43134007302137656 = (24840405.575204752 - 35555067.92989418) / 24840405.575204752 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.63451833333333 > 25.98356678076923.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -102.46315442679139 (T) = (0 -35799456.55555556) / Math.Sqrt((43892345266.28538 / (40)) + (52161638144.62067 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.44368026379587494 = (24797358.15008504 - 35799456.55555556) / 24797358.15008504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.86274 > 25.877211041666666.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -55.4197424645129 (T) = (0 -35652502.44047619) / Math.Sqrt((1086353824395.0903 / (40)) + (65402039106.38019 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.43523993908467973 = (24840795.932151575 - 35652502.44047619) / 24840795.932151575 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.52895740740741 > 4.850421179245283.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -277.03398454056565 (T) = (0 -13631982.339743586) / Math.Sqrt((5380391991.362862 / (39)) + (7163005136.123915 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -1.8841105685869963 = (4726581.043119391 - 13631982.339743586) / 4726581.043119391 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.379963333333336 > 4.622676640350877.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -257.3257595923636 (T) = (0 -13418753.572820941) / Math.Sqrt((388616420.7158112 / (41)) + (8555641081.84795 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -2.058259992925719 = (4387708.567571372 - 13418753.572820941) / 4387708.567571372 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.346645344129556 > 4.560258868421053.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -330.5488429797944 (T) = (0 -13390118.170058846) / Math.Sqrt((1270252717.0476055 / (48)) + (5015482039.909204 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -2.0569333491906368 = (4380245.376826438 - 13390118.170058846) / 4380245.376826438 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.630720370370367 > 4.893628481132076.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -186.37841011912263 (T) = (0 -13774561.825396825) / Math.Sqrt((5174943227.29271 / (44)) + (15681463826.411104 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -1.9155534688853324 = (4724510.1049932325 - 13774561.825396825) / 4724510.1049932325 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromUtf8Bytes - Duration of single invocation 88.43 μs 126.88 μs 1.43 0.12 False
DeserializeFromString - Duration of single invocation 90.53 μs 117.12 μs 1.29 0.09 False
DeserializeFromStream - Duration of single invocation 91.89 μs 123.88 μs 1.35 0.05 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<ImmutableSortedDictionary<String, String>>.DeserializeFromUtf8Bytes

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 126.87884405670665 > 93.44674888363753.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -15.093564133453233 (T) = (0 -122349.53026503544) / Math.Sqrt((8577828.952632686 / (44)) + (31325917.100208506 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.3635028109747599 = (89731.77706730836 - 122349.53026503544) / 89731.77706730836 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.DeserializeFromString

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 117.11652116402118 > 94.710010941281.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -44.7210062027822 (T) = (0 -118787.50621599889) / Math.Sqrt((5561388.486860711 / (42)) + (1516060.6713762507 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.3048470306043168 = (91035.57997981155 - 118787.50621599889) / 91035.57997981155 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;ImmutableSortedDictionary&lt;String, String&gt;&gt;.DeserializeFromStream

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 123.88406214848145 > 97.18642968222841.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -42.48526123188741 (T) = (0 -122290.10820691437) / Math.Sqrt((3894012.8238736885 / (41)) + (2619356.3480425016 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.3122745980182865 = (93189.41964706861 - 122290.10820691437) / 93189.41964706861 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Collections.CreateAddAndRemove<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 619.76 μs 787.62 μs 1.27 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CreateAddAndRemove&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<String>.SortedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 787.6249776785714 > 646.5344387019231.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -30.056386135391243 (T) = (0 -796635.3414540817) / Math.Sqrt((131707108.82811412 / (38)) + (155715729.57333577 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.2852790795698483 = (619815.0690515372 - 796635.3414540817) / 619815.0690515372 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Document.Tests.Perf_DocumentParse

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 7.79 ms 34.68 ms 4.45 0.01 False
Parse - Duration of single invocation 1.73 μs 5.87 μs 3.39 0.01 False
Parse - Duration of single invocation 372.74 ns 880.01 ns 2.36 0.05 False
Parse - Duration of single invocation 1.03 ms 3.30 ms 3.19 0.01 False
Parse - Duration of single invocation 1.18 μs 3.81 μs 3.22 0.00 False
Parse - Duration of single invocation 1.31 ms 3.48 ms 2.66 0.01 False
Parse - Duration of single invocation 2.74 μs 11.27 μs 4.11 0.01 False
Parse - Duration of single invocation 253.06 ns 598.53 ns 2.37 0.14 False
Parse - Duration of single invocation 341.45 ns 865.29 ns 2.53 0.11 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Document.Tests.Perf_DocumentParse*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: Json400KB)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.679205128205126 > 8.195383920258621.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -174.0640956758241 (T) = (0 -34834420.700854704) / Math.Sqrt((81839749060.4111 / (38)) + (111295043073.8263 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -3.5608686881779166 = (7637672.356397347 - 34834420.700854704) / 7637672.356397347 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: BasicJson)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.873740101886384 > 1.8323517044000273.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -377.8830708498929 (T) = (0 -5899.238011479477) / Math.Sqrt((530.5963157073443 / (41)) + (768.5553708637493 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -2.4441486307619797 = (1712.8291034798738 - 5899.238011479477) / 1712.8291034798738 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: True, TestCase: HelloWorld)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 880.0146206015332 > 383.7774515945803.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -165.2393728366054 (T) = (0 -878.5346506192565) / Math.Sqrt((102.81714065496443 / (44)) + (51.39194813096807 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -1.410435957263698 = (364.4712683495478 - 878.5346506192565) / 364.4712683495478 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.298995789473684 > 1.0844575165289256.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -27.565457616823004 (T) = (0 -3385905.086619702) / Math.Sqrt((42568795.51106625 / (41)) + (50865481786.85382 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -2.268392757733699 = (1035954.1638953717 - 3385905.086619702) / 1035954.1638953717 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: False, TestCase: Json400B)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.8082972142736913 > 1.2430666721085228.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -248.70257597351062 (T) = (0 -3770.5598371687543) / Math.Sqrt((137.83185141656696 / (43)) + (836.340704464138 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -2.171244664769936 = (1188.9842114853968 - 3770.5598371687543) / 1188.9842114853968 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: Json400KB)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4774224858757057 > 1.3769805185739217.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -155.3687197565704 (T) = (0 -3490765.4643662213) / Math.Sqrt((95889143.63787802 / (46)) + (1354358246.2651188 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -1.6484872006238698 = (1318022.4029566564 - 3490765.4643662213) / 1318022.4029566564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: Json400B)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.267857982425044 > 2.8832520834858073.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -235.6677479217037 (T) = (0 -11284.285503861869) / Math.Sqrt((769.4680399954575 / (38)) + (6489.577728704508 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -3.136460031856253 = (2728.005448368372 - 11284.285503861869) / 2728.005448368372 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: True, TestRandomAccess: False, TestCase: HelloWorld)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 598.5296444857834 > 265.5058965390008.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -46.22226634086381 (T) = (0 -606.2518619367687) / Math.Sqrt((37.1224238185706 / (45)) + (338.0608591442107 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -1.3611043622497612 = (256.76622839284664 - 606.2518619367687) / 256.76622839284664 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Document.Tests.Perf_DocumentParse.Parse(IsDataIndented: False, TestRandomAccess: True, TestCase: HelloWorld)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 865.2864859594152 > 361.0994340143165.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -151.3914207236 (T) = (0 -866.3494368562351) / Math.Sqrt((52.628371686665034 / (40)) + (83.1192638103327 / (8))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (8) - 2, .025) and -1.4867458004504923 = (348.38681006289005 - 866.3494368562351) / 348.38681006289005 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 1.37 μs 1.61 μs 1.18 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;XmlElement&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<XmlElement>.XmlSerializer_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6138510301697886 > 1.4438663405988013.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -14.027873759382443 (T) = (0 -1674.4005984888345) / Math.Sqrt((504.24617294077194 / (40)) + (2741.352248100017 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.22179252543933536 = (1370.4459338435952 - 1674.4005984888345) / 1370.4459338435952 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Baseline - Duration of single invocation 1.38 μs 2.22 μs 1.61 0.00 False
CaseInsensitiveNotMatching - Duration of single invocation 1.37 μs 2.14 μs 1.57 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive&lt;Location&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive<Location>.Baseline

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2200789268785077 > 1.440090644908384.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -47.338681716893355 (T) = (0 -2157.620266718382) / Math.Sqrt((130.3318648938388 / (38)) + (1360.0883332430672 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.5726569283005307 = (1371.9586439300424 - 2157.620266718382) / 1371.9586439300424 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadMissingAndCaseInsensitive&lt;Location&gt;.CaseInsensitiveNotMatching

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1403779072629883 > 1.4376528323749083.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -104.67562637804144 (T) = (0 -2153.8453380414817) / Math.Sqrt((249.51014238285993 / (41)) + (347.8280908444483 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.5697465543634921 = (1372.097509661254 - 2153.8453380414817) / 1372.097509661254 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArraySetAll - Duration of single invocation 3.74 ns 9.96 ns 2.66 0.03 False
BitArraySetLengthShrink - Duration of single invocation 14.15 ns 33.40 ns 2.36 0.23 False
BitArrayRightShift - Duration of single invocation 5.25 ns 12.75 ns 2.43 0.09 False
BitArrayCopyToByteArray - Duration of single invocation 9.57 ns 32.47 ns 3.39 0.38 False
BitArraySetLengthShrink - Duration of single invocation 187.15 ns 1.86 μs 9.92 0.00 False
BitArrayLengthValueCtor - Duration of single invocation 17.17 ns 25.10 ns 1.46 0.18 False
BitArrayLengthValueCtor - Duration of single invocation 16.54 ns 21.06 ns 1.27 0.21 False
BitArrayByteArrayCtor - Duration of single invocation 184.24 ns 1.86 μs 10.09 0.00 False
BitArrayLeftShift - Duration of single invocation 4.16 ns 11.14 ns 2.68 0.08 False
BitArrayByteArrayCtor - Duration of single invocation 11.43 ns 29.94 ns 2.62 0.17 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArraySetAll(Size: 4)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.95944551835046 > 3.9288938234230413.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -247.60192866336504 (T) = (0 -9.91303592732956) / Math.Sqrt((1.7238919739855913E-08 / (38)) + (0.003106053284043474 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -1.6492842127656393 = (3.741778960355918 - 9.91303592732956) / 3.741778960355918 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.402509856406944 > 14.856852047606342.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -157.55617257122745 (T) = (0 -33.62356843903909) / Math.Sqrt((0.21914778941892002 / (46)) + (0.07106564908640883 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -1.338078004923471 = (14.3808582811332 - 33.62356843903909) / 14.3808582811332 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.753684888418464 > 5.512927057068411.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -104.09131137591984 (T) = (0 -12.744269098075536) / Math.Sqrt((0.10488661762447232 / (42)) + (0.015165015723393146 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -1.3751502818477928 = (5.365668520208706 - 12.744269098075536) / 5.365668520208706 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayCopyToByteArray(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.46715815886035 > 10.771987891589553.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -26.816831863370425 (T) = (0 -30.84803027341543) / Math.Sqrt((1.516660946338197 / (38)) + (2.723560032236661 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -1.9823047500720363 = (10.343688140077 - 30.84803027341543) / 10.343688140077 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArraySetLengthShrink(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.855837258135818 > 196.31148587651936.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -2067.9492740025657 (T) = (0 -1856.143733421103) / Math.Sqrt((0.24167704268585374 / (44)) + (4.519212051710368 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -8.899074933128421 = (187.50678684220273 - 1856.143733421103) / 187.50678684220273 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayLengthValueCtor(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.103229224085023 > 18.019158112402316.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -54.66930775028433 (T) = (0 -25.342309849824698) / Math.Sqrt((0.1288431383992758 / (44)) + (0.13065432743158387 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.46412287836997046 = (17.308868145028008 - 25.342309849824698) / 17.308868145028008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayLengthValueCtor(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.061754903934336 > 16.333418581911555.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -47.0108901062717 (T) = (0 -21.12292462502577) / Math.Sqrt((0.3697195165527295 / (46)) + (0.042471707087629214 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -0.35928368379408415 = (15.53974705711663 - 21.12292462502577) / 15.53974705711663 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.8589290206463773 > 194.04648639859226.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -1637.223963849923 (T) = (0 -1855.7845787972954) / Math.Sqrt((0.4869026298539943 / (41)) + (7.2126460508372885 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -9.067778331104833 = (184.32910596211374 - 1855.7845787972954) / 184.32910596211374 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayLeftShift(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.141867446003923 > 4.35172052273976.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -61.00283003774314 (T) = (0 -11.32030315818399) / Math.Sqrt((0.017635771026742294 / (41)) + (0.09312133068582518 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -1.7137497547886644 = (4.17146169730951 - 11.32030315818399) / 4.17146169730951 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Tests.Perf_BitArray.BitArrayByteArrayCtor(Size: 4)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.935231461775395 > 12.043751516667333.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -106.39694606922856 (T) = (0 -29.994475288206324) / Math.Sqrt((1.2632354584091057 / (43)) + (0.0021356568700119536 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -1.5690121478165244 = (11.675489862396903 - 29.994475288206324) / 11.675489862396903 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 1.76 μs 3.31 μs 1.88 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<LoginViewModel>.XmlSerializer_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.313719027954256 > 1.8725378698561745.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -202.8047102895894 (T) = (0 -3341.0184978671823) / Math.Sqrt((211.6455734701265 / (38)) + (271.8301032243409 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.8865762669666382 = (1770.942715843178 - 3341.0184978671823) / 1770.942715843178 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.IO.Tests.Perf_StreamWriter

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteCharArray - Duration of single invocation 100.04 ms 316.26 ms 3.16 0.02 False
WritePartialCharArray - Duration of single invocation 428.06 ms 870.47 ms 2.03 0.03 False
WriteCharArray - Duration of single invocation 399.06 ms 684.63 ms 1.72 0.08 False
WriteString - Duration of single invocation 383.59 ms 737.15 ms 1.92 0.11 False
WritePartialCharArray - Duration of single invocation 126.04 ms 472.42 ms 3.75 0.01 False
WriteString - Duration of single invocation 100.20 ms 348.00 ms 3.47 0.02 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.Perf_StreamWriter*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_StreamWriter.WriteCharArray(writeLength: 2)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 316.26440833333334 > 105.056392875.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -151.0314969877279 (T) = (0 -319412661.47959185) / Math.Sqrt((1863400337939.7866 / (44)) + (14421104866056.39 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -2.180875651574586 = (100416582.24567102 - 319412661.47959185) / 100416582.24567102 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WritePartialCharArray(writeLength: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 870.47 > 424.70821400000005.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -35.11424234703785 (T) = (0 -880788385.5555557) / Math.Sqrt((104546509780028.77 / (40)) + (1035524831519457.4 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -1.1172606670383352 = (416003753.93910253 - 880788385.5555557) / 416003753.93910253 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WriteCharArray(writeLength: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 684.63306 > 418.706596.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -54.832273117471765 (T) = (0 -692879142.8571428) / Math.Sqrt((59268222129920.35 / (41)) + (189856023738471.9 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.7330355986064012 = (399806641.83373547 - 692879142.8571428) / 399806641.83373547 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WriteString(writeLength: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 737.1478333333333 > 409.28510000000006.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -40.14814265805707 (T) = (0 -745016061.025641) / Math.Sqrt((124568853401840.88 / (39)) + (356077973510519.2 / (5))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (5) - 2, .025) and -0.8686412075944893 = (398694012.52512443 - 745016061.025641) / 398694012.52512443 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WritePartialCharArray(writeLength: 2)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 472.42331666666666 > 132.34770836538462.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -155.33289381292528 (T) = (0 -481368523.2440476) / Math.Sqrt((550231259687.7588 / (39)) + (41710334452311.164 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -2.814148005988754 = (126206041.94914058 - 481368523.2440476) / 126206041.94914058 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.IO.Tests.Perf_StreamWriter.WriteString(writeLength: 2)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 348.0015708333333 > 105.22896515384615.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -171.03402206348522 (T) = (0 -352132648.8076923) / Math.Sqrt((2047824150873.234 / (40)) + (10561403685218.355 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -2.5016192762948677 = (100562802.81284343 - 352132648.8076923) / 100562802.81284343 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Memory.Slice<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
MemorySpanStart - Duration of single invocation 7.91 ns 20.41 ns 2.58 0.01 False
ReadOnlyMemorySpanStart - Duration of single invocation 7.53 ns 19.50 ns 2.59 0.06 False
ReadOnlySpanStartLength - Duration of single invocation 4.52 ns 9.61 ns 2.13 0.00 False
SpanStart - Duration of single invocation 4.58 ns 9.76 ns 2.13 0.02 False
ReadOnlyMemoryStartSpan - Duration of single invocation 8.94 ns 14.17 ns 1.58 0.02 False
ReadOnlyMemoryStartLengthSpan - Duration of single invocation 9.31 ns 14.48 ns 1.55 0.02 False
MemoryStartLengthSpan - Duration of single invocation 9.97 ns 15.75 ns 1.58 0.06 False
MemoryStartSpan - Duration of single invocation 9.33 ns 15.12 ns 1.62 0.04 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Slice&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Slice<String>.MemorySpanStart

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.411312377089477 > 8.30665902296617.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -192.10696662064464 (T) = (0 -20.46932363935708) / Math.Sqrt((0.023578627485733715 / (39)) + (0.01920512957924311 / (5))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (5) - 2, .025) and -1.672054448331384 = (7.660518913504754 - 20.46932363935708) / 7.660518913504754 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;String&gt;.ReadOnlyMemorySpanStart

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.49749969712635 > 7.904465101487435.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -3699.2119504222833 (T) = (0 -19.497397611841603) / Math.Sqrt((0.0004548805463823229 / (44)) + (5.881642078125692E-07 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -1.5807066285127 = (7.555061623985616 - 19.497397611841603) / 7.555061623985616 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;String&gt;.ReadOnlySpanStartLength

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.612243151256159 > 4.7483101040265785.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -3453.529072551206 (T) = (0 -9.616352339258347) / Math.Sqrt((8.02691960852145E-06 / (41)) + (1.3861307704142795E-05 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -1.1265713062438205 = (4.521998538691743 - 9.616352339258347) / 4.521998538691743 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;String&gt;.SpanStart

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.757862513988288 > 4.826876029575352.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -657.0929687313621 (T) = (0 -9.758119158670038) / Math.Sqrt((0.0023461678290031727 / (38)) + (4.985782499069172E-07 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -1.125579735364397 = (4.5908036270383255 - 9.758119158670038) / 4.5908036270383255 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;String&gt;.ReadOnlyMemoryStartSpan

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.165376045791033 > 9.401266191400769.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -99.16814566038353 (T) = (0 -14.243401058998865) / Math.Sqrt((0.006830164193518255 / (39)) + (0.02083530104246837 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.5799460717754337 = (9.015118498945423 - 14.243401058998865) / 9.015118498945423 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;String&gt;.ReadOnlyMemoryStartLengthSpan

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.477314435633259 > 9.784901600856713.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -68.88034140049989 (T) = (0 -14.601715353404538) / Math.Sqrt((0.0016143125579862545 / (40)) + (0.029108356442541377 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.5653692755450214 = (9.327968538491083 - 14.601715353404538) / 9.327968538491083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;String&gt;.MemoryStartLengthSpan

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.751153618386033 > 10.451146057137384.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -95.2962553849623 (T) = (0 -15.522506538371) / Math.Sqrt((0.016489308370506557 / (44)) + (0.02366949553892476 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.6032216215187726 = (9.682071605088593 - 15.522506538371) / 9.682071605088593 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Slice&lt;String&gt;.MemoryStartSpan

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.118690696479987 > 9.89462502393687.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -89.52372221487254 (T) = (0 -15.181787465461019) / Math.Sqrt((0.01787849811660481 / (42)) + (0.019278074805305793 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.6282295397685126 = (9.324107624051234 - 15.181787465461019) / 9.324107624051234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 137.35 ns 156.12 ns 1.14 0.00 False
ToString - Duration of single invocation 140.08 ns 156.01 ns 1.11 0.01 False
ToString - Duration of single invocation 136.83 ns 157.54 ns 1.15 0.00 False
ToString - Duration of single invocation 137.02 ns 157.38 ns 1.15 0.01 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: )

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 156.1185295669759 > 144.24479003750318.
IsChangePoint: Marked as a change because one of 5/24/2022 10:45:34 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -34.623355936764604 (T) = (0 -157.0850547134107) / Math.Sqrt((1.0135873775406832 / (40)) + (1.939608176670197 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.14960710868665034 = (136.64238288581038 - 157.0850547134107) / 136.64238288581038 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 156.01267618483888 > 143.88068264624337.
IsChangePoint: Marked as a change because one of 5/19/2022 10:26:24 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -80.10629722149197 (T) = (0 -156.28857337264407) / Math.Sqrt((1.11955088469584 / (41)) + (0.2246975762127074 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.14276084413174445 = (136.76402562724329 - 156.28857337264407) / 136.76402562724329 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: fr)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 157.53525073409486 > 143.59910920126043.
IsChangePoint: Marked as a change because one of 5/24/2022 7:55:26 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -66.855449405026 (T) = (0 -157.16002537063153) / Math.Sqrt((0.9175608961567188 / (44)) + (0.507770954047117 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.14942807759624763 = (136.72889016187418 - 157.16002537063153) / 136.72889016187418 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ja)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 157.378156051054 > 143.91756639242274.
IsChangePoint: Marked as a change because one of 5/24/2022 7:55:26 PM, 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -35.19366655171596 (T) = (0 -157.27551339786802) / Math.Sqrt((0.9316247843762708 / (39)) + (2.4910599879112953 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.1488560810165888 = (136.89748959565028 - 157.27551339786802) / 136.89748959565028 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromStream - Duration of single invocation 35.59 μs 52.86 μs 1.48 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;IndexViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<IndexViewModel>.DeserializeFromStream

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.85560593279911 > 37.47920494134519.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -121.62966167571392 (T) = (0 -53162.908174920696) / Math.Sqrt((93518.61941538294 / (38)) + (92429.04771578629 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.4950440898274544 = (35559.42499398551 - 53162.908174920696) / 35559.42499398551 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
JsonNet_ - Duration of single invocation 699.56 μs 814.26 μs 1.16 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Json_ToString&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Json_ToString<MyEventsListerViewModel>.JsonNet_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 814.264394888499 > 742.1390531021431.
IsChangePoint: Marked as a change because one of 6/2/2022 2:22:06 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -27.723755019603843 (T) = (0 -817032.9830466347) / Math.Sqrt((125069378.27818507 / (39)) + (51731973.28517677 / (5))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (5) - 2, .025) and -0.14275380541487043 = (714968.5078055946 - 817032.9830466347) / 714968.5078055946 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation 350.56 ns 432.76 ns 1.23 0.08 False
ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation 114.31 ns 196.16 ns 1.72 0.06 False
ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation 100.15 ns 191.15 ns 1.91 0.04 False
ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation 357.71 ns 442.98 ns 1.24 0.08 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 1000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 432.7644176576678 > 376.5464231906477.
IsChangePoint: Marked as a change because one of 6/6/2022 10:51:35 AM, 6/15/2022 12:09:49 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -26.400791765828238 (T) = (0 -430.51860818736014) / Math.Sqrt((56.42842267497937 / (44)) + (44.820211965992456 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.20482380721227317 = (357.3291012430241 - 430.51860818736014) / 357.3291012430241 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 196.15806609737044 > 122.5963306679865.
IsChangePoint: Marked as a change because one of 6/6/2022 10:51:35 AM, 6/17/2022 12:17:51 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -60.20578626158038 (T) = (0 -191.06087364499487) / Math.Sqrt((4.165072225668508 / (42)) + (8.3773815564617 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.6269177403084332 = (117.43732882817622 - 191.06087364499487) / 117.43732882817622 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 191.14846693326118 > 105.46570693109571.
IsChangePoint: Marked as a change because one of 6/23/2022 1:01:38 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -63.86741601701728 (T) = (0 -190.67289929660618) / Math.Sqrt((67.49331939134763 / (41)) + (0.2676444164273125 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.7690508001997082 = (107.78260255447786 - 190.67289929660618) / 107.78260255447786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 1000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 442.9805900106913 > 376.3936843837358.
IsChangePoint: Marked as a change because one of 5/23/2022 1:47:22 PM, 6/6/2022 10:51:35 AM, 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -22.14010817127633 (T) = (0 -434.9530944580048) / Math.Sqrt((59.77525564260802 / (39)) + (80.28226506156929 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.20937632804248227 = (359.6507425955885 - 434.9530944580048) / 359.6507425955885 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FirstSpanMemory - Duration of single invocation 8.66 ns 32.04 ns 3.70 0.01 False
FirstSpanTenSegments - Duration of single invocation 6.95 ns 21.99 ns 3.16 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Char>.FirstSpanMemory

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.04395178577317 > 9.119330919511368.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -214.02439566349403 (T) = (0 -32.1619727339952) / Math.Sqrt((0.03836201571059912 / (38)) + (0.053310918655962256 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -2.5578586823188134 = (9.039699326403213 - 32.1619727339952) / 9.039699326403213 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Char&gt;.FirstSpanTenSegments

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.985305754507767 > 7.294254148182471.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -142.17555548353835 (T) = (0 -22.0378165942997) / Math.Sqrt((0.3987864989959264 / (43)) + (0.005751198619626125 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -1.816249849992514 = (7.8252348932600135 - 22.0378165942997) / 7.8252348932600135 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<BinaryData>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 926.31 ns 1.09 μs 1.18 0.04 False
SerializeToString - Duration of single invocation 707.18 ns 846.22 ns 1.20 0.05 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<BinaryData>.SerializeObjectProperty

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0911776522616368 > 985.4218408887008.
IsChangePoint: Marked as a change because one of 5/24/2022 1:50:23 PM, 6/20/2022 7:45:04 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -28.749641424944993 (T) = (0 -1094.261582446698) / Math.Sqrt((322.3860306407578 / (38)) + (121.2893472387428 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.17693533681116752 = (929.755058091408 - 1094.261582446698) / 929.755058091408 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;BinaryData&gt;.SerializeToString

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 846.217615013369 > 746.9060609616083.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/10/2022 7:56:49 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -28.78796968653562 (T) = (0 -832.1277212008556) / Math.Sqrt((164.29221545195924 / (44)) + (116.0581082648251 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.18472844686619821 = (702.3784424202613 - 832.1277212008556) / 702.3784424202613 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_FromStream<XmlElement>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 3.02 μs 4.41 μs 1.46 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_FromStream&lt;XmlElement&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_FromStream<XmlElement>.XmlSerializer_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.412531959940173 > 3.1849103990382672.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -24.923173216799853 (T) = (0 -4268.391739625441) / Math.Sqrt((6131.080949690331 / (38)) + (9926.843136349205 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.37087071707069236 = (3113.6355065970315 - 4268.391739625441) / 3113.6355065970315 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
NewCustomizedOptions - Duration of single invocation 1.32 μs 1.71 μs 1.30 0.08 False
CachedDefaultOptions - Duration of single invocation 745.16 ns 1.05 μs 1.41 0.03 False
NewDefaultOptions - Duration of single invocation 1.23 μs 1.56 μs 1.27 0.03 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ColdStartSerialization<SimpleStructWithProperties>.NewCustomizedOptions

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7144746437010783 > 1.3907630895909087.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -42.07556494105553 (T) = (0 -1714.012506031482) / Math.Sqrt((1221.0283541014937 / (48)) + (383.76166605599155 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.28192315566013315 = (1337.0633789268297 - 1714.012506031482) / 1337.0633789268297 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;.CachedDefaultOptions

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0528167860024205 > 766.020789930514.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -44.8416302540353 (T) = (0 -1063.8873423963448) / Math.Sqrt((255.0506368979054 / (40)) + (294.44537013169213 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.4574323383736743 = (729.9737451849874 - 1063.8873423963448) / 729.9737451849874 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ColdStartSerialization&lt;SimpleStructWithProperties&gt;.NewDefaultOptions

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5573152506177956 > 1.2883511587839902.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -64.08911760794363 (T) = (0 -1561.4809840045548) / Math.Sqrt((563.9572985886587 / (40)) + (69.76246354162292 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.27776379426905573 = (1222.0419697349457 - 1561.4809840045548) / 1222.0419697349457 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 23.85 μs 30.29 μs 1.27 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;Dictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<Dictionary<String, String>>.DeserializeFromString

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.285387019894998 > 24.588714261227764.
IsChangePoint: Marked as a change because one of 5/28/2022 6:51:39 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -45.486199494259274 (T) = (0 -30246.243218431027) / Math.Sqrt((602533.5068870219 / (38)) + (28541.96504894567 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.28343390313822414 = (23566.654382803496 - 30246.243218431027) / 23566.654382803496 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 6.55 μs 11.95 μs 1.83 0.01 False
SerializeToString - Duration of single invocation 7.02 μs 12.46 μs 1.78 0.01 False
SerializeToStream - Duration of single invocation 6.55 μs 12.18 μs 1.86 0.01 False
SerializeObjectProperty - Duration of single invocation 7.29 μs 12.69 μs 1.74 0.02 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<HashSet<String>>.SerializeToUtf8Bytes

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.953969539416377 > 6.8489622736956175.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -168.26393352596546 (T) = (0 -11966.402037470962) / Math.Sqrt((4775.5455105495475 / (44)) + (6415.747369491746 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.8188391579741614 = (6579.142517911942 - 11966.402037470962) / 6579.142517911942 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeToString

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.460422006721352 > 7.361173320140841.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -88.63132476252092 (T) = (0 -12502.088024525081) / Math.Sqrt((4444.983479218433 / (42)) + (22635.057812223407 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.7905007278836775 = (6982.453472276554 - 12502.088024525081) / 6982.453472276554 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeToStream

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.183314176245212 > 6.9458334212748.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -143.86428536492653 (T) = (0 -11994.237907673338) / Math.Sqrt((1788.1343024543876 / (41)) + (9431.585573001545 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.8094422793044188 = (6628.693296745631 - 11994.237907673338) / 6628.693296745631 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;HashSet&lt;String&gt;&gt;.SerializeObjectProperty

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.694101827318079 > 7.661503300511152.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -157.94536306985012 (T) = (0 -12703.22317942094) / Math.Sqrt((8795.63165256792 / (39)) + (7775.221615752808 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.7551479949096433 = (7237.693468735048 - 12703.22317942094) / 7237.693468735048 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 2.14 secs 3.26 secs 1.52 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_SliceSlice.Count(Options: IgnoreCase, Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.256655575 > 2.2487487925000003.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -248.33046225412664 (T) = (0 -3259801251.666667) / Math.Sqrt((661018867425779.2 / (38)) + (12266484032409.824 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.5137596194078099 = (2153447092.836257 - 3259801251.666667) / 2153447092.836257 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in MicroBenchmarks.Serializers.Binary_ToStream<MyEventsListerViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BinaryFormatter_ - Duration of single invocation 42.19 μs 52.03 μs 1.23 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Binary_ToStream&lt;MyEventsListerViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Binary_ToStream<MyEventsListerViewModel>.BinaryFormatter_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.02951889100299 > 44.7970951549312.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -20.99731121848116 (T) = (0 -52223.58671024508) / Math.Sqrt((249053.1940346929 / (39)) + (1030726.5737578395 / (5))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (5) - 2, .025) and -0.22753120955744607 = (42543.59180739112 - 52223.58671024508) / 42543.59180739112 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
XmlSerializer_ - Duration of single invocation 1.60 μs 2.76 μs 1.73 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'MicroBenchmarks.Serializers.Xml_ToStream&lt;SimpleStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

MicroBenchmarks.Serializers.Xml_ToStream<SimpleStructWithProperties>.XmlSerializer_

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7631729188681464 > 1.6786296185342477.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -102.1355486552905 (T) = (0 -2751.3190002957117) / Math.Sqrt((344.0359868259677 / (39)) + (604.9706707014318 / (5))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (5) - 2, .025) and -0.7329823298310855 = (1587.6209196915954 - 2751.3190002957117) / 1587.6209196915954 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Net.NetworkInformation.Tests.PhysicalAddressTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
PAMedium - Duration of single invocation 22.95 ns 59.68 ns 2.60 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Net.NetworkInformation.Tests.PhysicalAddressTests*'

Payloads

Baseline
Compare

Histogram

System.Net.NetworkInformation.Tests.PhysicalAddressTests.PAMedium

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 59.67532722672105 > 24.15898295479492.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -116.2542477903999 (T) = (0 -59.31011829598193) / Math.Sqrt((0.156934825772471 / (38)) + (0.4660446236637649 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -1.574250262233335 = (23.039763913445782 - 59.31011829598193) / 23.039763913445782 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToUtf8Bytes - Duration of single invocation 764.98 ns 1.37 μs 1.79 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToUtf8Bytes

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3731363014110782 > 802.1935937211485.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -39.55963900658707 (T) = (0 -1397.7223266405674) / Math.Sqrt((154.54560263740294 / (38)) + (1243.9223130440191 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.8183586949091693 = (768.6725015002535 - 1397.7223266405674) / 768.6725015002535 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FirstSpanMemory - Duration of single invocation 8.13 ns 29.72 ns 3.65 0.00 False
FirstSpanArray - Duration of single invocation 7.09 ns 28.91 ns 4.08 0.01 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanMemory

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 29.715228543951834 > 8.617958260745782.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -1405.4895626622226 (T) = (0 -29.716371956450207) / Math.Sqrt((0.008290435401755204 / (38)) + (6.909727223752372E-05 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -2.5763607651875504 = (8.309109149644703 - 29.716371956450207) / 8.309109149644703 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;.FirstSpanArray

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.90535609119973 > 7.4438169627346875.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -1255.0368323899804 (T) = (0 -28.830408668656876) / Math.Sqrt((0.004829962137910303 / (41)) + (0.0012579226306541882 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -3.013680351890524 = (7.183035553660164 - 28.830408668656876) / 7.183035553660164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in BenchmarksGame.RegexRedux_5

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 30.85 ms 36.74 ms 1.19 0.28 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.RegexRedux_5*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.RegexRedux_5.RunBench(options: None)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.74021 > 31.556302968750003.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -10.02723576373787 (T) = (0 -40349931.163312696) / Math.Sqrt((964827195428.2745 / (38)) + (4904257726064.199 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.33205675596872175 = (30291450.40743306 - 40349931.163312696) / 30291450.40743306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Collections.AddGivenSize<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 356.86 μs 450.49 μs 1.26 0.11 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.AddGivenSize&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<String>.SortedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 450.4864404761905 > 363.27619089673914.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -42.431022437358365 (T) = (0 -452678.1331937315) / Math.Sqrt((50920003.27905442 / (40)) + (27433754.40726607 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.29302443409684 = (350092.4818253114 - 452678.1331937315) / 350092.4818253114 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromStream - Duration of single invocation 1.74 μs 2.42 μs 1.39 0.00 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;LargeStructWithProperties&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<LargeStructWithProperties>.DeserializeFromStream

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4190362319243914 > 1.8304591101794099.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -137.8019321288499 (T) = (0 -2414.286933507186) / Math.Sqrt((325.34445277978386 / (38)) + (71.6745025089817 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.375740979827054 = (1754.89933709083 - 2414.286933507186) / 1754.89933709083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Collections.ContainsTrueComparer<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 304.91 μs 418.17 μs 1.37 0.14 False
ImmutableSortedSet - Duration of single invocation 324.29 μs 426.21 μs 1.31 0.22 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrueComparer&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrueComparer<String>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 418.1715972222222 > 306.7294929108797.
IsChangePoint: Marked as a change because one of 5/28/2022 1:41:21 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -7.082212255388545 (T) = (0 -436673.82334566704) / Math.Sqrt((38549192.57954434 / (40)) + (2143932646.7844276 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.4429841117094934 = (302618.5942049927 - 436673.82334566704) / 302618.5942049927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsTrueComparer&lt;String&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 426.20926801801807 > 338.40939999999995.
IsChangePoint: Marked as a change because one of 5/11/2022 9:40:50 AM, 5/11/2022 7:02:26 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -19.180985819954085 (T) = (0 -419246.4743057835) / Math.Sqrt((52427262.05216668 / (39)) + (112999212.71044688 / (5))) is less than -2.0180817028167235 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (5) - 2, .025) and -0.2884483522780845 = (325388.65338647115 - 419246.4743057835) / 325388.65338647115 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.ReadJson<ArrayList>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DeserializeFromString - Duration of single invocation 39.23 μs 53.30 μs 1.36 0.03 False
DeserializeFromUtf8Bytes - Duration of single invocation 39.44 μs 52.45 μs 1.33 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.ReadJson&lt;ArrayList&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.ReadJson<ArrayList>.DeserializeFromString

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 53.30351068999028 > 41.000440251623665.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -42.9895765047128 (T) = (0 -53488.36859439459) / Math.Sqrt((646924.0880254011 / (48)) + (665984.633545377 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.36032894346923783 = (39320.17241211053 - 53488.36859439459) / 39320.17241211053 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.ReadJson&lt;ArrayList&gt;.DeserializeFromUtf8Bytes

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.45255388487352 > 40.89151795321342.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -89.0088259649321 (T) = (0 -52684.27291695993) / Math.Sqrt((240565.99033591268 / (44)) + (134838.25623111622 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.36181027380856384 = (38686.94041323264 - 52684.27291695993) / 38686.94041323264 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfValue - Duration of single invocation 9.56 ns 18.41 ns 1.93 0.06 False
BinarySearch - Duration of single invocation 15.94 ns 35.62 ns 2.23 0.03 False
SequenceEqual - Duration of single invocation 12.64 ns 27.11 ns 2.14 0.07 False
Clear - Duration of single invocation 6.14 ns 14.45 ns 2.35 0.35 False
Fill - Duration of single invocation 4.81 ns 14.51 ns 3.02 0.18 False
LastIndexOfAnyValues - Duration of single invocation 19.27 ns 22.14 ns 1.15 0.09 False
SequenceCompareTo - Duration of single invocation 14.32 ns 25.89 ns 1.81 0.25 False
IndexOfAnyThreeValues - Duration of single invocation 13.61 ns 21.44 ns 1.58 0.10 False
LastIndexOfValue - Duration of single invocation 12.76 ns 24.40 ns 1.91 0.06 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.412603949014397 > 10.027727740000033.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 6/2/2022 5:58:32 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -27.256356559912604 (T) = (0 -18.325846800436242) / Math.Sqrt((0.23725631342214965 / (38)) + (0.5108898368428066 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.9597678329116485 = (9.351029490676623 - 18.325846800436242) / 9.351029490676623 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.BinarySearch(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 35.61847068065715 > 16.876703277125756.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 6/17/2022 12:17:51 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -108.38233735833529 (T) = (0 -35.17721584230513) / Math.Sqrt((0.006090310827856127 / (38)) + (0.15488500820251688 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -1.1914084030155703 = (16.05233227813592 - 35.17721584230513) / 16.05233227813592 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.SequenceEqual(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.109822851656563 > 13.277843385255203.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -48.02464879307614 (T) = (0 -26.068309003776342) / Math.Sqrt((0.10609915268212057 / (46)) + (0.5115413528763081 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -1.0235277810900163 = (12.882604947352931 - 26.068309003776342) / 12.882604947352931 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.Clear(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.450024954966024 > 6.532242993459745.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -82.14854081199947 (T) = (0 -14.553931832136954) / Math.Sqrt((0.14893586228766273 / (43)) + (0.05043997844190274 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -1.3982332952062029 = (6.068605527756043 - 14.553931832136954) / 6.068605527756043 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.Fill(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.512698595465716 > 5.0498661265740985.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -170.03761263561637 (T) = (0 -14.463088975808068) / Math.Sqrt((0.11211293012823241 / (45)) + (0.004083091524043604 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -1.9597363551473737 = (4.886613954872987 - 14.463088975808068) / 4.886613954872987 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfAnyValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.143219542761717 > 20.239833551387477.
IsChangePoint: Marked as a change because one of 6/22/2022 10:05:16 PM, 6/30/2022 12:21:07 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -9.981410021454616 (T) = (0 -22.05722957458205) / Math.Sqrt((8.767159602650754 / (41)) + (0.19983502611389722 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.28664891046987506 = (17.14316111807603 - 22.05722957458205) / 17.14316111807603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.SequenceCompareTo(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.893290535756726 > 15.07275738805472.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -73.27052789467407 (T) = (0 -25.538637813020706) / Math.Sqrt((0.28989723773272696 / (44)) + (0.11320893441199593 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.7631787238768637 = (14.484429438251471 - 25.538637813020706) / 14.484429438251471 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyThreeValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.438413710025767 > 14.274880887467331.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -54.472793492132745 (T) = (0 -21.286872873637666) / Math.Sqrt((0.09910079904169791 / (43)) + (0.137619647699347 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.5561977054850777 = (13.678771533082552 - 21.286872873637666) / 13.678771533082552 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.LastIndexOfValue(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.401571659617453 > 13.39636294796118.
IsChangePoint: Marked as a change because one of 6/6/2022 10:51:35 AM, 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -69.43954542752356 (T) = (0 -24.33377696427555) / Math.Sqrt((0.31508581268064895 / (40)) + (0.1469751314570115 / (8))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (8) - 2, .025) and -0.8598840801931446 = (13.083491182820676 - 24.33377696427555) / 13.083491182820676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation 204.21 ns 290.74 ns 1.42 0.23 False
WriteByteArrayAsBase64_NoEscaping - Duration of single invocation 76.20 ns 147.80 ns 1.94 0.18 False
WriteByteArrayAsBase64_NoEscaping - Duration of single invocation 208.41 ns 284.60 ns 1.37 0.22 False
WriteByteArrayAsBase64_HeavyEscaping - Duration of single invocation 73.64 ns 147.35 ns 2.00 0.16 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Base64*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 1000)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 290.73740552607563 > 218.52057351221708.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -16.92933709500371 (T) = (0 -286.27491651731816) / Math.Sqrt((45.68023331686689 / (44)) + (129.10923497356336 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.35322065949907744 = (211.5508025300831 - 286.27491651731816) / 211.5508025300831 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 147.7976743525293 > 77.58553313220153.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -63.98913731661093 (T) = (0 -150.05600529133167) / Math.Sqrt((4.427334731540456 / (38)) + (8.888451961011304 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -1.0083856785026515 = (74.71473576888164 - 150.05600529133167) / 74.71473576888164 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_NoEscaping(NumberOfBytes: 1000)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 284.6026517251254 > 220.94663813594607.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -17.68440271853305 (T) = (0 -287.1813220535497) / Math.Sqrt((24.8633476820495 / (42)) + (83.67473351244645 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.3446777505117582 = (213.5688806810807 - 287.1813220535497) / 213.5688806810807 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Base64.WriteByteArrayAsBase64_HeavyEscaping(NumberOfBytes: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 147.3514132587178 > 78.06929910769422.
IsChangePoint: Marked as a change because one of 5/9/2022 6:32:22 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -64.60040183941307 (T) = (0 -148.56081985694146) / Math.Sqrt((8.935243571420758 / (42)) + (6.3360067196283545 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.9600127581344426 = (75.7958432874401 - 148.56081985694146) / 75.7958432874401 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeObjectProperty - Duration of single invocation 625.42 ns 892.00 ns 1.43 0.05 False
SerializeToUtf8Bytes - Duration of single invocation 368.12 ns 631.48 ns 1.72 0.06 False
SerializeToString - Duration of single invocation 400.81 ns 639.85 ns 1.60 0.03 False
SerializeToStream - Duration of single invocation 523.88 ns 779.70 ns 1.49 0.05 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<LoginViewModel>.SerializeObjectProperty

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 891.9992466804946 > 661.3699007913078.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -79.19998536651106 (T) = (0 -894.237277580776) / Math.Sqrt((98.55968925834641 / (44)) + (68.53219879691646 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.4434025044246359 = (619.5342427628901 - 894.237277580776) / 619.5342427628901 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;.SerializeToUtf8Bytes

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 631.4793766749649 > 379.88301416122283.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -32.54632342828424 (T) = (0 -604.4060726272672) / Math.Sqrt((80.72214269885694 / (42)) + (272.8062196489218 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.6798241355931713 = (359.80318404809816 - 604.4060726272672) / 359.80318404809816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;.SerializeToString

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 639.8518066629465 > 426.8882597207746.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -78.68081678920369 (T) = (0 -647.0165807163809) / Math.Sqrt((65.75672179140331 / (38)) + (53.63557240878723 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -0.5941084347579001 = (405.879905412234 - 647.0165807163809) / 405.879905412234 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Serialization.Tests.WriteJson&lt;LoginViewModel&gt;.SerializeToStream

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 779.698366181785 > 557.4515926573955.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -65.39163576801576 (T) = (0 -777.8825484559233) / Math.Sqrt((113.33445325799244 / (42)) + (74.75139306416945 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.4864948996926706 = (523.2998435559709 - 777.8825484559233) / 523.2998435559709 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 51.88 ns 104.72 ns 2.02 0.02 False
IsMatch - Duration of single invocation 124.10 ns 145.70 ns 1.17 0.03 False
IsMatch - Duration of single invocation 53.22 ns 152.26 ns 2.86 0.03 False
IsMatch - Duration of single invocation 178.48 ns 231.87 ns 1.30 0.00 False
IsMatch - Duration of single invocation 51.52 ns 79.35 ns 1.54 0.05 False
IsMatch - Duration of single invocation 55.69 ns 150.97 ns 2.71 0.01 False
IsMatch - Duration of single invocation 126.35 ns 170.53 ns 1.35 0.01 False
IsMatch - Duration of single invocation 228.48 ns 278.95 ns 1.22 0.02 False
IsMatch - Duration of single invocation 91.92 ns 262.39 ns 2.85 0.00 False
IsMatch - Duration of single invocation 353.33 ns 401.96 ns 1.14 0.04 False
IsMatch - Duration of single invocation 217.27 ns 251.96 ns 1.16 0.02 False
IsMatch - Duration of single invocation 217.36 ns 249.98 ns 1.15 0.01 False
IsMatch - Duration of single invocation 130.25 ns 167.34 ns 1.28 0.01 False
IsMatch - Duration of single invocation 55.91 ns 150.84 ns 2.70 0.03 False
IsMatch - Duration of single invocation 85.52 ns 138.95 ns 1.62 0.01 False
IsMatch - Duration of single invocation 108.52 ns 148.84 ns 1.37 0.01 False
IsMatch - Duration of single invocation 405.82 ns 459.07 ns 1.13 0.01 False
IsMatch - Duration of single invocation 125.01 ns 201.46 ns 1.61 0.03 False
IsMatch - Duration of single invocation 118.41 ns 161.77 ns 1.37 0.00 False
IsMatch - Duration of single invocation 129.78 ns 168.62 ns 1.30 0.02 False
IsMatch - Duration of single invocation 127.66 ns 179.91 ns 1.41 0.01 False
IsMatch - Duration of single invocation 121.73 ns 196.90 ns 1.62 0.04 False
IsMatch - Duration of single invocation 347.32 ns 392.31 ns 1.13 0.05 False
IsMatch - Duration of single invocation 82.72 ns 119.23 ns 1.44 0.01 False
IsMatch - Duration of single invocation 142.21 ns 230.84 ns 1.62 0.02 False
IsMatch - Duration of single invocation 53.54 ns 107.88 ns 2.02 0.02 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 104.72425727252808 > 54.26775281040343.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -161.08180077751226 (T) = (0 -104.43871699318522) / Math.Sqrt((0.31448235740649555 / (38)) + (0.4947000283772126 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -1.0203115637429756 = (51.69436183382255 - 104.43871699318522) / 51.69436183382255 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 7, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 145.70359199061713 > 129.95045679440037.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -18.969684843881012 (T) = (0 -147.10564559147969) / Math.Sqrt((8.845748090312044 / (46)) + (6.708593448112571 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -0.16053266671824315 = (126.75700547703268 - 147.10564559147969) / 126.75700547703268 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 152.26068663652782 > 55.88946050055428.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -483.78631253929797 (T) = (0 -152.61084876730416) / Math.Sqrt((0.4354106728689776 / (41)) + (0.22044797386392256 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -1.861543749952279 = (53.33164966282595 - 152.61084876730416) / 53.33164966282595 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 231.8708204974078 > 187.5593806712202.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -98.2162107417936 (T) = (0 -231.83258467483552) / Math.Sqrt((0.2282058927979999 / (41)) + (1.9667146027834645 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.2932802126974849 = (179.25936111810302 - 231.83258467483552) / 179.25936111810302 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.35387372265853 > 54.09502519535711.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -136.24986809517367 (T) = (0 -79.34208693518632) / Math.Sqrt((0.8358744942924055 / (41)) + (0.14601123468583468 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.5355253144140557 = (51.67097291747523 - 79.34208693518632) / 51.67097291747523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 150.96539822139246 > 58.51777614607.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -417.7874997785331 (T) = (0 -151.15477165342597) / Math.Sqrt((0.9175592828461824 / (41)) + (0.20273928325101429 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -1.6758257069428077 = (56.48901991681804 - 151.15477165342597) / 56.48901991681804 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 10, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 170.52800539713493 > 132.564296995983.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -104.64593074028495 (T) = (0 -168.8816711693114) / Math.Sqrt((0.8224936221469157 / (43)) + (1.222990526069982 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.3458659881103653 = (125.48178842562635 - 168.8816711693114) / 125.48178842562635 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 278.9495119880591 > 234.2224956413281.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -39.88432602505173 (T) = (0 -277.5162305298873) / Math.Sqrt((31.541870376006372 / (38)) + (7.9135293063755805 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -0.2519324461716863 = (221.67029169865413 - 277.5162305298873) / 221.67029169865413 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 262.38814816703206 > 95.7671563906347.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -430.4918107980545 (T) = (0 -263.8946978632602) / Math.Sqrt((1.1946714792784434 / (41)) + (0.9383344605218659 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -1.9324261641360903 = (89.99193264973651 - 263.8946978632602) / 89.99193264973651 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 401.96388960814784 > 370.8899025310064.
IsChangePoint: Marked as a change because one of 6/13/2022 2:27:15 PM, 6/17/2022 12:17:51 PM, 6/19/2022 10:30:09 AM, 6/22/2022 6:53:40 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -78.70178009716156 (T) = (0 -401.80117874091326) / Math.Sqrt((1.0483362444532 / (40)) + (1.8127702855424634 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.13911832861370804 = (352.729974268696 - 401.80117874091326) / 352.729974268696 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 12, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 251.9642502944276 > 226.48057144949544.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -39.365105664499715 (T) = (0 -251.80140596515372) / Math.Sqrt((8.926617144839124 / (38)) + (3.5294561488447442 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -0.15527264197924884 = (217.95842541009173 - 251.80140596515372) / 217.95842541009173 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 249.98335029261492 > 228.4201741410402.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -53.19608482506593 (T) = (0 -251.27032196180653) / Math.Sqrt((8.689537835230064 / (44)) + (1.287965695534315 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.1504310836368994 = (218.41405846533334 - 251.27032196180653) / 218.41405846533334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 8, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 167.33880600586008 > 136.73834223760113.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -126.33056715738083 (T) = (0 -166.94770052843097) / Math.Sqrt((1.580339691206928 / (44)) + (0.3921495732680145 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.2977635419768044 = (128.64261872708312 - 166.94770052843097) / 128.64261872708312 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 150.84470918828364 > 58.67714929313566.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -671.7716762808614 (T) = (0 -150.95173616444612) / Math.Sqrt((0.27709663956906244 / (42)) + (0.0799939942634516 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -1.690014656915691 = (56.115581294833504 - 150.95173616444612) / 56.115581294833504 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 138.9531853194703 > 89.82007549298102.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -219.52555468065248 (T) = (0 -139.16579976281466) / Math.Sqrt((0.501302091646789 / (43)) + (0.3915975073343613 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.6349070854840698 = (85.12153442751145 - 139.16579976281466) / 85.12153442751145 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 11, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 148.83646412807857 > 113.9548853865343.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -94.09189263558565 (T) = (0 -148.1833470823372) / Math.Sqrt((1.936267221058013 / (41)) + (1.030584657394331 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.38889067633964847 = (106.69187259062532 - 148.1833470823372) / 106.69187259062532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 459.0704702933805 > 426.10933327536526.
IsChangePoint: Marked as a change because one of 6/9/2022 8:09:33 PM, 6/19/2022 10:30:09 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -75.32099609960018 (T) = (0 -459.2007125345567) / Math.Sqrt((12.889695521378545 / (42)) + (0.5459455761472417 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.11832405583688217 = (410.6150718459868 - 459.2007125345567) / 410.6150718459868 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 5, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 201.4584969237307 > 130.0766735145734.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -172.69411285306634 (T) = (0 -199.94570738300044) / Math.Sqrt((4.81658459022904 / (48)) + (0.6361043899202854 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.6069023721797526 = (124.42928135812967 - 199.94570738300044) / 124.42928135812967 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 13, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 161.77278987408184 > 124.28960486079126.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -113.98355048439322 (T) = (0 -161.68744474317967) / Math.Sqrt((0.9251493532631764 / (42)) + (0.6442499194178297 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.3770842787135794 = (117.41288985901576 - 161.68744474317967) / 117.41288985901576 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 9, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 168.6235328093171 > 136.24777572070334.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -53.51284049162814 (T) = (0 -170.31122612338308) / Math.Sqrt((2.3953119836375762 / (38)) + (2.8713681425355304 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.33482080206925896 = (127.59107878702827 - 170.31122612338308) / 127.59107878702827 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 0, Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 179.91278930148832 > 132.49013585818147.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -153.44032547116956 (T) = (0 -180.41642390259457) / Math.Sqrt((4.29876684835683 / (40)) + (0.2428332568884753 / (8))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (8) - 2, .025) and -0.461425340089953 = (123.45237143040757 - 180.41642390259457) / 123.45237143040757 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 196.8997320777276 > 127.71639878323636.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -187.20828937434035 (T) = (0 -197.77155579114876) / Math.Sqrt((1.3787235183717057 / (39)) + (1.0211780521161609 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.6185620161116265 = (122.18966825025822 - 197.77155579114876) / 122.18966825025822 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 4, Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 392.3114303637537 > 364.70956998184994.
IsChangePoint: Marked as a change because one of 6/13/2022 2:27:15 PM, 6/17/2022 12:17:51 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -100.00953181739742 (T) = (0 -393.33795439185604) / Math.Sqrt((1.9761210529984539 / (41)) + (1.2286270336364793 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.13670058076159458 = (346.0347967169312 - 393.33795439185604) / 346.0347967169312 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 1, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 119.22886001753217 > 86.75461637329501.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -100.9259317793484 (T) = (0 -119.76002922252118) / Math.Sqrt((0.38765229200698764 / (38)) + (0.6186299579179563 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.4459390844796844 = (82.82508613813172 - 119.76002922252118) / 82.82508613813172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 230.8434642050911 > 149.47455941446358.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -206.26564455264403 (T) = (0 -231.91278248251362) / Math.Sqrt((0.26833838551935124 / (42)) + (1.0874417319290566 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.6267007283559269 = (142.56634821630843 - 231.91278248251362) / 142.56634821630843 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 6, Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 107.88018380057888 > 56.28189380091076.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -303.19409328063597 (T) = (0 -108.28344996772911) / Math.Sqrt((0.5088720833884399 / (43)) + (0.14177819472029451 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -1.0063007075012684 = (53.97169505192962 - 108.28344996772911) / 53.97169505192962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Common

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Date_IsMatch - Duration of single invocation 75.84 ns 282.64 ns 3.73 0.01 False
Email_IsNotMatch - Duration of single invocation 610.87 ns 718.95 ns 1.18 0.01 False
Backtracking - Duration of single invocation 124.48 ns 204.43 ns 1.64 0.05 False
IP_IsNotMatch - Duration of single invocation 617.86 ns 862.96 ns 1.40 0.01 False
MatchesWord - Duration of single invocation 1.30 μs 1.71 μs 1.32 0.01 False
IP_IsMatch - Duration of single invocation 136.12 ns 422.31 ns 3.10 0.00 False
Email_IsNotMatch - Duration of single invocation 160.84 ns 243.06 ns 1.51 0.01 False
MatchWord - Duration of single invocation 160.55 ns 305.94 ns 1.91 0.03 False
CtorInvoke - Duration of single invocation 6.55 μs 7.62 μs 1.16 0.01 False
Uri_IsNotMatch - Duration of single invocation 335.96 ns 452.96 ns 1.35 0.06 False
Email_IsMatch - Duration of single invocation 143.44 ns 216.97 ns 1.51 0.00 False
ReplaceWords - Duration of single invocation 3.63 μs 7.94 μs 2.19 0.00 False
MatchesWord - Duration of single invocation 1.03 μs 1.22 μs 1.18 0.01 False
Date_IsNotMatch - Duration of single invocation 126.14 ns 532.69 ns 4.22 0.01 False
Email_IsNotMatch - Duration of single invocation 146.56 ns 232.56 ns 1.59 0.01 False
OneNodeBacktracking - Duration of single invocation 99.96 ns 237.89 ns 2.38 0.01 False
MatchesWords - Duration of single invocation 3.69 μs 7.55 μs 2.05 0.01 False
MatchesWords - Duration of single invocation 3.23 μs 6.21 μs 1.92 0.00 False
MatchesSet - Duration of single invocation 50.75 μs 228.72 μs 4.51 0.01 False
OneNodeBacktracking - Duration of single invocation 92.31 ns 234.26 ns 2.54 0.01 False
MatchWord - Duration of single invocation 165.87 ns 361.26 ns 2.18 0.08 False
Date_IsMatch - Duration of single invocation 76.02 ns 281.70 ns 3.71 0.01 False
MatchesSet - Duration of single invocation 138.83 μs 189.91 μs 1.37 0.00 False
Date_IsMatch - Duration of single invocation 174.29 ns 240.90 ns 1.38 0.01 False
SplitWords - Duration of single invocation 3.60 μs 7.16 μs 1.99 0.01 False
Uri_IsNotMatch - Duration of single invocation 109.77 ns 332.29 ns 3.03 0.01 False
Uri_IsNotMatch - Duration of single invocation 109.46 ns 328.36 ns 3.00 0.01 False
MatchesWord - Duration of single invocation 1.95 μs 3.57 μs 1.83 0.01 False
MatchesWords - Duration of single invocation 57.81 μs 91.49 μs 1.58 0.09 False
MatchWord - Duration of single invocation 1.68 μs 2.39 μs 1.42 0.01 False
ReplaceWords - Duration of single invocation 3.22 μs 6.50 μs 2.02 0.00 False
Ctor - Duration of single invocation 6.33 μs 7.31 μs 1.15 0.01 False
Date_IsNotMatch - Duration of single invocation 125.07 ns 531.83 ns 4.25 0.00 False
Uri_IsMatch - Duration of single invocation 330.40 ns 387.85 ns 1.17 0.03 False
Email_IsMatch - Duration of single invocation 380.32 ns 416.22 ns 1.09 0.01 False
MatchesBoundary - Duration of single invocation 54.37 μs 200.41 μs 3.69 0.00 False
Email_IsMatch - Duration of single invocation 131.18 ns 209.38 ns 1.60 0.01 False
MatchesBoundary - Duration of single invocation 54.26 μs 193.99 μs 3.58 0.01 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Common*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 282.6398839968325 > 79.62235526416029.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -275.8469626960446 (T) = (0 -283.7917837375439) / Math.Sqrt((0.10377615537255488 / (48)) + (3.9633553389686136 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -2.742349745848124 = (75.83251246156003 - 283.7917837375439) / 75.83251246156003 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 718.9496419984678 > 640.9827750214387.
IsChangePoint: Marked as a change because one of 5/4/2022 3:27:51 PM, 5/30/2022 12:57:05 PM, 6/26/2022 6:11:06 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -20.638294038535445 (T) = (0 -715.8389427408739) / Math.Sqrt((453.97134554508983 / (41)) + (4.027068451179365 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.10913504893019703 = (645.4028690476672 - 715.8389427408739) / 645.4028690476672 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Backtracking(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 204.43106065837418 > 129.908299797792.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -121.66392443251344 (T) = (0 -202.5612892973224) / Math.Sqrt((6.966843269560269 / (38)) + (1.2079258181694577 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.6434554841964313 = (123.25328628926319 - 202.5612892973224) / 123.25328628926319 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsNotMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 862.9553594807916 > 646.4880436725928.
IsChangePoint: Marked as a change because one of 5/21/2022 4:54:11 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -135.98005180217913 (T) = (0 -859.5265452743187) / Math.Sqrt((1.8364505296605922 / (44)) + (22.174881637760397 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.3955294996259547 = (615.9142787771226 - 859.5265452743187) / 615.9142787771226 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.705527317727219 > 1.3493871663170525.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -54.80708446341327 (T) = (0 -1710.369149838289) / Math.Sqrt((222.63154837114425 / (42)) + (291.25450966734417 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.34309777412515785 = (1273.450960003532 - 1710.369149838289) / 1273.450960003532 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.IP_IsMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 422.30883164066074 > 142.94750191828706.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -298.04399492553324 (T) = (0 -424.799296761417) / Math.Sqrt((0.07828798094819708 / (46)) + (6.563072179731889 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -2.1247868071127636 = (135.94504936927922 - 424.799296761417) / 135.94504936927922 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 243.0587607051998 > 168.95246779963156.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -191.111183550226 (T) = (0 -243.31705906724468) / Math.Sqrt((0.2887882569596546 / (40)) + (1.0730157706245476 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.512386695419111 = (160.88283492854777 - 243.31705906724468) / 160.88283492854777 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 305.9381554010023 > 163.96114709846137.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -372.6349172353619 (T) = (0 -306.680703341632) / Math.Sqrt((6.435027843945972 / (48)) + (0.20487025687847416 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.9648501006688059 = (156.08351152957798 - 306.680703341632) / 156.08351152957798 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.CtorInvoke(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.621142590791179 > 6.846332044652745.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -44.81490251601888 (T) = (0 -7600.535238051467) / Math.Sqrt((4220.497182597214 / (38)) + (2189.135578133375 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.16028236845141605 = (6550.590998116783 - 7600.535238051467) / 6550.590998116783 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 452.9647501461133 > 341.11916637520176.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -114.1002994311206 (T) = (0 -453.8981596543563) / Math.Sqrt((17.80440798935109 / (43)) + (6.508752292495073 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.3860512236298085 = (327.47574686719156 - 453.8981596543563) / 327.47574686719156 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 216.97394381318466 > 150.63746236346813.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -180.9756509230384 (T) = (0 -217.4750992355039) / Math.Sqrt((0.5178181994367705 / (44)) + (1.1243665415550657 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.5279239963571425 = (142.33371539029778 - 217.4750992355039) / 142.33371539029778 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.944428489764808 > 3.8239655200190743.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -534.5738130597911 (T) = (0 -7952.027554986795) / Math.Sqrt((1255.370953371525 / (45)) + (224.74748771220575 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -1.1903992916589126 = (3630.4008978035586 - 7952.027554986795) / 3630.4008978035586 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.218096034400382 > 1.0833946589471215.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -29.842584290945606 (T) = (0 -1193.5226133731715) / Math.Sqrt((192.6244007137922 / (42)) + (202.98194927555977 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.1833987944155192 = (1008.5548667156218 - 1193.5226133731715) / 1008.5548667156218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 532.6947997373092 > 131.48429103068185.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -270.5008769359958 (T) = (0 -528.2965191487671) / Math.Sqrt((0.791004179086626 / (43)) + (15.431168297337914 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -3.226343668644826 = (125.0008424700978 - 528.2965191487671) / 125.0008424700978 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsNotMatch(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 232.5598415170471 > 153.96948368899103.
IsChangePoint: Marked as a change because one of 6/16/2022 2:51:40 PM, 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -221.43282844466745 (T) = (0 -233.3616110610257) / Math.Sqrt((1.7394377714560643 / (39)) + (0.9224722784245901 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.6114642815505416 = (144.81339346627436 - 233.3616110610257) / 144.81339346627436 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 237.8911990074795 > 104.9606301771352.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -315.3600169041326 (T) = (0 -237.23046545098742) / Math.Sqrt((0.048595946816455834 / (41)) + (1.3220962933693288 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -1.378300234296176 = (99.74790483977411 - 237.23046545098742) / 99.74790483977411 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.546203524086647 > 3.898973390349601.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -454.16943282832 (T) = (0 -7570.462770560531) / Math.Sqrt((825.9780886477955 / (40)) + (317.5243676638993 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -1.0600389080424397 = (3674.912517916661 - 7570.462770560531) / 3674.912517916661 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.205039995156419 > 3.395438160939545.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -317.6434941819527 (T) = (0 -6199.401261322514) / Math.Sqrt((1147.5213189034828 / (39)) + (452.08979145318233 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.9046875579529837 = (3254.8127042868746 - 6199.401261322514) / 3254.8127042868746 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 228.7192546583851 > 53.606018787833456.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -1634.9501195943071 (T) = (0 -228692.75940967244) / Math.Sqrt((90975.62761627133 / (38)) + (47072.00516682735 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -3.48172981074781 = (51027.78816813889 - 228692.75940967244) / 51027.78816813889 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.OneNodeBacktracking(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 234.25933857496867 > 97.02366632651314.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -354.45027415716316 (T) = (0 -234.78048736830195) / Math.Sqrt((0.5231163479837593 / (38)) + (1.0392541631974341 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -1.5515379760165653 = (92.01528238072271 - 234.78048736830195) / 92.01528238072271 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 361.2562176623256 > 174.74367845812333.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -449.6250386100882 (T) = (0 -360.6593927983367) / Math.Sqrt((6.353818513890346 / (44)) + (0.29064198532768176 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -1.1624165485348197 = (166.78534625657915 - 360.6593927983367) / 166.78534625657915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 281.7038148141279 > 79.84816439305357.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -222.8620029891157 (T) = (0 -283.7872327879794) / Math.Sqrt((0.03449704306154001 / (44)) + (6.06825428992434 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -2.724581988701221 = (76.19304223906676 - 283.7872327879794) / 76.19304223906676 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesSet(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 189.912371031746 > 145.46567961836283.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -52.64752578402791 (T) = (0 -189114.4169657249) / Math.Sqrt((2796192.486960623 / (41)) + (5376947.268392507 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.3415458582194872 = (140967.5381628172 - 189114.4169657249) / 140967.5381628172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 240.90017394916524 > 182.99698001798495.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -125.87644113427272 (T) = (0 -241.74212092664953) / Math.Sqrt((7.363445211362057 / (40)) + (0.7528927001633298 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.4310844028232631 = (168.92233641128178 - 241.74212092664953) / 168.92233641128178 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.SplitWords(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.161517244197449 > 3.69583029996142.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -456.26022075366217 (T) = (0 -7191.958001099386) / Math.Sqrt((920.8335579817096 / (46)) + (308.6284457141323 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -1.0323150951011548 = (3538.8006606039694 - 7191.958001099386) / 3538.8006606039694 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 332.2927924652501 > 115.0976978407707.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -1472.6837976353486 (T) = (0 -332.33624354444) / Math.Sqrt((0.1211047543077601 / (41)) + (0.13998153248129944 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -2.042534623494156 = (109.23006133707464 - 332.33624354444) / 109.23006133707464 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsNotMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 328.3561177178453 > 115.42035916052947.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -139.1613385765933 (T) = (0 -332.3288835117585) / Math.Sqrt((2.123963194217737 / (40)) + (19.984206797580377 / (8))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (8) - 2, .025) and -2.019605996504226 = (110.05703522131464 - 332.3288835117585) / 110.05703522131464 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWord(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5726841071020035 > 2.0441034228262103.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -747.9760360334849 (T) = (0 -3572.8564036238567) / Math.Sqrt((60.928390674176626 / (44)) + (23.278226184128272 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.8326805846560504 = (1949.524883679823 - 3572.8564036238567) / 1949.524883679823 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesWords(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 91.4877225378788 > 60.57383808681837.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -30.979765652844794 (T) = (0 -87178.33273563259) / Math.Sqrt((1239026.4523027851 / (41)) + (5503779.33843921 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.472971842482845 = (59185.335538176056 - 87178.33273563259) / 59185.335538176056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchWord(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.388150565298463 > 1.7599145874692677.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -285.0165182192992 (T) = (0 -2390.9931825244958) / Math.Sqrt((156.9232863277666 / (40)) + (12.996211428526141 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.437677712503235 = (1663.0940034268046 - 2390.9931825244958) / 1663.0940034268046 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.ReplaceWords(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.501325712500743 > 3.363491257117867.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -341.5031841334134 (T) = (0 -6507.901198104455) / Math.Sqrt((1086.2338810849717 / (43)) + (478.8363102914807 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -1.03195084874059 = (3202.7847534491666 - 6507.901198104455) / 3202.7847534491666 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Ctor(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.311083873446479 > 6.632450776751166.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -37.62977378779911 (T) = (0 -7226.685676511544) / Math.Sqrt((3743.8193549493108 / (44)) + (3439.769281124247 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.14288551428513072 = (6323.192993684762 - 7226.685676511544) / 6323.192993684762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Date_IsNotMatch(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 531.8341523726056 > 130.87484825502358.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -102.40089985404592 (T) = (0 -531.7104663187733) / Math.Sqrt((0.8835447403258896 / (45)) + (94.5173423439657 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -3.2526971246956977 = (125.02899941571079 - 531.7104663187733) / 125.02899941571079 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Uri_IsMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 387.8500324335113 > 346.7122746284533.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -124.23810134048402 (T) = (0 -388.519315261328) / Math.Sqrt((3.485808605771298 / (38)) + (0.550900785079043 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.16779987834566912 = (332.69340275297253 - 388.519315261328) / 332.69340275297253 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 416.2198692558847 > 399.3774739023191.
IsChangePoint: Marked as a change because one of 5/31/2022 6:06:22 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -119.28044150680012 (T) = (0 -415.97335037820693) / Math.Sqrt((3.123063564278077 / (42)) + (0.12367469901593822 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.09922299437025492 = (378.424898777266 - 415.97335037820693) / 378.424898777266 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: IgnoreCase, Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 200.40881834215168 > 56.69470733382472.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -128.68889314582327 (T) = (0 -196417.50982972837) / Math.Sqrt((344309.3400694772 / (38)) + (8415643.60409966 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -2.5853103430021114 = (54783.963182741056 - 196417.50982972837) / 54783.963182741056 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.Email_IsMatch(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 209.3755171514399 > 137.6541653581344.
IsChangePoint: Marked as a change because one of 6/16/2022 10:41:45 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -85.6426288031931 (T) = (0 -209.95694175744964) / Math.Sqrt((1.9525574996191812 / (42)) + (4.536661082392039 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.5758807955089036 = (133.23148702351415 - 209.95694175744964) / 133.23148702351415 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Common.MatchesBoundary(Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 193.99318910256406 > 57.03216493318376.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -129.19609343941244 (T) = (0 -196254.26912932194) / Math.Sqrt((1328690.8531713083 / (41)) + (8190572.100970418 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -2.5956945464378243 = (54580.35063733284 - 196254.26912932194) / 54580.35063733284 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Count - Duration of single invocation 167.13 μs 235.56 μs 1.41 0.01 False
Count - Duration of single invocation 2.91 ms 3.88 ms 1.33 0.01 False
Count - Duration of single invocation 386.12 μs 523.52 μs 1.36 0.03 False
Count - Duration of single invocation 17.07 ms 24.50 ms 1.44 0.00 False
Count - Duration of single invocation 254.41 μs 359.17 μs 1.41 0.01 False
Count - Duration of single invocation 65.92 μs 70.39 μs 1.07 0.07 False
Count - Duration of single invocation 80.31 μs 90.03 μs 1.12 0.01 False
Count - Duration of single invocation 62.79 μs 69.94 μs 1.11 0.09 False
Count - Duration of single invocation 151.93 μs 187.80 μs 1.24 0.01 False
Count - Duration of single invocation 30.86 ms 36.68 ms 1.19 0.02 False
Count - Duration of single invocation 32.93 ms 56.04 ms 1.70 0.01 False
Count - Duration of single invocation 962.01 μs 2.12 ms 2.21 0.00 False
Count - Duration of single invocation 113.50 μs 163.35 μs 1.44 0.00 False
Count - Duration of single invocation 157.48 μs 230.04 μs 1.46 0.01 False
Count - Duration of single invocation 123.95 μs 149.08 μs 1.20 0.01 False
Count - Duration of single invocation 52.05 μs 67.67 μs 1.30 0.09 False
Count - Duration of single invocation 100.67 μs 172.94 μs 1.72 0.00 False
Count - Duration of single invocation 241.40 μs 311.02 μs 1.29 0.01 False
Count - Duration of single invocation 4.31 ms 9.92 ms 2.30 0.00 False
Count - Duration of single invocation 119.74 μs 155.34 μs 1.30 0.01 False
Count - Duration of single invocation 31.69 ms 46.24 ms 1.46 0.02 False
Count - Duration of single invocation 11.76 ms 15.37 ms 1.31 0.00 False
Count - Duration of single invocation 11.85 ms 15.78 ms 1.33 0.03 False
Count - Duration of single invocation 2.65 ms 3.03 ms 1.14 0.01 False
Count - Duration of single invocation 6.24 ms 10.17 ms 1.63 0.05 False
Count - Duration of single invocation 86.96 ns 120.59 ns 1.39 0.00 False
Count - Duration of single invocation 1.52 ms 1.62 ms 1.07 0.00 False
Count - Duration of single invocation 4.18 ms 18.02 ms 4.32 0.00 False
Count - Duration of single invocation 127.16 μs 162.47 μs 1.28 0.01 False
Count - Duration of single invocation 697.92 μs 1.30 ms 1.86 0.01 False
Count - Duration of single invocation 8.73 ms 13.03 ms 1.49 0.17 False
Count - Duration of single invocation 1.59 ms 2.01 ms 1.26 0.01 False
Count - Duration of single invocation 2.67 ms 3.37 ms 1.26 0.01 False
Count - Duration of single invocation 155.11 μs 216.78 μs 1.40 0.01 False
Count - Duration of single invocation 1.33 ms 1.67 ms 1.26 0.01 False
Count - Duration of single invocation 11.50 ms 37.62 ms 3.27 0.00 False
Count - Duration of single invocation 555.92 μs 884.16 μs 1.59 0.00 False
Count - Duration of single invocation 1.32 ms 1.67 ms 1.26 0.00 False
Count - Duration of single invocation 70.80 μs 79.80 μs 1.13 0.28 False
Count - Duration of single invocation 1.30 ms 2.56 ms 1.97 0.00 False
Count - Duration of single invocation 3.10 ms 5.93 ms 1.92 0.01 False
Count - Duration of single invocation 92.92 μs 109.59 μs 1.18 0.02 False
Count - Duration of single invocation 128.84 μs 164.07 μs 1.27 0.01 False
Count - Duration of single invocation 4.18 ms 17.75 ms 4.25 0.02 False
Count - Duration of single invocation 179.73 μs 227.49 μs 1.27 0.04 False
Count - Duration of single invocation 211.36 μs 264.60 μs 1.25 0.01 False
Count - Duration of single invocation 2.31 ms 5.67 ms 2.45 0.00 False
Count - Duration of single invocation 162.13 μs 231.82 μs 1.43 0.02 False
Count - Duration of single invocation 81.87 μs 94.88 μs 1.16 0.04 False
Count - Duration of single invocation 1.57 ms 1.94 ms 1.23 0.01 False
Count - Duration of single invocation 661.58 μs 1.53 ms 2.31 0.01 False
Count - Duration of single invocation 1.59 ms 2.33 ms 1.46 0.01 False
Count - Duration of single invocation 716.99 μs 1.32 ms 1.84 0.01 False
Count - Duration of single invocation 128.49 μs 163.46 μs 1.27 0.02 False
Count - Duration of single invocation 62.96 μs 68.59 μs 1.09 0.11 False
Count - Duration of single invocation 45.88 ms 63.27 ms 1.38 0.01 False
Count - Duration of single invocation 749.87 μs 1.08 ms 1.45 0.03 False
Count - Duration of single invocation 12.08 ms 15.98 ms 1.32 0.04 False
Count - Duration of single invocation 1.95 ms 3.23 ms 1.65 0.05 False
Count - Duration of single invocation 1.97 ms 3.22 ms 1.63 0.01 False
Count - Duration of single invocation 5.64 ms 8.33 ms 1.48 0.00 False
Count - Duration of single invocation 9.68 ms 12.37 ms 1.28 0.02 False
Count - Duration of single invocation 1.02 ms 1.45 ms 1.42 0.00 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: NonBacktracking)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 235.55503654273915 > 172.89764104957183.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -81.62059980046018 (T) = (0 -235913.4614356351) / Math.Sqrt((4050065.530058328 / (40)) + (4133337.0762453377 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.4441234719154096 = (163361.00480572612 - 235913.4614356351) / 163361.00480572612 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.8821505208333336 > 3.057075122395833.
IsChangePoint: Marked as a change because one of 6/16/2022 2:51:40 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -230.60650371528985 (T) = (0 -3893924.7916666665) / Math.Sqrt((7725736.545725542 / (38)) + (90190070.35590169 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.33857497971259154 = (2909007.601877289 - 3893924.7916666665) / 2909007.601877289 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 523.521611111111 > 405.97457164634136.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -76.75925899110501 (T) = (0 -528148.140376984) / Math.Sqrt((21331835.310235016 / (41)) + (19072166.62091643 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.35465192138458707 = (389877.378859925 - 528148.140376984) / 389877.378859925 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{L}", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.49920777777778 > 17.9153295.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -175.7458938086713 (T) = (0 -24486269.841269847) / Math.Sqrt((7136292093.77751 / (43)) + (11304629310.464075 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.4344913712593562 = (17069652.931946933 - 24486269.841269847) / 17069652.931946933 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 359.17117424242423 > 268.9394075307377.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -91.67032699707359 (T) = (0 -356100.2695286196) / Math.Sqrt((9397986.656496782 / (38)) + (4587215.495275016 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.384709733304458 = (257166.0045162139 - 356100.2695286196) / 257166.0045162139 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?m)^Sherlock Holmes|Sherlock Holmes$", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.39173895474137 > 69.64896750811906.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -9.715833675959152 (T) = (0 -71527.32483251255) / Math.Sqrt((2105019.953496574 / (42)) + (1324304.2783634656 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.0825254609998815 = (66074.49654481649 - 71527.32483251255) / 66074.49654481649 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 90.03096520935961 > 84.43195095738675.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -69.25558085850194 (T) = (0 -89791.10704199199) / Math.Sqrt((230302.948070559 / (41)) + (99695.1479315842 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.12194849273591557 = (80031.3986099601 - 89791.10704199199) / 80031.3986099601 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.94422199170126 > 66.15190543177975.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -9.734339752540663 (T) = (0 -69529.81423055528) / Math.Sqrt((1891989.6984574664 / (42)) + (1743726.6257562267 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.09631759872393146 = (63421.23332826648 - 69529.81423055528) / 63421.23332826648 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 187.7982740348221 > 159.952645143629.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -36.64105831659693 (T) = (0 -188326.5814063306) / Math.Sqrt((3964954.171585462 / (40)) + (7470876.465511396 / (8))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (8) - 2, .025) and -0.24648473219962436 = (151086.15175253517 - 188326.5814063306) / 151086.15175253517 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.67814666666666 > 32.538120125000006.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -57.81474825396659 (T) = (0 -36784090.71428571) / Math.Sqrt((63292817716.79443 / (41)) + (61158339237.831024 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.1895746380477827 = (30922053.596109174 - 36784090.71428571) / 30922053.596109174 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\b\w+n\b", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 56.04298 > 34.671112875000006.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -302.2549086144732 (T) = (0 -56171827.38095238) / Math.Sqrt((166909782898.6278 / (41)) + (11269735238.756628 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.6822807008520404 = (33390282.223711245 - 56171827.38095238) / 33390282.223711245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1234840625 > 1004.7667656250002.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -769.3994433976819 (T) = (0 -2118909.5070684524) / Math.Sqrt((48722779.47372689 / (42)) + (6582667.562821933 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -1.2003875812181044 = (962971.0352643661 - 2118909.5070684524) / 962971.0352643661 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 163.3467729591837 > 119.56601247785096.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -237.28157114638736 (T) = (0 -163291.59877252122) / Math.Sqrt((161688.24378025442 / (41)) + (281320.11834573717 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.439399159148781 = (113444.27828420237 - 163291.59877252122) / 113444.27828420237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 230.04025735294115 > 164.82424816735315.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -193.231071871728 (T) = (0 -230741.17021822665) / Math.Sqrt((962237.5877280239 / (41)) + (874272.8829264282 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.47615776188844683 = (156311.99874127257 - 230741.17021822665) / 156311.99874127257 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 149.08286276483778 > 128.8988108097392.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -26.971419532031387 (T) = (0 -147752.6182363057) / Math.Sqrt((2396300.1245996077 / (38)) + (5390787.209252671 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -0.19993781937625774 = (123133.56229834417 - 147752.6182363057) / 123133.56229834417 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 67.67336291152264 > 55.13810334668844.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -16.055970144797943 (T) = (0 -64784.5708353629) / Math.Sqrt((1305465.8965675924 / (48)) + (3175741.429014012 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.20752879219250647 = (53650.539228744805 - 64784.5708353629) / 53650.539228744805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 172.93751358695653 > 105.96650557074472.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -451.14735343126205 (T) = (0 -173119.70365330265) / Math.Sqrt((360406.95238602126 / (41)) + (114501.47550764486 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.7043275968677682 = (101576.5419579334 - 173119.70365330265) / 101576.5419579334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 311.016913580247 > 253.38418269813985.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -63.24138135540334 (T) = (0 -311966.37265401945) / Math.Sqrt((8672387.68184039 / (40)) + (6525772.733805215 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.3012852516966428 = (239737.11547661913 - 311966.37265401945) / 239737.11547661913 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.917140625 > 4.525439078125.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -296.1067067853468 (T) = (0 -9886988.443550497) / Math.Sqrt((311001424.1861532 / (43)) + (2406469750.430569 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -1.2784646362036616 = (4339320.56107047 - 9886988.443550497) / 4339320.56107047 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 155.3369348184819 > 125.55183821225073.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -57.78718572392048 (T) = (0 -154741.5049540035) / Math.Sqrt((1596848.0833744523 / (43)) + (2869893.063205525 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.3071290863216381 = (118382.72636825641 - 154741.5049540035) / 118382.72636825641 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Ll}", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 46.24337666666666 > 33.2786685.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -332.990600155162 (T) = (0 -46295132.38095238) / Math.Sqrt((36555722020.06696 / (44)) + (8136616797.089944 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.47301946118253524 = (31428730.984848496 - 46295132.38095238) / 31428730.984848496 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.36708125 > 12.351585124999998.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -55.0168076505917 (T) = (0 -15421228.929563493) / Math.Sqrt((23309903448.95489 / (38)) + (21074016175.993 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.3296038444411903 = (11598363.673538242 - 15421228.929563493) / 11598363.673538242 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.77966 > 12.569804718750003.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -85.25762839883804 (T) = (0 -15840442.686011905) / Math.Sqrt((23846528093.50936 / (43)) + (11942872582.88736 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.32194707044782767 = (11982660.304731973 - 15840442.686011905) / 11982660.304731973 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.026894779116466 > 2.757307780739082.
IsChangePoint: Marked as a change because one of 5/9/2022 5:23:46 PM, 5/19/2022 10:26:24 PM, 5/30/2022 12:57:05 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -112.21757644380979 (T) = (0 -3031693.725034132) / Math.Sqrt((428095491.9055238 / (43)) + (28995957.386065155 / (8))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (8) - 2, .025) and -0.15794952856458655 = (2618157.05283137 - 3031693.725034132) / 2618157.05283137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-zA-Z]+ing", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.173646933333334 > 6.5497861625.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -57.157358934575775 (T) = (0 -9995278.612698413) / Math.Sqrt((1576749428.6883528 / (45)) + (25641473687.46781 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -0.6009182441571951 = (6243465.991581873 - 9995278.612698413) / 6243465.991581873 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 120.58693927910463 > 90.86676546784805.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -138.87804823933888 (T) = (0 -120.20266787940102) / Math.Sqrt((0.130242305516926 / (48)) + (0.3853087254327452 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.3844022621821941 = (86.8264023853363 - 120.20266787940102) / 86.8264023853363 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?s).*", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6182524675324677 > 1.5920932600036513.
IsChangePoint: Marked as a change because one of 5/19/2022 10:26:24 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -68.32912352669159 (T) = (0 -1619772.692949662) / Math.Sqrt((3282122.0187344216 / (40)) + (11023866.993610363 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.06813906651172602 = (1516443.6389724354 - 1619772.692949662) / 1516443.6389724354 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.018793956043957 > 4.38723636328125.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -209.2798336045336 (T) = (0 -17904042.671980035) / Math.Sqrt((1074738689.2566993 / (41)) + (30033667142.31276 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -3.3101100197746383 = (4153964.188811166 - 17904042.671980035) / 4153964.188811166 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 162.4700987972509 > 133.35977898341974.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 6/22/2022 6:53:40 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -117.12676474458159 (T) = (0 -161741.51399346467) / Math.Sqrt((966396.9388281758 / (43)) + (474020.8481549272 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.27792129424392936 = (126566.09974494367 - 161741.51399346467) / 126566.09974494367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3009349107142856 > 742.3016363636364.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -214.86835422320226 (T) = (0 -1303341.946031746) / Math.Sqrt((113786915.16339143 / (40)) + (23773868.230160706 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.8330792163980214 = (711012.3416230736 - 1303341.946031746) / 711012.3416230736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\b\w+n\b", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.03138484848485 > 9.226766125000001.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -31.329442859372268 (T) = (0 -12790199.297202796) / Math.Sqrt((177216091114.2523 / (40)) + (68725979720.90474 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.446592638355301 = (8841604.027340118 - 12790199.297202796) / 8841604.027340118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.012625714285714 > 1.671630640699004.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -57.64132570023221 (T) = (0 -1982858.7954417595) / Math.Sqrt((261693853.40772805 / (38)) + (278873073.2730822 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -0.24798784423780704 = (1588844.63866935 - 1982858.7954417595) / 1588844.63866935 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{Lu}", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3662394594594596 > 2.8089442782258067.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/19/2022 10:26:24 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -63.55294369576659 (T) = (0 -3327338.673616474) / Math.Sqrt((295534394.4486412 / (46)) + (657643785.0079411 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -0.23664264226452583 = (2690622.626051039 - 3327338.673616474) / 2690622.626051039 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 216.7838020833333 > 162.80618236386138.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -178.014422698646 (T) = (0 -216367.3393229488) / Math.Sqrt((1276183.5930157239 / (39)) + (711054.5647109341 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.4023378924705485 = (154290.44632158286 - 216367.3393229488) / 154290.44632158286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6678657142857143 > 1.3965467578125002.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -293.9964409137024 (T) = (0 -1666987.424107143) / Math.Sqrt((32307542.31723395 / (39)) + (3715628.2480666 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.25083214898717077 = (1332702.7335018078 - 1666987.424107143) / 1332702.7335018078 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\b\w+n\b", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.61842884615385 > 12.106360116883117.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -178.16379484075205 (T) = (0 -38067396.21489622) / Math.Sqrt((9115922319.529743 / (39)) + (176793885060.41205 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -2.3269288968145037 = (11442203.123530928 - 38067396.21489622) / 11442203.123530928 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 884.1597685185185 > 581.1643868534484.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -572.7950168178228 (T) = (0 -884163.047760771) / Math.Sqrt((9508501.10133951 / (48)) + (966165.9113836939 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.6015082691056469 = (552081.4752049496 - 884163.047760771) / 552081.4752049496 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6745215416666668 > 1.4033461432291665.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -139.21777655793124 (T) = (0 -1675879.4841269841) / Math.Sqrt((70340824.4640076 / (42)) + (24814032.604860116 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.25037950431077766 = (1340296.6686108203 - 1675879.4841269841) / 1340296.6686108203 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 79.79539447731754 > 75.19066442390296.
IsChangePoint: Marked as a change because one of 6/7/2022 9:00:42 AM, 7/3/2022 3:44:27 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -5.625737511997669 (T) = (0 -80363.0517817523) / Math.Sqrt((5648029.531552367 / (45)) + (6938303.916089599 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -0.08608192807373821 = (73993.5447819146 - 80363.0517817523) / 73993.5447819146 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5612055555555555 > 1.3640666609051164.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -294.490128702942 (T) = (0 -2571026.6948892474) / Math.Sqrt((68793583.39481014 / (44)) + (119420677.05227506 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.9774801981988434 = (1300152.9407126436 - 2571026.6948892474) / 1300152.9407126436 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.9329979853479875 > 3.2461295458333335.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -66.59058918243272 (T) = (0 -5966378.57895081) / Math.Sqrt((444569583.63678694 / (40)) + (14849466222.983795 / (8))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (8) - 2, .025) and -0.9315856119283717 = (3088850.1871756846 - 5966378.57895081) / 3088850.1871756846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "The", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.59235328638498 > 97.67561056886227.
IsChangePoint: Marked as a change because one of 5/30/2022 12:57:05 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -70.1128828033909 (T) = (0 -108841.48395023892) / Math.Sqrt((955402.5288655473 / (41)) + (207649.62882558713 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.17405824836351644 = (92705.35265345626 - 108841.48395023892) / 92705.35265345626 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 164.07158854166667 > 134.54709039256196.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 6/22/2022 6:53:40 PM, 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -177.62615212690545 (T) = (0 -163596.5962077124) / Math.Sqrt((1007095.9089181017 / (39)) + (108976.67919864462 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.2749244685522157 = (128318.65749152194 - 163596.5962077124) / 128318.65749152194 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes\s+\w+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.751574871794872 > 4.391098214102564.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -136.06640323370428 (T) = (0 -17825205.89010989) / Math.Sqrt((426144586.2304623 / (38)) + (50432495021.00646 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -3.292903207849737 = (4152249.661142097 - 17825205.89010989) / 4152249.661142097 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 227.49488685015294 > 188.50006726483534.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -16.660709215894286 (T) = (0 -232475.00820720266) / Math.Sqrt((11659241.023731004 / (38)) + (68186071.09164445 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -0.29394504708220687 = (179663.74130912614 - 232475.00820720266) / 179663.74130912614 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 264.6007771836007 > 219.3273892740566.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 6/7/2022 9:37:06 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -98.08156999721176 (T) = (0 -265063.19792911765) / Math.Sqrt((5035543.685192223 / (38)) + (1277376.0094344842 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -0.26211344479164056 = (210015.35085689253 - 265063.19792911765) / 210015.35085689253 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Holmes|Watson|Irene|Adler|John|Baker", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.669996180555556 > 2.4260168437500003.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -1376.1146019289251 (T) = (0 -5658099.610260772) / Math.Sqrt((7386742.421997565 / (41)) + (40188002.47306246 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -1.4499256456902312 = (2309498.5026236107 - 5658099.610260772) / 2309498.5026236107 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 231.82070741758244 > 173.44398811008446.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -11.174174469147497 (T) = (0 -239991.4539067332) / Math.Sqrt((6629146.877125808 / (45)) + (271152381.1991618 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -0.4566947278942632 = (164750.68475991176 - 239991.4539067332) / 164750.68475991176 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[a-q][^u-z]{13}x", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 94.87862373737373 > 85.55936866179779.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -20.48370978087728 (T) = (0 -95496.16856665301) / Math.Sqrt((2112372.284488515 / (40)) + (2674616.0056721983 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.19680488609450497 = (79792.59583262782 - 95496.16856665301) / 79792.59583262782 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9362279575892858 > 1.651340184375.
IsChangePoint: Marked as a change because one of 5/19/2022 10:26:24 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -272.7602333435654 (T) = (0 -1940021.7777423467) / Math.Sqrt((3144496.962760785 / (48)) + (12271586.146479087 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.23396119978730248 = (1572190.2585565476 - 1940021.7777423467) / 1572190.2585565476 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.5309963425925928 > 696.4209765625001.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -263.1094775434416 (T) = (0 -1529854.4815138564) / Math.Sqrt((17047655.15139979 / (41)) + (73847896.28868604 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -1.3229283448996294 = (658588.752801137 - 1529854.4815138564) / 658588.752801137 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3256610416666663 > 1.6663310562500002.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -164.69530602148862 (T) = (0 -2325113.5612899004) / Math.Sqrt((193519355.62536845 / (41)) + (108170234.81996246 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.4665809298073292 = (1585397.3783740392 - 2325113.5612899004) / 1585397.3783740392 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3215870899470898 > 747.5192538579303.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -16.204519094624914 (T) = (0 -1356112.9197027166) / Math.Sqrt((85803396.9010654 / (44)) + (10701319894.933668 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.877953810953975 = (722122.6165375334 - 1356112.9197027166) / 722122.6165375334 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock|Street", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 163.4552938596491 > 134.15020146163.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -84.88678832724919 (T) = (0 -163243.16307388246) / Math.Sqrt((6199773.197848354 / (44)) + (212373.12050886604 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.2741852029845448 = (128115.72657688642 - 163243.16307388246) / 128115.72657688642 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.58932139121337 > 65.80516089722262.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -13.581532648752463 (T) = (0 -68969.36207175888) / Math.Sqrt((1621130.4533416114 / (41)) + (1016712.2914515365 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.09247787740425116 = (63131.12924138238 - 68969.36207175888) / 63131.12924138238 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\p{L}", Options: NonBacktracking)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 63.272643333333335 > 48.65404775.
IsChangePoint: Marked as a change because one of 5/27/2022 11:47:05 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -85.08585172825312 (T) = (0 -63158716.38888889) / Math.Sqrt((224940501251.39777 / (45)) + (193450960286.0185 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -0.3513059688120802 = (46739019.76797386 - 63158716.38888889) / 46739019.76797386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.084652833333333 > 789.7353020833334.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -122.48861673282083 (T) = (0 -1083333.5444444444) / Math.Sqrt((17998612.905327372 / (38)) + (34718885.37214429 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.44495255075189954 = (749736.4144453864 - 1083333.5444444444) / 749736.4144453864 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes\s+\w+", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.98119 > 12.6924885625.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -118.50061686450013 (T) = (0 -15968481.84981685) / Math.Sqrt((25745138920.673252 / (41)) + (3588053111.7732463 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.3344268690708733 = (11966547.002261193 - 15968481.84981685) / 11966547.002261193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "[^\n]*", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2269379166666665 > 2.05045030859375.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -465.91572674348464 (T) = (0 -3223263.8452380956) / Math.Sqrt((80241352.07223265 / (41)) + (39312182.18617635 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.6605387399435755 = (1941095.2407816881 - 3223263.8452380956) / 1941095.2407816881 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2166223333333335 > 2.0697593710937503.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -394.4287920959257 (T) = (0 -3214759.175) / Math.Sqrt((256150532.89084652 / (42)) + (18017158.542013347 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.6185679633133965 = (1986174.969396413 - 3214759.175) / 1986174.969396413 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sher[a-z]+|Hol[a-z]+", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.334524444444446 > 5.916914891608391.
IsChangePoint: Marked as a change because one of 5/16/2022 6:32:29 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -77.19162901572068 (T) = (0 -8338815.859829059) / Math.Sqrt((4138605427.124738 / (42)) + (5762064965.182818 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.4867799892247312 = (5608641.440067581 - 8338815.859829059) / 5608641.440067581 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)Sherlock|Holmes|Watson", Options: None)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.365708333333334 > 10.166823173076924.
IsChangePoint: Marked as a change because one of 5/23/2022 3:43:39 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -52.25880810263597 (T) = (0 -12489287.771062272) / Math.Sqrt((6160912634.966605 / (40)) + (13300706829.023827 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.2852993914258451 = (9717026.129769888 - 12489287.771062272) / 9717026.129769888 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "(?i)the", Options: Compiled)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4539754545454548 > 1.0779615615234375.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -97.46856918519288 (T) = (0 -1454203.2575757576) / Math.Sqrt((47701859.87175078 / (38)) + (91461007.24360134 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.4211339497882771 = (1023269.6627875277 - 1454203.2575757576) / 1023269.6627875277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jul 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in Span.Sorting

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BubbleSortArray - Duration of single invocation 210.45 μs 370.63 μs 1.76 0.15 False
QuickSortSpan - Duration of single invocation 10.56 μs 15.94 μs 1.51 0.10 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Span.Sorting*'

Payloads

Baseline
Compare

Histogram

Span.Sorting.BubbleSortArray(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 370.6262333333333 > 220.98006.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/18/2022 10:04:15 AM, 6/2/2022 5:58:32 PM, 6/6/2022 7:24:37 AM, 6/20/2022 2:50:37 PM, 6/21/2022 5:13:43 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -4450.313888940316 (T) = (0 -370633.0293333333) / Math.Sqrt((1242.726865079531 / (38)) + (6312.517946670288 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -0.7609716209678818 = (210470.75655292076 - 370633.0293333333) / 210470.75655292076 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Span.Sorting.QuickSortSpan(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.940666666666665 > 11.063448750000001.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -276.868224164585 (T) = (0 -15901.42) / Math.Sqrt((4213.673759541298 / (44)) + (1952.9266370370053 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.5084385122227278 = (10541.642812187814 - 15901.42) / 10541.642812187814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Diagnostics.Perf_Activity

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ActivityAllocations - Duration of single invocation 305.88 ns 392.17 ns 1.28 0.35 False
ActivityAllocations - Duration of single invocation 220.62 ns 282.05 ns 1.28 0.10 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Diagnostics.Perf_Activity*'

Payloads

Baseline
Compare

Histogram

System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: W3C)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 392.16956322217777 > 329.78009389908675.
IsChangePoint: Marked as a change because one of 5/5/2022 5:56:13 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -10.10135659675694 (T) = (0 -374.9990637510533) / Math.Sqrt((98.45145763892566 / (44)) + (285.5184467990511 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.2146107871623482 = (308.74010647242005 - 374.9990637510533) / 308.74010647242005 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Diagnostics.Perf_Activity.ActivityAllocations(idFormat: Hierarchical)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 282.0478992096036 > 229.10168242281077.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -7.976706264971178 (T) = (0 -290.4189452719451) / Math.Sqrt((21.065130345402654 / (42)) + (493.44679458504663 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.3330504673958484 = (217.86042792459816 - 290.4189452719451) / 217.86042792459816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 6b8d056a665fa20d321ea1d49a93a37a83960663
Compare 59616049ef3f6f95d0e8943db446b5311a9cd141
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Multiply - Duration of single invocation 9.57 ns 25.88 ns 2.70 0.04 False
Parse - Duration of single invocation 170.70 ns 225.04 ns 1.32 0.01 False
Remainder - Duration of single invocation 5.85 ms 6.33 ms 1.08 0.00 False
ToByteArray - Duration of single invocation 14.26 ns 21.47 ns 1.51 0.12 False
Remainder - Duration of single invocation 669.25 ns 888.63 ns 1.33 0.00 False
Subtract - Duration of single invocation 61.83 ns 122.63 ns 1.98 0.01 False
ToByteArray - Duration of single invocation 13.78 ns 19.81 ns 1.44 0.02 False
GreatestCommonDivisor - Duration of single invocation 11.10 μs 14.33 μs 1.29 0.00 False
Ctor_ByteArray - Duration of single invocation 126.40 ns 133.68 ns 1.06 0.00 False
Ctor_ByteArray - Duration of single invocation 12.91 ns 21.29 ns 1.65 0.08 False
ModPow - Duration of single invocation 188.56 ns 207.57 ns 1.10 0.00 False
Parse - Duration of single invocation 112.04 ns 152.98 ns 1.37 0.03 False
Divide - Duration of single invocation 5.83 ms 6.33 ms 1.09 0.00 False
Subtract - Duration of single invocation 10.89 ns 25.01 ns 2.30 0.05 False
ToStringX - Duration of single invocation 409.26 ns 490.29 ns 1.20 0.06 False
Ctor_ByteArray - Duration of single invocation 8.64 ns 16.67 ns 1.93 0.08 False
ModPow - Duration of single invocation 171.88 μs 324.55 μs 1.89 0.01 False
ToStringX - Duration of single invocation 73.05 ns 124.64 ns 1.71 0.16 False
Divide - Duration of single invocation 9.28 ns 16.16 ns 1.74 0.02 False
Divide - Duration of single invocation 674.52 ns 891.53 ns 1.32 0.00 False
GreatestCommonDivisor - Duration of single invocation 5.28 ms 7.84 ms 1.49 0.00 False
Remainder - Duration of single invocation 9.04 ns 14.74 ns 1.63 0.03 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Multiply(arguments: 16,16 bits)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.883521119584547 > 10.06085914901361.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -123.00883589175996 (T) = (0 -25.992973301838994) / Math.Sqrt((0.026721366086589777 / (38)) + (0.08508570569445775 / (5))) is less than -2.019540970439573 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (5) - 2, .025) and -1.7024459030396204 = (9.618314014205787 - 25.992973301838994) / 9.618314014205787 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: -2147483648)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 225.04295009513817 > 179.50742283258955.
IsChangePoint: Marked as a change because one of 5/18/2022 10:04:15 AM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -44.25720705009699 (T) = (0 -227.6338933812416) / Math.Sqrt((5.075884177381712 / (45)) + (9.477566987527009 / (6))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (45) + (6) - 2, .025) and -0.33855973479390455 = (170.05882327417382 - 227.6338933812416) / 170.05882327417382 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 65536,32768 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.334582564102563 > 6.1414091395348835.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -44.576083406800194 (T) = (0 -6305707.694017095) / Math.Sqrt((820888275.4980255 / (40)) + (409620842.16413724 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.07706521057422899 = (5854527.313768918 - 6305707.694017095) / 5854527.313768918 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: -2147483648)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.47123331517534 > 15.041591407987406.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -163.98564153439384 (T) = (0 -21.438799198759206) / Math.Sqrt((0.07352874090411553 / (44)) + (0.00045954361784016334 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.4679289429897059 = (14.604793577470561 - 21.438799198759206) / 14.604793577470561 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 1024,512 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 888.6329226344873 > 704.7394789757419.
IsChangePoint: Marked as a change because one of 5/19/2022 10:26:24 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -55.20122185520757 (T) = (0 -883.5102833620449) / Math.Sqrt((4.200441648723003 / (40)) + (87.97084482218335 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.3159515120604505 = (671.385134835773 - 883.5102833620449) / 671.385134835773 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 1024,1024 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 122.63316052447469 > 64.17190502626556.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -378.51456975973036 (T) = (0 -123.07766698086726) / Math.Sqrt((0.60293615766456 / (40)) + (0.09315036649500512 / (8))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (8) - 2, .025) and -1.0108115024533018 = (61.20795849372538 - 123.07766698086726) / 61.20795849372538 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ToByteArray(numberString: 123)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.813782520836124 > 14.50247553431271.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -103.12343879034043 (T) = (0 -19.822564311838132) / Math.Sqrt((0.037571495528921715 / (48)) + (0.017858523171599607 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.4293566028791816 = (13.86817276522118 - 19.822564311838132) / 13.86817276522118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.GreatestCommonDivisor(arguments: 1024,1024 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.328880158730161 > 11.647295966312058.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -180.4550831946057 (T) = (0 -14353.751330848992) / Math.Sqrt((117.06601251806734 / (38)) + (2260.988669256459 / (7))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (38) + (7) - 2, .025) and -0.2936954442466279 = (11095.154887252287 - 14353.751330848992) / 11095.154887252287 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 133.6839544655768 > 132.97780579885395.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -47.46790209108131 (T) = (0 -133.91000269274974) / Math.Sqrt((0.3882282777846115 / (40)) + (0.07222089892481501 / (5))) is less than -2.016692199226234 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (5) - 2, .025) and -0.05829784198200873 = (126.53337971658289 - 133.91000269274974) / 126.53337971658289 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: -2147483648)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.290427550792586 > 13.553001920689782.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -146.17785992527263 (T) = (0 -21.284792958203894) / Math.Sqrt((0.1296787091388161 / (40)) + (0.000852288028276873 / (6))) is less than -2.0153675744421933 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (40) + (6) - 2, .025) and -0.6653084562753813 = (12.78129158474901 - 21.284792958203894) / 12.78129158474901 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 16,16,16 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 207.5695079527319 > 197.94527127876688.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -533.049570974244 (T) = (0 -207.65107434535776) / Math.Sqrt((0.045900130995477524 / (48)) + (0.0023236545432902176 / (7))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (48) + (7) - 2, .025) and -0.1014856084376943 = (188.51909889216094 - 207.65107434535776) / 188.51909889216094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Parse(numberString: 123)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 152.9831756567107 > 118.77453706861638.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -27.85452304155045 (T) = (0 -153.56985489028756) / Math.Sqrt((3.384450328223309 / (43)) + (13.977679156203765 / (7))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (7) - 2, .025) and -0.3537446827808612 = (113.4407815917138 - 153.56985489028756) / 113.4407815917138 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 65536,32768 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.332497264957265 > 6.1612663740310065.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -41.102326399658644 (T) = (0 -6330845.141206779) / Math.Sqrt((1035047804.4832227 / (41)) + (705347264.8658751 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -0.0786084535379935 = (5869456.261390016 - 6330845.141206779) / 5869456.261390016 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Subtract(arguments: 16,16 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.01388881583408 > 11.429524535878597.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -269.0057712626292 (T) = (0 -25.032693112846637) / Math.Sqrt((0.045993540249135455 / (41)) + (0.011240652894917665 / (7))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .025) and -1.279122437312254 = (10.983478861437316 - 25.032693112846637) / 10.983478861437316 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: 12345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 490.2850704195723 > 433.30667393244676.
IsChangePoint: Marked as a change because one of 6/2/2022 5:58:32 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -24.00089662878037 (T) = (0 -482.5317765667605) / Math.Sqrt((122.16849505366969 / (44)) + (24.534678160816863 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.14241566061192953 = (422.37846801600625 - 482.5317765667605) / 422.37846801600625 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Ctor_ByteArray(numberString: 123)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.67483935328024 > 9.077256154586326.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -267.12923119071024 (T) = (0 -16.625123624216616) / Math.Sqrt((0.02851773930902598 / (39)) + (0.001119647116636337 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.9020570762387387 = (8.740601863058865 - 16.625123624216616) / 8.740601863058865 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ModPow(arguments: 1024,1024,64 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 324.551768707483 > 180.4678039428303.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -60.0658936314643 (T) = (0 -325187.78117624606) / Math.Sqrt((1184058.3778510476 / (44)) + (46031950.88957127 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.9034442457492102 = (170841.7684954306 - 325187.78117624606) / 170841.7684954306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.ToStringX(numberString: -2147483648)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 124.63699706116067 > 75.63740021702216.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -109.72496413610074 (T) = (0 -123.45741885332438) / Math.Sqrt((2.385550850149438 / (46)) + (1.198120638558178 / (7))) is less than -2.007583770314729 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (7) - 2, .025) and -0.7233071136124724 = (71.63982431113367 - 123.45741885332438) / 71.63982431113367 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 16,8 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.155719327542684 > 9.710909537224191.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -128.8725906541274 (T) = (0 -16.118356003003814) / Math.Sqrt((0.015264711603234326 / (42)) + (0.014828451811200485 / (6))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (6) - 2, .025) and -0.7412547498456742 = (9.25674775872535 - 16.118356003003814) / 9.25674775872535 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 1024,512 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 891.5264766679861 > 707.419881870757.
IsChangePoint: Marked as a change because one of 5/19/2022 10:26:24 PM, 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -368.6500529477386 (T) = (0 -890.5118948244648) / Math.Sqrt((7.426872888487337 / (44)) + (1.1928348607490447 / (7))) is less than -2.0095752371279447 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (7) - 2, .025) and -0.3177002482444929 = (675.8076398717006 - 890.5118948244648) / 675.8076398717006 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.GreatestCommonDivisor(arguments: 65536,65536 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.839415833333333 > 5.539405691489362.
IsChangePoint: Marked as a change because one of 7/2/2022 7:35:06 PM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -161.56410408970288 (T) = (0 -7869490.520161291) / Math.Sqrt((2226018.427149312 / (42)) + (1289360720.4164717 / (5))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (42) + (5) - 2, .025) and -0.4919135064105946 = (5274763.239522212 - 7869490.520161291) / 5274763.239522212 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Numerics.Tests.Perf_BigInteger.Remainder(arguments: 16,8 bits)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.741197078198523 > 9.484905262332438.
IsChangePoint: Marked as a change because one of 7/2/2022 10:13:03 AM, 7/5/2022 10:15:28 AM falls between 6/26/2022 6:11:06 PM and 7/5/2022 10:15:28 AM.
IsRegressionStdDev: Marked as regression because -84.91730986227651 (T) = (0 -14.765549528346044) / Math.Sqrt((0.0426609555518767 / (39)) + (0.026923197489159573 / (8))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (39) + (8) - 2, .025) and -0.6234868287473484 = (9.09496108430942 - 14.765549528346044) / 9.09496108430942 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo EgorBo mentioned this issue Jul 5, 2022
Closed
@kunalspathak
Copy link
Member

Most of the issues are from dotnet/runtime#71498 and fixed by dotnet/runtime#71673.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kunalspathak kunalspathak

Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kunalspathak