Regressions in System.Hashing.GetStringHashCode #82243

performanceautofiler · 2023-02-16T11:28:48Z

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	90603686d314147017c8bbe1fa8965776ce607d0
Compare	e0f7c43484f39d49c7c8d22d56dcc386215064bd
Diff	Diff

Regressions in BenchmarksGame.KNucleotide_9

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	93.28 ms	99.24 ms	1.06	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.KNucleotide_9*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.KNucleotide_9.RunBench

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.24291315789473 > 97.7951874375.
IsChangePoint: Marked as a change because one of 1/13/2023 3:30:06 PM, 1/20/2023 1:51:38 AM, 1/27/2023 7:12:54 PM, 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -6.967215695366574 (T) = (0 -101790024.60891813) / Math.Sqrt((15460792902377.416 / (34)) + (3103458023225.232 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.0821686865460456 = (94061143.95510836 - 101790024.60891813) / 94061143.95510836 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	90603686d314147017c8bbe1fa8965776ce607d0
Compare	e0f7c43484f39d49c7c8d22d56dcc386215064bd
Diff	Diff

Regressions in System.Hashing

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetStringHashCode - Duration of single invocation	4.16 ns	5.53 ns	1.33	0.24	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Hashing*'

Payloads

Baseline
Compare

Histogram

System.Hashing.GetStringHashCode(BytesCount: 10)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.5265806523917655 > 4.363518774344228.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -24.987708712815202 (T) = (0 -5.629406136543158) / Math.Sqrt((0.0057768736181892585 / (34)) + (0.014051827940764663 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.36869056931471206 = (4.112986720849357 - 5.629406136543158) / 4.112986720849357 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	90603686d314147017c8bbe1fa8965776ce607d0
Compare	e0f7c43484f39d49c7c8d22d56dcc386215064bd
Diff	Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IsPrefix_FirstHalf - Duration of single invocation	16.17 ns	17.44 ns	1.08	0.06	False
IsSuffix_SecondHalf - Duration of single invocation	16.54 ns	17.78 ns	1.07	0.04	False
IsPrefix_DifferentFirstChar - Duration of single invocation	7.06 ns	8.14 ns	1.15	0.38	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.43783382139816 > 16.950781266987555.
IsChangePoint: Marked as a change because one of 11/21/2022 5:30:14 PM, 1/19/2023 10:17:02 AM, 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -16.09097395475194 (T) = (0 -17.481281285541336) / Math.Sqrt((0.07128748632478656 / (34)) + (0.016875300948141537 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.07892307005409573 = (16.202528030719417 - 17.481281285541336) / 16.202528030719417 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, Ordinal, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.775503697862852 > 17.344547173602958.
IsChangePoint: Marked as a change because one of 12/8/2022 3:49:24 AM, 1/11/2023 1:25:06 PM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -8.25190043137271 (T) = (0 -17.26867198855478) / Math.Sqrt((0.11334528580185768 / (15)) + (0.1431512144800443 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.05942872404747338 = (16.29998469607377 - 17.26867198855478) / 16.29998469607377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.138259166531808 > 7.503412403424104.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -13.400591693918166 (T) = (0 -8.263086290403939) / Math.Sqrt((0.10403698019916145 / (34)) + (0.01145605786260191 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.1426229552001009 = (7.231682378512055 - 8.263086290403939) / 7.231682378512055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	90603686d314147017c8bbe1fa8965776ce607d0
Compare	e0f7c43484f39d49c7c8d22d56dcc386215064bd
Diff	Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedSet - Duration of single invocation	328.08 μs	383.34 μs	1.17	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 383.3440984320557 > 341.87644531250004.
IsChangePoint: Marked as a change because one of 11/9/2022 4:06:23 PM, 11/11/2022 5:53:56 PM, 2/9/2023 1:41:04 PM, 2/10/2023 11:47:22 AM, 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -5.638843398813679 (T) = (0 -372360.6713960698) / Math.Sqrt((394203067.15192485 / (34)) + (146594287.54615495 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.1175459831359816 = (333194.9441142248 - 372360.6713960698) / 333194.9441142248 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	90603686d314147017c8bbe1fa8965776ce607d0
Compare	e0f7c43484f39d49c7c8d22d56dcc386215064bd
Diff	Diff

Regressions in System.Xml.Linq.Perf_XDocument

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetRootElement - Duration of single invocation	7.90 ns	8.97 ns	1.14	0.48	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XDocument*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XDocument.GetRootElement

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.966666919080053 > 8.595365599327472.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -4.534938859631019 (T) = (0 -9.129912772531144) / Math.Sqrt((0.12475987032270999 / (34)) + (0.13366942312050983 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.1057744639642542 = (8.256577692886822 - 9.129912772531144) / 8.256577692886822 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

The text was updated successfully, but these errors were encountered:

dotnet-issue-labeler · 2023-02-16T17:31:52Z

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

cincuranet · 2023-02-16T17:33:06Z

The commit range is 8a96ee7...cfdd030.

Probably caused by #82041.

cc @stephentoub

stephentoub · 2023-02-16T17:50:45Z

I don't see how #82041 could possibly impact string.GetHashCode(). I also just tried reverting my change and running this locally, and I don't see any impact before/after. The Diff link for that benchmark also suggests a larger commit range: 9060368...e0f7c43

cincuranet · 2023-02-16T18:23:02Z

The commit range is for the regression itself. Manually taken from here.

Let's leave it for now and I'll look again when more data is available. Maybe it's just a noise (although higher than before).

ghost · 2023-02-17T12:46:51Z

Tagging subscribers to this area: @dotnet/area-system-runtime
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	90603686d314147017c8bbe1fa8965776ce607d0
Compare	e0f7c43484f39d49c7c8d22d56dcc386215064bd
Diff	Diff

Regressions in BenchmarksGame.KNucleotide_9

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	93.28 ms	99.24 ms	1.06	0.13	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.KNucleotide_9*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.KNucleotide_9.RunBench

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 99.24291315789473 > 97.7951874375.
IsChangePoint: Marked as a change because one of 1/13/2023 3:30:06 PM, 1/20/2023 1:51:38 AM, 1/27/2023 7:12:54 PM, 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -6.967215695366574 (T) = (0 -101790024.60891813) / Math.Sqrt((15460792902377.416 / (34)) + (3103458023225.232 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.0821686865460456 = (94061143.95510836 - 101790024.60891813) / 94061143.95510836 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	90603686d314147017c8bbe1fa8965776ce607d0
Compare	e0f7c43484f39d49c7c8d22d56dcc386215064bd
Diff	Diff

Regressions in System.Hashing

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetStringHashCode - Duration of single invocation	4.16 ns	5.53 ns	1.33	0.24	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Hashing*'

Payloads

Baseline
Compare

Histogram

System.Hashing.GetStringHashCode(BytesCount: 10)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.5265806523917655 > 4.363518774344228.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -24.987708712815202 (T) = (0 -5.629406136543158) / Math.Sqrt((0.0057768736181892585 / (34)) + (0.014051827940764663 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.36869056931471206 = (4.112986720849357 - 5.629406136543158) / 4.112986720849357 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	90603686d314147017c8bbe1fa8965776ce607d0
Compare	e0f7c43484f39d49c7c8d22d56dcc386215064bd
Diff	Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector
IsPrefix_FirstHalf - Duration of single invocation	16.17 ns	17.44 ns	1.08	0.06	False
IsSuffix_SecondHalf - Duration of single invocation	16.54 ns	17.78 ns	1.07	0.04	False
IsPrefix_DifferentFirstChar - Duration of single invocation	7.06 ns	8.14 ns	1.15	0.38	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.43783382139816 > 16.950781266987555.
IsChangePoint: Marked as a change because one of 11/21/2022 5:30:14 PM, 1/19/2023 10:17:02 AM, 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -16.09097395475194 (T) = (0 -17.481281285541336) / Math.Sqrt((0.07128748632478656 / (34)) + (0.016875300948141537 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.07892307005409573 = (16.202528030719417 - 17.481281285541336) / 16.202528030719417 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, Ordinal, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.775503697862852 > 17.344547173602958.
IsChangePoint: Marked as a change because one of 12/8/2022 3:49:24 AM, 1/11/2023 1:25:06 PM, 2/9/2023 1:41:04 PM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -8.25190043137271 (T) = (0 -17.26867198855478) / Math.Sqrt((0.11334528580185768 / (15)) + (0.1431512144800443 / (23))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (23) - 2, .025) and -0.05942872404747338 = (16.29998469607377 - 17.26867198855478) / 16.29998469607377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.138259166531808 > 7.503412403424104.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -13.400591693918166 (T) = (0 -8.263086290403939) / Math.Sqrt((0.10403698019916145 / (34)) + (0.01145605786260191 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.1426229552001009 = (7.231682378512055 - 8.263086290403939) / 7.231682378512055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	90603686d314147017c8bbe1fa8965776ce607d0
Compare	e0f7c43484f39d49c7c8d22d56dcc386215064bd
Diff	Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SortedSet - Duration of single invocation	328.08 μs	383.34 μs	1.17	0.02	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 383.3440984320557 > 341.87644531250004.
IsChangePoint: Marked as a change because one of 11/9/2022 4:06:23 PM, 11/11/2022 5:53:56 PM, 2/9/2023 1:41:04 PM, 2/10/2023 11:47:22 AM, 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -5.638843398813679 (T) = (0 -372360.6713960698) / Math.Sqrt((394203067.15192485 / (34)) + (146594287.54615495 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.1175459831359816 = (333194.9441142248 - 372360.6713960698) / 333194.9441142248 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.19041
Baseline	90603686d314147017c8bbe1fa8965776ce607d0
Compare	e0f7c43484f39d49c7c8d22d56dcc386215064bd
Diff	Diff

Regressions in System.Xml.Linq.Perf_XDocument

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
GetRootElement - Duration of single invocation	7.90 ns	8.97 ns	1.14	0.48	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Linq.Perf_XDocument*'

Payloads

Baseline
Compare

Histogram

System.Xml.Linq.Perf_XDocument.GetRootElement

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.966666919080053 > 8.595365599327472.
IsChangePoint: Marked as a change because one of 2/14/2023 11:21:41 AM, 2/16/2023 5:39:51 AM falls between 2/7/2023 3:56:56 PM and 2/16/2023 5:39:51 AM.
IsRegressionStdDev: Marked as regression because -4.534938859631019 (T) = (0 -9.129912772531144) / Math.Sqrt((0.12475987032270999 / (34)) + (0.13366942312050983 / (4))) is less than -2.028094000977961 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (4) - 2, .025) and -0.1057744639642542 = (8.256577692886822 - 9.129912772531144) / 8.256577692886822 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	EgorBo
Labels:	`area-System.Runtime`, `untriaged`
Milestone:	-

cincuranet · 2023-02-22T10:04:50Z

So far seems to be stable in ~5.6ns range, compared to ~4.0ns before. Let's wait bit more whether it changes back.

cincuranet · 2023-03-07T12:42:18Z

The numbers are back to previous values. Closing.

performanceautofiler bot assigned EgorBo Feb 16, 2023

performanceautofiler bot added the untriaged New issue has not been triaged by the area owner label Feb 16, 2023

cincuranet changed the title ~~[Perf] Windows/arm64: 7 Regressions on 2/14/2023 6:31:57 PM~~ Regressions in System.Hashing.GetStringHashCode Feb 16, 2023

cincuranet removed the untriaged New issue has not been triaged by the area owner label Feb 16, 2023

cincuranet transferred this issue from dotnet/perf-autofiling-issues Feb 16, 2023

ghost added the untriaged New issue has not been triaged by the area owner label Feb 16, 2023

jakobbotsch mentioned this issue Feb 16, 2023

[Perf] Windows/x64: 23 Regressions on 2/9/2023 12:40:40 PM #82111

Closed

dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Feb 17, 2023

vcsjones added area-System.IO.Hashing area-System.Runtime and removed area-System.IO.Hashing needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners labels Feb 17, 2023

dakersnar added tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark bug and removed untriaged New issue has not been triaged by the area owner labels Feb 17, 2023

dakersnar added this to the 8.0.0 milestone Feb 17, 2023

cincuranet mentioned this issue Feb 23, 2023

[Perf] Windows/arm64: 2 Regressions on 2/14/2023 6:31:57 PM dotnet/perf-autofiling-issues#13353

Closed

ericstj mentioned this issue Feb 24, 2023

PR/issue labeling bot isn't labeling #82643

Closed

EgorBo removed their assignment Mar 6, 2023

cincuranet closed this as completed Mar 7, 2023

ghost locked as resolved and limited conversation to collaborators Apr 6, 2023

Regressions in System.Hashing.GetStringHashCode #82243

Regressions in System.Hashing.GetStringHashCode #82243

Comments

performanceautofiler bot commented Feb 16, 2023

Run Information

Regressions in BenchmarksGame.KNucleotide_9

Repro

Payloads

Payloads

Histogram

BenchmarksGame.KNucleotide_9.RunBench

Description of detection logic

Docs

Run Information

Regressions in System.Hashing

Repro

Payloads

Payloads

Histogram

System.Hashing.GetStringHashCode(BytesCount: 10)

Description of detection logic

Docs

Run Information

Regressions in System.Globalization.Tests.StringSearch

Repro

Payloads

Payloads

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False))

Description of detection logic

Description of detection logic

Description of detection logic

Docs

Run Information

Regressions in System.Collections.ContainsFalse<String>

Repro

Payloads

Payloads

Histogram

System.Collections.ContainsFalse<String>.SortedSet(Size: 512)

Description of detection logic

Docs

Run Information

Regressions in System.Xml.Linq.Perf_XDocument

Repro

Payloads

Payloads

Histogram

System.Xml.Linq.Perf_XDocument.GetRootElement

Description of detection logic

Docs

dotnet-issue-labeler bot commented Feb 16, 2023

cincuranet commented Feb 16, 2023

stephentoub commented Feb 16, 2023

cincuranet commented Feb 16, 2023

ghost commented Feb 17, 2023

Run Information

Regressions in BenchmarksGame.KNucleotide_9

Repro

Payloads

Payloads

Histogram

BenchmarksGame.KNucleotide_9.RunBench

Description of detection logic

Docs

Run Information

Regressions in System.Hashing

Repro

Payloads

Payloads

Histogram

System.Hashing.GetStringHashCode(BytesCount: 10)

Description of detection logic

Docs

Run Information

Regressions in System.Globalization.Tests.StringSearch

Repro

Payloads

Payloads

Histogram