[Perf] Changes at 6/7/2022 6:41:39 PM

[performanceautofiler]

Run Information

Architecture	arm64
OS	Windows 10.0.25094
Baseline	bab1a1f239325746873d2ae795ea343e7901dbce
Compare	31cd7eb907133929846e5d229d18cc8bc77bde63
Diff	Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	225.04 ms	154.77 ms	0.69	0.58	False
Count - Duration of single invocation	233.02 ms	140.73 ms	0.60	0.59	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{0,2}(Tom|Sawyer|Huckleberry|Finn)", Options: NonBacktracking)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 154.77449142857142 < 211.5840703.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsImprovementStdDev: Marked as improvement because 38.6036296656767 (T) = (0 -149766788.04360902) / Math.Sqrt((178798483322549.5 / (118)) + (14976891759625.998 / (5))) is greater than 1.9797637625051276 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (5) - 2, .975) and 0.3537683400176119 = (231754024.62901717 - 149766788.04360902) / 231754024.62901717 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_Leipzig.Count(Pattern: ".{2,4}(Tom|Sawyer|Huckleberry|Finn)", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 140.73485 < 223.88938324999998.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsImprovementStdDev: Marked as improvement because 29.611635425229174 (T) = (0 -146946284.83157894) / Math.Sqrt((169762776289177.7 / (118)) + (32207566315249.17 / (5))) is greater than 1.9797637625051276 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (5) - 2, .975) and 0.3613004412439332 = (230071060.51203787 - 146946284.83157894) / 230071060.51203787 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	arm64
OS	Windows 10.0.25094
Baseline	bab1a1f239325746873d2ae795ea343e7901dbce
Compare	31cd7eb907133929846e5d229d18cc8bc77bde63
Diff	Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Count - Duration of single invocation	32.50 ms	14.95 ms	0.46	0.48	False
Count - Duration of single invocation	3.53 ms	2.57 ms	0.73	0.71	False
Count - Duration of single invocation	549.51 μs	474.35 μs	0.86	0.64	False
Count - Duration of single invocation	573.41 μs	396.17 μs	0.69	0.66	False
Count - Duration of single invocation	9.43 ms	5.67 ms	0.60	0.74	False
Count - Duration of single invocation	8.90 ms	7.76 ms	0.87	0.67	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: ".*", Options: NonBacktracking)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.9519135 < 31.33014788392857.
IsChangePoint: Marked as a change because one of 5/27/2022 6:33:44 PM, 6/7/2022 9:37:06 AM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsImprovementStdDev: Marked as improvement because 14.107473388852906 (T) = (0 -16150561.333333332) / Math.Sqrt((44284792700521.04 / (118)) + (1009580597495.8738 / (5))) is greater than 1.9797637625051276 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (5) - 2, .975) and 0.3989064040966862 = (26868629.84967013 - 16150561.333333332) / 26868629.84967013 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "the\s+\w+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 2.566841876923077 < 3.6900214242221216.
IsChangePoint: Marked as a change because one of 5/27/2022 6:33:44 PM, 6/7/2022 9:37:06 AM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsImprovementStdDev: Marked as improvement because 11.574283198707532 (T) = (0 -2619252.4548204523) / Math.Sqrt((245026887956.07703 / (117)) + (15946333057.329706 / (5))) is greater than 1.9799304050821813 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (117) + (5) - 2, .975) and 0.24311343206421127 = (3460561.4180256655 - 2619252.4548204523) / 3460561.4180256655 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sher[a-z]+|Hol[a-z]+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 474.35187288135586 < 520.0398213922654.
IsChangePoint: Marked as a change because one of 5/31/2022 10:14:36 PM, 6/7/2022 9:37:06 AM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsImprovementStdDev: Marked as improvement because 4.1081693259956085 (T) = (0 -489673.40927098505) / Math.Sqrt((2433135523.9181185 / (117)) + (943919659.7516412 / (5))) is greater than 1.9799304050821813 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (117) + (5) - 2, .975) and 0.10830155724636913 = (549146.8705034824 - 489673.40927098505) / 549146.8705034824 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Holmes.{0,25}Watson|Watson.{0,25}Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 396.1696431322208 < 463.7252831943128.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsImprovementStdDev: Marked as improvement because 7.380541315414207 (T) = (0 -434976.63516157184) / Math.Sqrt((1397638909.6096232 / (117)) + (790511849.6192782 / (5))) is greater than 1.9799304050821813 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (117) + (5) - 2, .975) and 0.18117529940054714 = (531220.7055345669 - 434976.63516157184) / 531220.7055345669 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes\s+\w+", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 5.66772268292683 < 9.204016620307693.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsImprovementStdDev: Marked as improvement because 23.626736946240406 (T) = (0 -5832649.363143631) / Math.Sqrt((1732932408373.6067 / (118)) + (52974541694.08628 / (5))) is greater than 1.9797637625051276 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (5) - 2, .975) and 0.3917533970431769 = (9589283.910160476 - 5832649.363143631) / 9589283.910160476 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "\w+\s+Holmes", Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 7.7553452 < 8.332167810416665.
IsChangePoint: Marked as a change because one of 6/7/2022 9:37:06 AM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsImprovementStdDev: Marked as improvement because 7.671235060277466 (T) = (0 -6258379.894417344) / Math.Sqrt((888216086378.3363 / (118)) + (716970823608.9163 / (5))) is greater than 1.9797637625051276 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (5) - 2, .975) and 0.32257969123234015 = (9238547.786974939 - 6258379.894417344) / 9238547.786974939 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	arm64
OS	Windows 10.0.25094
Baseline	bab1a1f239325746873d2ae795ea343e7901dbce
Compare	31cd7eb907133929846e5d229d18cc8bc77bde63
Diff	Diff

Improvements in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IsMatch - Duration of single invocation	694.55 ns	376.10 ns	0.54	0.64	False
IsMatch - Duration of single invocation	465.59 ns	328.99 ns	0.71	0.56	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 3, Options: NonBacktracking)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 376.10067960449464 < 647.5192578923223.
IsChangePoint: Marked as a change because one of 5/27/2022 3:44:45 PM, 6/7/2022 9:37:06 AM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsImprovementStdDev: Marked as improvement because 3.507171738024196 (T) = (0 -495.6248679267777) / Math.Sqrt((3808.5346295897116 / (118)) + (5429.847381691687 / (5))) is greater than 1.9797637625051276 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (118) + (5) - 2, .975) and 0.19135163476097164 = (612.9052988071966 - 495.6248679267777) / 612.9052988071966 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: NonBacktracking)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 328.98859872626974 < 412.9010463789289.
IsChangePoint: Marked as a change because one of 5/31/2022 4:05:26 AM, 6/7/2022 9:37:06 AM, 6/9/2022 4:58:43 AM falls between 5/31/2022 10:47:41 AM and 6/9/2022 4:58:43 AM.
IsImprovementStdDev: Marked as improvement because 4.717598320923721 (T) = (0 -364.2225747280161) / Math.Sqrt((1054.0431386187652 / (117)) + (1520.4677099122218 / (5))) is greater than 1.9799304050821813 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (117) + (5) - 2, .975) and 0.18645670613023813 = (447.69906835016405 - 364.2225747280161) / 447.69906835016405 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[AndyAyersMS]

Likely from dotnet/runtime#70217