Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/arm64: 85 Regressions on 1/14/2023 12:48:25 AM #11811

Closed
performanceautofiler bot opened this issue Jan 17, 2023 · 6 comments
Closed

[Perf] Linux/arm64: 85 Regressions on 1/14/2023 12:48:25 AM #11811

performanceautofiler bot opened this issue Jan 17, 2023 · 6 comments
Assignees
@naricc
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro_mono LinkedFromMaster Is this issue linked from the master perf issue mono-aot os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Doubles

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDoubles - Duration of single invocation 86.15 ms 144.18 ms 1.67 0.03 False
WriteDoubles - Duration of single invocation 88.89 ms 141.13 ms 1.59 0.02 False
WriteDoubles - Duration of single invocation 88.49 ms 143.80 ms 1.63 0.04 False
WriteDoubles - Duration of single invocation 87.64 ms 134.97 ms 1.54 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Doubles*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: False)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 144.1792567857143 > 91.12610216062501.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -21.998899790251347 (T) = (0 -138907805.8384782) / Math.Sqrt((118146921285548.23 / (22)) + (8805566641360.295 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6485609125668402 = (84260038.42478356 - 138907805.8384782) / 84260038.42478356 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 141.12740340000002 > 92.6995007375.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -21.766820008789065 (T) = (0 -141507488.67056277) / Math.Sqrt((124037079474261.22 / (22)) + (9699455169313.46 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6468101452391395 = (85928234.7024975 - 141507488.67056277) / 85928234.7024975 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 143.8019899 > 93.08843226125.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -22.04275855356004 (T) = (0 -141643927.17772228) / Math.Sqrt((122001493646991.12 / (22)) + (9619682272154.936 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6510071829134964 = (85792435.45613553 - 141643927.17772228) / 85792435.45613553 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Doubles.WriteDoubles(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 134.97435863333334 > 91.530344908125.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -22.703359701645876 (T) = (0 -139630866.35) / Math.Sqrt((119035950847292.22 / (22)) + (6017566346836.912 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6580237000432874 = (84215241.52299786 - 139630866.35) / 84215241.52299786 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterDouble - Duration of single invocation 581.03 ns 1.04 μs 1.80 0.06 False
FormatterDouble - Duration of single invocation 855.51 ns 1.31 μs 1.53 0.03 False
FormatterDouble - Duration of single invocation 891.42 ns 1.32 μs 1.48 0.02 False
FormatterInt32 - Duration of single invocation 101.60 ns 168.04 ns 1.65 0.19 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0443771298653797 > 621.2602250860122.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -22.330410683870443 (T) = (0 -1039.5295575664525) / Math.Sqrt((6681.479719904247 / (22)) + (1524.1876049497682 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.8239976895515045 = (569.918242507236 - 1039.5295575664525) / 569.918242507236 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.308134972532904 > 900.4946825477134.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -18.66479157836275 (T) = (0 -1307.655033672269) / Math.Sqrt((12907.709260648911 / (22)) + (551.986549304299 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.5630290552995387 = (836.6159472458879 - 1307.655033672269) / 836.6159472458879 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: 1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.3163259766395987 > 903.0462250780093.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -19.137088288076054 (T) = (0 -1307.7313040674373) / Math.Sqrt((12970.904616639655 / (22)) + (209.7872979514165 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.5650202670139751 = (835.6002357481034 - 1307.7313040674373) / 835.6002357481034 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8FormatterTests.FormatterInt32(value: -2147483648)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 168.03948883326922 > 104.37242863578727.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/15/2023 4:47:52 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -11.955916890703222 (T) = (0 -169.50046304284956) / Math.Sqrt((186.38173647227663 / (29)) + (120.30957495108424 / (4))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (29) + (4) - 2, .025) and -0.7426806784514022 = (97.26421204914759 - 169.50046304284956) / 97.26421204914759 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 534.28 ns 1.09 μs 2.05 0.03 False
ToStringWithCultureInfo - Duration of single invocation 531.45 ns 1.04 μs 1.96 0.02 False
TryParse - Duration of single invocation 267.78 ns 371.95 ns 1.39 0.03 False
ToStringWithFormat - Duration of single invocation 423.24 ns 922.78 ns 2.18 0.03 False
ToStringWithFormat - Duration of single invocation 545.90 ns 1.08 μs 1.97 0.02 False
ToStringWithFormat - Duration of single invocation 518.21 ns 1.00 μs 1.93 0.01 False
ToStringWithFormat - Duration of single invocation 462.70 ns 910.65 ns 1.97 0.03 False
ToString - Duration of single invocation 547.86 ns 1.04 μs 1.90 0.02 False
ToStringWithFormat - Duration of single invocation 520.66 ns 1.03 μs 1.97 0.01 False
ToString - Duration of single invocation 458.19 ns 960.61 ns 2.10 0.05 False
ToStringWithFormat - Duration of single invocation 576.17 ns 1.14 μs 1.98 0.05 False
Parse - Duration of single invocation 262.09 ns 374.71 ns 1.43 0.01 False
TryParse - Duration of single invocation 260.33 ns 380.08 ns 1.46 0.02 False
ToStringWithFormat - Duration of single invocation 564.15 ns 1.08 μs 1.92 0.02 False
ToStringWithFormat - Duration of single invocation 432.09 ns 942.63 ns 2.18 0.03 False
ToStringWithFormat - Duration of single invocation 469.16 ns 899.87 ns 1.92 0.05 False
ToStringWithCultureInfo - Duration of single invocation 521.32 ns 1.07 μs 2.06 0.01 False
Parse - Duration of single invocation 270.43 ns 381.11 ns 1.41 0.01 False
ToStringWithFormat - Duration of single invocation 553.26 ns 1.09 μs 1.98 0.02 False
ToStringWithFormat - Duration of single invocation 464.88 ns 968.84 ns 2.08 0.02 False
ToStringWithCultureInfo - Duration of single invocation 412.27 ns 904.28 ns 2.19 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToString(value: -3.4028235E+38)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0940144647550774 > 568.9109405414339.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -32.71252641752556 (T) = (0 -1090.4646682314806) / Math.Sqrt((4748.190065288139 / (22)) + (876.6795090449701 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.064848191776336 = (528.1088811150721 - 1090.4646682314806) / 528.1088811150721 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: -3.4028235E+38, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.040349428310654 > 557.9644782691596.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -31.29578767348893 (T) = (0 -1040.6490146822532) / Math.Sqrt((4577.228591246492 / (22)) + (820.8363582288528 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0227491564427342 = (514.4725985264377 - 1040.6490146822532) / 514.4725985264377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "-3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 371.95443351042417 > 282.39947623095395.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -16.431120793303368 (T) = (0 -377.6572530711475) / Math.Sqrt((773.2123208010917 / (22)) + (146.1390364532415 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.4342755548899039 = (263.3087148306985 - 377.6572530711475) / 263.3087148306985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 922.7775361234185 > 445.7259508563003.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -36.92009768495071 (T) = (0 -945.8801060422472) / Math.Sqrt((2707.5331263597477 / (22)) + (854.7993881529243 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.237521869446558 = (422.73558035712375 - 945.8801060422472) / 422.73558035712375 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0753949432203556 > 574.0708735525249.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -32.36347001260134 (T) = (0 -1098.914945690417) / Math.Sqrt((5164.04742338528 / (22)) + (742.673358592602 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0491839778665861 = (536.269538294215 - 1098.914945690417) / 536.269538294215 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0017189662173236 > 543.8827610272366.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -35.59338497924322 (T) = (0 -1039.0135704345462) / Math.Sqrt((3099.198370193167 / (22)) + (865.8201896164315 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.03099643413455 = (511.578234689167 - 1039.0135704345462) / 511.578234689167 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 910.6513666510841 > 495.17323282284536.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -27.479313710534907 (T) = (0 -921.1396540033235) / Math.Sqrt((4043.4850905928274 / (22)) + (1093.8541684764155 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0082696960354314 = (458.6732826879603 - 921.1396540033235) / 458.6732826879603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0408563785331675 > 565.9772102263504.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -32.43358030103064 (T) = (0 -1092.156708430564) / Math.Sqrt((4775.586869694899 / (22)) + (986.1551660296449 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0837464937087677 = (524.1312759147985 - 1092.156708430564) / 524.1312759147985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0264138500122475 > 548.7963137560877.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -38.651646408104014 (T) = (0 -1039.4520785671532) / Math.Sqrt((3258.210677001105 / (22)) + (376.1141052998564 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0082391919183546 = (517.5937621126818 - 1039.4520785671532) / 517.5937621126818 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 960.6132379754689 > 483.53541965030905.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -30.696747476814945 (T) = (0 -914.4239507190159) / Math.Sqrt((3884.1428082055645 / (22)) + (557.1680104333811 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0243002835591295 = (451.72347113999984 - 914.4239507190159) / 451.72347113999984 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1412196820015554 > 577.9730969832802.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -28.987840526748034 (T) = (0 -1097.3907755941605) / Math.Sqrt((5284.215565521317 / (22)) + (1436.552088679689 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0351989871586214 = (539.205641570335 - 1097.3907755941605) / 539.205641570335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 374.70599184570017 > 274.81415679826904.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -19.777931025285636 (T) = (0 -370.6458304871487) / Math.Sqrt((671.9325573103959 / (22)) + (27.196412566223085 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.4421746882020833 = (257.00480913946836 - 370.6458304871487) / 257.00480913946836 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.TryParse(value: "3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 380.0756396795303 > 273.09033978773266.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -18.431980693095404 (T) = (0 -369.0071955489105) / Math.Sqrt((710.9146382427269 / (22)) + (57.14412114194219 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.4407536649002356 = (256.1209487358564 - 369.0071955489105) / 256.1209487358564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0807208957671595 > 581.8720651281561.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -33.821788849279024 (T) = (0 -1111.3488259639898) / Math.Sqrt((5322.7212608941645 / (22)) + (452.26683607156855 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0492456777601997 = (542.3209320508024 - 1111.3488259639898) / 542.3209320508024 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 942.6272693407433 > 452.0084506995151.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -39.56013834869683 (T) = (0 -964.5858709159372) / Math.Sqrt((2785.3104064194617 / (22)) + (636.0441203450191 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.2571678162046038 = (427.3434451754123 - 964.5858709159372) / 427.3434451754123 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 899.8723393220198 > 501.9359168411381.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -32.223033490616935 (T) = (0 -919.115349254561) / Math.Sqrt((4033.666376233582 / (22)) + (202.84633421848218 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.9921040264343877 = (461.3791935854175 - 919.115349254561) / 461.3791935854175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 3.4028235E+38, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0732752713692664 > 547.3890969902443.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -35.622211353745875 (T) = (0 -1056.8819589817374) / Math.Sqrt((4527.19473701261 / (22)) + (345.1040503465932 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0792235914953983 = (508.30606352519186 - 1056.8819589817374) / 508.30606352519186 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.Parse(value: "-3.4028235E+38")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 381.1094532303394 > 282.38532515518887.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -19.02514610185643 (T) = (0 -378.4708978813185) / Math.Sqrt((716.632762929477 / (22)) + (46.06666719492705 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.4384069224102335 = (263.1181009940792 - 378.4708978813185) / 263.1181009940792 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.092825987304167 > 583.4021907886233.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -34.01319294247653 (T) = (0 -1106.300624707004) / Math.Sqrt((5208.814660020266 / (22)) + (414.35401288165866 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0379957066036587 = (542.8375639469161 - 1106.300624707004) / 542.8375639469161 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 968.840055648648 > 488.4063912455135.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -40.33328120880908 (T) = (0 -986.5445207086457) / Math.Sqrt((3108.1413241387168 / (22)) + (376.1279214892162 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.1813398403889845 = (452.26539324231175 - 986.5445207086457) / 452.26539324231175 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 904.2762007359582 > 436.1892330330309.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -33.369371008294934 (T) = (0 -849.1041108975164) / Math.Sqrt((2928.8415276772885 / (22)) + (449.0652879924252 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0760949686672556 = (408.99097763461026 - 849.1041108975164) / 408.99097763461026 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Collections.CopyTo<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadOnlySpan - Duration of single invocation 1.93 μs 2.10 μs 1.08 0.03 False
List - Duration of single invocation 1.94 μs 2.11 μs 1.09 0.03 False
ImmutableArray - Duration of single invocation 2.00 μs 2.16 μs 1.08 0.03 False
ReadOnlyMemory - Duration of single invocation 1.96 μs 2.12 μs 1.08 0.03 False
Span - Duration of single invocation 1.93 μs 2.10 μs 1.09 0.03 False
Array - Duration of single invocation 1.97 μs 2.14 μs 1.09 0.03 False
Memory - Duration of single invocation 1.96 μs 2.13 μs 1.09 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CopyTo&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CopyTo<String>.ReadOnlySpan(Size: 2048)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.096664712245538 > 2.028645950933168.
IsChangePoint: Marked as a change because one of 11/17/2022 1:45:21 AM, 12/15/2022 2:06:03 PM, 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -20.730838143462634 (T) = (0 -2098.297231313561) / Math.Sqrt((1252.5977804165263 / (22)) + (9.47142376043482 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.08121084446225353 = (1940.6919955165279 - 2098.297231313561) / 1940.6919955165279 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CopyTo&lt;String&gt;.List(Size: 2048)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.107606974478533 > 2.0351061923830653.
IsChangePoint: Marked as a change because one of 11/17/2022 1:45:21 AM, 12/15/2022 2:06:03 PM, 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -22.699217941315126 (T) = (0 -2111.0218757895286) / Math.Sqrt((1107.5231993815023 / (22)) + (29.650432216160937 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.08496185344438853 = (1945.71068935534 - 2111.0218757895286) / 1945.71068935534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CopyTo&lt;String&gt;.ImmutableArray(Size: 2048)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.161274196986167 > 2.098044355061525.
IsChangePoint: Marked as a change because one of 11/17/2022 1:45:21 AM, 12/15/2022 2:06:03 PM, 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -21.752783689394608 (T) = (0 -2169.2138598353904) / Math.Sqrt((1117.8474471283455 / (22)) + (58.69331968056956 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.08124565397703168 = (2006.2174140137352 - 2169.2138598353904) / 2006.2174140137352 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CopyTo&lt;String&gt;.ReadOnlyMemory(Size: 2048)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1221657912354757 > 2.0579648765831102.
IsChangePoint: Marked as a change because one of 11/17/2022 1:45:21 AM, 12/15/2022 2:06:03 PM, 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -22.555333652490013 (T) = (0 -2128.8748204632557) / Math.Sqrt((1068.5331284854396 / (22)) + (25.584867079133343 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.08176595096137267 = (1967.962495557667 - 2128.8748204632557) / 1967.962495557667 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CopyTo&lt;String&gt;.Span(Size: 2048)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.100663356225873 > 2.0290696832029242.
IsChangePoint: Marked as a change because one of 11/17/2022 1:45:21 AM, 12/15/2022 2:06:03 PM, 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -20.657821273410175 (T) = (0 -2101.6526158413635) / Math.Sqrt((1294.4818821151598 / (22)) + (8.43242660082031 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.08211948752869758 = (1942.1631715006224 - 2101.6526158413635) / 1942.1631715006224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CopyTo&lt;String&gt;.Array(Size: 2048)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.144406309454225 > 2.0723983443214746.
IsChangePoint: Marked as a change because one of 11/17/2022 1:45:21 AM, 12/15/2022 2:06:03 PM, 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -20.111240072809633 (T) = (0 -2138.859278354528) / Math.Sqrt((1316.7083356839773 / (22)) + (21.34597277102707 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.07981178790542637 = (1980.7704475086327 - 2138.859278354528) / 1980.7704475086327 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.CopyTo&lt;String&gt;.Memory(Size: 2048)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1336786791507203 > 2.0613416883843025.
IsChangePoint: Marked as a change because one of 11/17/2022 1:45:21 AM, 12/15/2022 2:06:03 PM, 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -22.68270443483433 (T) = (0 -2129.886432054736) / Math.Sqrt((1050.7433810456603 / (22)) + (12.24632921016985 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.08044150990543802 = (1971.311183926233 - 2129.886432054736) / 1971.311183926233 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DotBenchmark - Duration of single invocation 56.71 ns 66.95 ns 1.18 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Single&gt;*'

Payloads

Baseline
Compare

Histogram

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Single>.DotBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.95086894185903 > 59.168338288261594.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -39.99491180575241 (T) = (0 -67.43422876643261) / Math.Sqrt((0.1857559318555801 / (22)) + (0.7511814414525844 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.1965891656761907 = (56.35537300584335 - 67.43422876643261) / 56.35537300584335 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseR - Duration of single invocation 191.62 ns 280.34 ns 1.46 0.03 False
ParseO - Duration of single invocation 193.62 ns 269.06 ns 1.39 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_DateTime.ParseR

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 280.34280964756726 > 201.09986729563755.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -21.037707288502016 (T) = (0 -284.3192969894871) / Math.Sqrt((342.74815399543786 / (22)) + (54.97961101279365 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.5052356933698355 = (188.88689541567362 - 284.3192969894871) / 188.88689541567362 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_DateTime.ParseO

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 269.0585002827427 > 202.9154882665593.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -18.75638539744059 (T) = (0 -274.2567498576222) / Math.Sqrt((318.5609250446261 / (22)) + (61.25827351517745 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.4413855112752763 = (190.2730031016973 - 274.2567498576222) / 190.2730031016973 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Mod - Duration of single invocation 20.69 ns 32.19 ns 1.56 0.05 False
Round - Duration of single invocation 19.34 ns 97.78 ns 5.06 0.05 False
Divide - Duration of single invocation 107.36 ns 423.33 ns 3.94 0.04 False
Floor - Duration of single invocation 15.34 ns 109.67 ns 7.15 0.08 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.Mod

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.1948620517696 > 21.6842898666347.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -62.89682274160877 (T) = (0 -31.962862283452456) / Math.Sqrt((0.5309304554471843 / (22)) + (0.10605584733704967 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.5665182143075894 = (20.403760384988715 - 31.962862283452456) / 20.403760384988715 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Round

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 97.78043394125277 > 20.462981025910743.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -180.92552640656714 (T) = (0 -98.16643315835046) / Math.Sqrt((0.1904053619153127 / (22)) + (1.9916300827003481 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -4.0698985308808115 = (19.36260312911936 - 98.16643315835046) / 19.36260312911936 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Divide

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 423.33330119500266 > 112.8480351496989.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -42.49065558344479 (T) = (0 -431.9027980761312) / Math.Sqrt((71.1723754530143 / (22)) + (612.6173367560209 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -3.0851061546235004 = (105.72621168909063 - 431.9027980761312) / 105.72621168909063 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Decimal.Floor

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 109.67116614111316 > 16.100546930634575.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -40.555025228532834 (T) = (0 -99.0572755675155) / Math.Sqrt((0.09972825915161813 / (22)) + (46.93831575282203 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -5.500570733565482 = (15.238242860128656 - 99.0572755675155) / 15.238242860128656 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 606.16 ns 1.08 μs 1.79 0.01 False
ToStringWithFormat - Duration of single invocation 486.33 ns 1.01 μs 2.07 0.05 False
ToStringWithFormat - Duration of single invocation 747.65 ns 1.23 μs 1.65 0.02 False
ToStringWithCultureInfo - Duration of single invocation 717.59 ns 1.14 μs 1.59 0.02 False
ToString - Duration of single invocation 733.39 ns 1.21 μs 1.64 0.02 False
ToStringWithFormat - Duration of single invocation 744.18 ns 1.16 μs 1.56 0.03 False
ToStringWithFormat - Duration of single invocation 605.49 ns 1.03 μs 1.70 0.02 False
ToStringWithCultureInfo - Duration of single invocation 696.67 ns 1.12 μs 1.61 0.03 False
ToStringWithFormat - Duration of single invocation 486.77 ns 943.07 ns 1.94 0.07 False
ToStringWithFormat - Duration of single invocation 515.25 ns 921.65 ns 1.79 0.03 False
ToStringWithFormat - Duration of single invocation 501.42 ns 982.30 ns 1.96 0.02 False
ToString - Duration of single invocation 500.91 ns 943.45 ns 1.88 0.07 False
ToStringWithFormat - Duration of single invocation 744.49 ns 1.24 μs 1.66 0.03 False
ToStringWithFormat - Duration of single invocation 509.62 ns 948.96 ns 1.86 0.05 False
ToStringWithCultureInfo - Duration of single invocation 468.86 ns 898.46 ns 1.92 0.03 False
ToString - Duration of single invocation 722.74 ns 1.16 μs 1.60 0.03 False
ToStringWithFormat - Duration of single invocation 734.78 ns 1.19 μs 1.62 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.083392700169577 > 634.6917303867356.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -26.698211226057516 (T) = (0 -1053.7599702154405) / Math.Sqrt((5247.220764828385 / (22)) + (720.9160051418672 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.7914259941633212 = (588.2241151176302 - 1053.7599702154405) / 588.2241151176302 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0058803159994851 > 511.1313038964619.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -30.203022132531146 (T) = (0 -1023.706205747294) / Math.Sqrt((4051.82266584406 / (22)) + (1566.5985786848755 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.1421750359102791 = (477.88168034190926 - 1023.706205747294) / 477.88168034190926 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.230442912260086 > 779.6188407383734.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -21.170279281614146 (T) = (0 -1195.8627832790332) / Math.Sqrt((9750.279013220357 / (22)) + (695.9326286330742 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6622313953582339 = (719.4321961541992 - 1195.8627832790332) / 719.4321961541992 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: -1.7976931348623157E+308, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1375299866083173 > 753.563166813998.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -20.43491330754242 (T) = (0 -1135.024963874581) / Math.Sqrt((9123.425758715566 / (22)) + (486.29358390150526 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6278252432158669 = (697.2646287461858 - 1135.024963874581) / 697.2646287461858 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: -1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2053540735206603 > 770.9228826664951.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -23.555303135441925 (T) = (0 -1207.9297009900781) / Math.Sqrt((9252.371811566403 / (22)) + (254.5180322301183 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6971116534217313 = (711.7561761800644 - 1207.9297009900781) / 711.7561761800644 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1622193238924667 > 774.6923703026289.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -19.539635209235264 (T) = (0 -1190.9177487147836) / Math.Sqrt((9753.906518923202 / (22)) + (1583.047333771337 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6600473602044439 = (717.3998629581963 - 1190.9177487147836) / 717.3998629581963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0291029060732486 > 631.6785809759439.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -27.29494132346031 (T) = (0 -1061.0997025597992) / Math.Sqrt((5342.725908344476 / (22)) + (648.6504961546983 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.8079549832190296 = (586.9060415821479 - 1061.0997025597992) / 586.9060415821479 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 1.7976931348623157E+308, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1207703258152408 > 742.107600883793.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -21.466485318834128 (T) = (0 -1141.6466026269084) / Math.Sqrt((8916.511517867992 / (22)) + (416.0996372929786 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6551203410188496 = (689.7665229128536 - 1141.6466026269084) / 689.7665229128536 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 943.0733968461249 > 529.4850326383313.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -27.479708072163756 (T) = (0 -955.5396470890205) / Math.Sqrt((4991.197466198652 / (22)) + (705.7612562208323 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.9631233230040301 = (486.74458496414024 - 955.5396470890205) / 486.74458496414024 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 921.6497828722476 > 527.479762804755.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -29.7972650227841 (T) = (0 -943.938930036577) / Math.Sqrt((4357.648203368075 / (22)) + (392.2866073554727 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.9327789594336746 = (488.3843159763915 - 943.938930036577) / 488.3843159763915 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 982.3030244225673 > 513.4327858014667.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -25.58184496985121 (T) = (0 -934.5537375155836) / Math.Sqrt((4063.2447843068207 / (22)) + (1459.547549343012 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.9518271610236629 = (478.809679554538 - 934.5537375155836) / 478.809679554538 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: 12345)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 943.4531786008727 > 525.1005534132838.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -28.965073366524123 (T) = (0 -936.9938116018465) / Math.Sqrt((4645.715487734963 / (22)) + (373.37399982556303 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.9378790888232714 = (483.51510525396736 - 936.9938116018465) / 483.51510525396736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.2377173571995241 > 770.8375325742082.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -18.99242907288719 (T) = (0 -1191.114677486036) / Math.Sqrt((9705.442287967238 / (22)) + (2045.8029037999975 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6647483773630979 = (715.4923192493026 - 1191.114677486036) / 715.4923192493026 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 948.9564080570738 > 527.7335234710706.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -29.92419044847566 (T) = (0 -971.1995667558366) / Math.Sqrt((4804.923967950588 / (22)) + (509.73819567945236 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.0053551053193308 = (484.30303649446853 - 971.1995667558366) / 484.30303649446853 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithCultureInfo(value: 12345, culture: zh)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 898.4571325558063 > 486.8775696881619.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -31.151630308788373 (T) = (0 -880.0468720964526) / Math.Sqrt((3851.8387943743533 / (22)) + (225.18309869594992 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.9802158396090764 = (444.41967107494037 - 880.0468720964526) / 444.41967107494037 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToString(value: 1.7976931348623157E+308)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1593118260201423 > 766.9323423368817.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -20.498774237382573 (T) = (0 -1192.2649666546527) / Math.Sqrt((9424.862329067753 / (22)) + (1344.7295891405097 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6763148125935687 = (711.2416818711985 - 1192.2649666546527) / 711.2416818711985 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "R")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.189282468771175 > 775.1692993878692.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -20.213821886233962 (T) = (0 -1191.0426490663745) / Math.Sqrt((9418.194189737778 / (22)) + (1295.0355517244182 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.6570176901706628 = (718.7869243228805 - 1191.0426490663745) / 718.7869243228805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in LinqBenchmarks

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Where01LinqMethodNestedX - Duration of single invocation 1.18 secs 2.85 secs 2.42 0.39 False
Where01LinqMethodX - Duration of single invocation 951.74 ms 2.39 secs 2.52 0.43 False
Where01LinqQueryX - Duration of single invocation 940.41 ms 2.45 secs 2.61 0.49 False
Where01ForX - Duration of single invocation 852.46 ms 2.33 secs 2.73 0.26 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'LinqBenchmarks*'

Payloads

Baseline
Compare

Histogram

LinqBenchmarks.Where01LinqMethodNestedX

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.85310249 > 1.2527008232625.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -50.405064106484936 (T) = (0 -2698046373.258309) / Math.Sqrt((127103250156125.08 / (22)) + (9910264738753366 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.285961824939169 = (1180267467.2093904 - 2698046373.258309) / 1180267467.2093904 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqMethodX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3949662426666665 > 994.0707636919232.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -50.85455699817577 (T) = (0 -2448405407.72997) / Math.Sqrt((401249315886602.9 / (22)) + (9419211247394822 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.5922467723965565 = (944510929.2069428 - 2448405407.72997) / 944510929.2069428 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01LinqQueryX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.451369792 > 990.1514559374999.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -91.86629775261788 (T) = (0 -2414696969.84329) / Math.Sqrt((246136740804225.16 / (22)) + (2694381977169009.5 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.556701983015477 = (944457737.305503 - 2414696969.84329) / 944457737.305503 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### LinqBenchmarks.Where01ForX

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.329273445357143 > 899.2718472375.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -61.96576374042858 (T) = (0 -2320753572.9805193) / Math.Sqrt((466226740400332.7 / (22)) + (5884995303706310 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -1.7005272315812625 = (859370550.254229 - 2320753572.9805193) / 859370550.254229 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 505.72 ns 882.83 ns 1.75 0.04 False
ToString - Duration of single invocation 500.21 ns 938.72 ns 1.88 0.02 False
ToString - Duration of single invocation 501.66 ns 930.57 ns 1.86 0.04 False
ToString - Duration of single invocation 498.61 ns 979.63 ns 1.96 0.04 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ja)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 882.8340916027623 > 529.0638936189229.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -24.002431006475316 (T) = (0 -925.9519540484645) / Math.Sqrt((4269.543132499557 / (22)) + (1583.1963701040847 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.9104613368144044 = (484.67453185545304 - 925.9519540484645) / 484.67453185545304 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 938.7228855457048 > 527.816756611863.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -28.429813520243115 (T) = (0 -918.2464644513751) / Math.Sqrt((4292.2575781187525 / (22)) + (382.7119225238837 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.8847973121574654 = (487.18578837545596 - 918.2464644513751) / 487.18578837545596 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: )

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 930.5718056084107 > 527.7321321052906.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -30.58441047561833 (T) = (0 -923.9518655354209) / Math.Sqrt((4205.646401209997 / (22)) + (147.89706934634927 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.8993163627731345 = (486.4654902390177 - 923.9518655354209) / 486.4654902390177 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: fr)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 979.6328102544593 > 522.8718533569131.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -29.26860312321526 (T) = (0 -929.6837338466677) / Math.Sqrt((4228.83022558604 / (22)) + (450.9500033517661 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.9259602053817372 = (482.7118085040592 - 929.6837338466677) / 482.7118085040592 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseSingle - Duration of single invocation 207.55 ns 319.36 ns 1.54 0.09 False
TryParseSingle - Duration of single invocation 214.00 ns 306.58 ns 1.43 0.09 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: 3.4028235E+38)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 319.35581651005725 > 217.59876484307756.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -19.817974381846348 (T) = (0 -319.8859166774356) / Math.Sqrt((697.4354868512286 / (22)) + (30.046264405435164 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.5711905421706135 = (203.59460427728288 - 319.8859166774356) / 203.59460427728288 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Buffers.Text.Tests.Utf8ParserTests.TryParseSingle(value: -3.4028235E+38)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 306.57663839834373 > 217.57358157800252.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -18.9902555857763 (T) = (0 -318.598881107262) / Math.Sqrt((680.683686189914 / (22)) + (62.734054415395335 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.5644903551259054 = (203.6438767828787 - 318.598881107262) / 203.6438767828787 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

performanceautofiler bot commented Jan 17, 2023 •
edited
Loading

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableSortedDictionary - Duration of single invocation 64.15 μs 68.62 μs 1.07 0.06 False
SortedDictionary - Duration of single invocation 79.57 μs 85.66 μs 1.08 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.ImmutableSortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 68.62023159722222 > 67.33129125064036.
IsChangePoint: Marked as a change because one of 10/26/2022 2:40:44 AM, 11/4/2022 2:24:58 AM, 11/21/2022 12:46:21 PM, 12/17/2022 10:44:50 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -9.765407744448533 (T) = (0 -68572.094445548) / Math.Sqrt((1716809.9094114034 / (22)) + (673363.0194669389 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.05612528470978327 = (64927.992387182734 - 68572.094445548) / 64927.992387182734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;.SortedDictionary(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.66220924408013 > 84.19356887419872.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -25.69929673645614 (T) = (0 -85884.25204517214) / Math.Sqrt((722475.6373150594 / (22)) + (134204.3599100633 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.06781134820744064 = (80430.1735408113 - 85884.25204517214) / 80430.1735408113 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Collections.AddGivenSize<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ICollection - Duration of single invocation 3.65 μs 4.13 μs 1.13 0.36 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.AddGivenSize&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.AddGivenSize<Int32>.ICollection(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.126479721709312 > 3.8462311311851063.
IsChangePoint: Marked as a change because one of 10/26/2022 7:25:48 AM, 12/13/2022 6:54:23 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -99.45057724004454 (T) = (0 -4127.121307819194) / Math.Sqrt((177.47222584942986 / (22)) + (148.218204066676 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.12592289330691148 = (3665.5452450190087 - 4127.121307819194) / 3665.5452450190087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Collections.CtorFromCollection<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 115.55 μs 127.32 μs 1.10 0.00 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CtorFromCollection&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CtorFromCollection<Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.31802701502733 > 121.29431068704044.
IsChangePoint: Marked as a change because one of 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -10.938563177935904 (T) = (0 -123236.4380502226) / Math.Sqrt((860846.9549643085 / (22)) + (4082750.4703463702 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.060281936998166724 = (116229.87598857461 - 123236.4380502226) / 116229.87598857461 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedSet - Duration of single invocation 51.57 μs 58.12 μs 1.13 0.04 False
ImmutableSortedSet - Duration of single invocation 64.94 μs 70.37 μs 1.08 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.SortedSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 58.11863026641883 > 54.28737598648927.
IsChangePoint: Marked as a change because one of 10/26/2022 2:40:44 AM, 11/4/2022 2:24:58 AM, 11/18/2022 1:00:00 AM, 12/5/2022 9:44:53 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -9.404932050807652 (T) = (0 -58434.17745984392) / Math.Sqrt((2392623.7137752 / (22)) + (2573974.0290893996 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.10403047651227362 = (52928.047461554204 - 58434.17745984392) / 52928.047461554204 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;Int32&gt;.ImmutableSortedSet(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 70.37089360986548 > 68.0914001697687.
IsChangePoint: Marked as a change because one of 11/24/2022 8:21:24 PM, 12/5/2022 9:44:53 PM, 1/10/2023 12:53:48 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -12.737965482114367 (T) = (0 -69885.17877482472) / Math.Sqrt((1367297.7751395602 / (18)) + (741807.0316552139 / (15))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (15) - 2, .025) and -0.06900305358375798 = (65374.16197319506 - 69885.17877482472) / 65374.16197319506 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 74.28 μs 80.13 μs 1.08 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 80.1345897102369 > 78.05584157879146.
IsChangePoint: Marked as a change because one of 12/17/2022 10:44:50 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -17.07747269504395 (T) = (0 -79621.54846541671) / Math.Sqrt((2144329.5302831987 / (22)) + (57326.01563441055 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.07380091613286259 = (74149.26479310718 - 79621.54846541671) / 74149.26479310718 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in StoreBlock.LocalAddress

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
InitBlockAllOnes8 - Duration of single invocation 6.45 ns 8.12 ns 1.26 0.10 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'StoreBlock.LocalAddress*'

Payloads

Baseline
Compare

Histogram

StoreBlock.LocalAddress.InitBlockAllOnes8

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.115781234818515 > 6.781785283174364.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -6.95330789674373 (T) = (0 -8.139328256314805) / Math.Sqrt((0.0891423392559746 / (22)) + (0.49546664681618247 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.23348070825149972 = (6.598666847293117 - 8.139328256314805) / 6.598666847293117 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Xml.Tests.Perf_XmlConvert

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
DateTime_ToString - Duration of single invocation 242.06 ns 325.33 ns 1.34 0.08 False
DateTime_ToString_Unspecified - Duration of single invocation 230.89 ns 308.52 ns 1.34 0.07 False
DateTime_ToString_RoundtripKind - Duration of single invocation 224.49 ns 319.02 ns 1.42 0.16 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Xml.Tests.Perf_XmlConvert*'

Payloads

Baseline
Compare

Histogram

System.Xml.Tests.Perf_XmlConvert.DateTime_ToString

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 325.32667860654584 > 253.64821885835784.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -15.932467434321515 (T) = (0 -334.61575498820997) / Math.Sqrt((702.4866570339464 / (22)) + (73.4287044453567 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.4201458678818771 = (235.620694011724 - 334.61575498820997) / 235.620694011724 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_Unspecified

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 308.5197129109659 > 238.44000547004237.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -15.767888881187814 (T) = (0 -311.5484134137803) / Math.Sqrt((570.6296573165 / (22)) + (72.643946730693 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.40590466556631905 = (221.59995698448301 - 311.5484134137803) / 221.59995698448301 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Xml.Tests.Perf_XmlConvert.DateTime_ToString_RoundtripKind

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 319.01886459086376 > 235.68364322224224.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -17.019200840222734 (T) = (0 -312.26815228985987) / Math.Sqrt((620.2774104133423 / (22)) + (15.998396113202944 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.4220032832193028 = (219.5973497212394 - 312.26815228985987) / 219.5973497212394 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Globalization.Tests.Perf_DateTimeCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringHebrewIsrael - Duration of single invocation 1.16 μs 1.76 μs 1.52 0.05 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_DateTimeCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_DateTimeCultureInfo.ToStringHebrewIsrael

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7568816267518712 > 1.2109952867596425.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -18.028063803285818 (T) = (0 -1717.0591784316823) / Math.Sqrt((21547.31325096694 / (22)) + (1191.2111738490826 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.5296957581199688 = (1122.484107913058 - 1717.0591784316823) / 1122.484107913058 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WhereSingle_LastElementMatches - Duration of single invocation 570.71 ns 635.57 ns 1.11 0.07 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.WhereSingle_LastElementMatches(input: Array)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 635.5673427994302 > 632.8152577648996.
IsChangePoint: Marked as a change because one of 10/26/2022 2:40:44 AM, 11/11/2022 5:53:56 PM, 11/24/2022 8:21:24 PM, 12/5/2022 9:44:53 PM, 12/21/2022 10:31:08 AM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -4.247792372544856 (T) = (0 -655.6406279101207) / Math.Sqrt((1156.6384458288628 / (22)) + (304.16109448981035 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.06160508853681489 = (617.5937125676127 - 655.6406279101207) / 617.5937125676127 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetDateTime - Duration of single invocation 16.36 μs 26.33 μs 1.61 0.03 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetDateTime

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.330293795674212 > 17.09356314844933.
IsChangePoint: Marked as a change because one of 1/5/2023 4:05:11 PM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -19.0965649938346 (T) = (0 -25197.359541300753) / Math.Sqrt((3999122.4367749896 / (22)) + (669357.8972240329 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.5956895615708371 = (15790.890752268779 - 25197.359541300753) / 15790.890752268779 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture arm64
OS ubuntu 20.04
Baseline 52381a1f88c5b3c6c0bde46c7f6d2b4dba49aab5
Compare bd21232171668c318bdcd3446f27ee663a74e7bb
Diff Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedDictionary - Duration of single invocation 79.66 μs 85.44 μs 1.07 0.02 False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.SortedDictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 85.44172003642986 > 83.57720743574134.
IsChangePoint: Marked as a change because one of 10/26/2022 2:40:44 AM, 1/11/2023 1:25:06 PM, 1/17/2023 2:54:07 AM falls between 1/8/2023 8:00:22 AM and 1/17/2023 2:54:07 AM.
IsRegressionStdDev: Marked as regression because -12.235247731540113 (T) = (0 -84872.3847370097) / Math.Sqrt((1883525.7199351727 / (22)) + (987497.4007284476 / (11))) is less than -2.0395134463927462 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (22) + (11) - 2, .025) and -0.06425251220220381 = (79748.3527301125 - 84872.3847370097) / 79748.3527301125 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added ampere arm64 PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Jan 17, 2023
@lewing
Copy link
Member

lewing commented Jan 29, 2023

the regression on the 14th should be fixed by dotnet/runtime#81201

@naricc
Copy link

naricc commented Jan 31, 2023

Leaving this open to check fix.

@naricc naricc removed the untriaged label Jan 31, 2023
@naricc naricc self-assigned this Jan 31, 2023
@naricc naricc added the LinkedFromMaster Is this issue linked from the master perf issue label Jan 31, 2023
@naricc naricc mentioned this issue Jan 31, 2023
Closed
@kotlarmilos
Copy link
Member

Mitigated by dotnet/runtime#81695

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@naricc naricc

Labels
ampere arch-arm64 branch-refs/heads/main kind-micro_mono LinkedFromMaster Is this issue linked from the master perf issue mono-aot os-linux perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-mono
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lewing @kotlarmilos @naricc