Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding P50, P90 and P100 percentiles to benchmarks #2411

Merged
merged 2 commits into from
Nov 21, 2023
Merged

Conversation

jmprieur
Copy link
Contributor

Adding P50, P90 and P100 percentiles to benchmarks

Adds 3 columns to the benchmark summaries to display the P50, P90 and P100 percentiles

Description

The benchmark summaries now contains the pecentiles

Runtime = ; GC =
Mean = 10.711 us, StdErr = 0.057 us (0.53%), N = 29, StdDev = 0.308 us
Min = 10.261 us, Q1 = 10.438 us, Median = 10.682 us, Q3 = 10.955 us, Max = 11.354 us
IQR = 0.517 us, LowerFence = 9.662 us, UpperFence = 11.730 us
ConfidenceInterval = [10.500 us; 10.921 us] (CI 99.9%), Margin = 0.210 us (1.96% of Mean)
Skewness = 0.36, Kurtosis = 1.97, MValue = 2
-------------------- Histogram --------------------
[10.235 us ; 10.532 us) | @@@@@@@@@
[10.532 us ; 10.795 us) | @@@@@@@@@@@
[10.795 us ; 11.203 us) | @@@@@@@@
[11.203 us ; 11.486 us) | @
---------------------------------------------------

// * Summary *

BenchmarkDotNet=v0.13.5, OS=Windows 11 (10.0.22631.2715)
Intel Core i7-1065G7 CPU 1.30GHz, 1 CPU, 8 logical and 4 physical cores
.NET SDK=8.0.100
  [Host] : .NET 6.0.25 (6.0.2523.51912), X64 RyuJIT AVX2

Job=MediumRun  Toolchain=InProcessEmitToolchain  IterationCount=15
LaunchCount=2  WarmupCount=10
Method Mean Error StdDev P50 P99 P100 Gen0 Allocated
JsonWebTokenHandler_ValidateTokenAsync 1.471 us 0.0600 us 0.0898 us 1.46 us 1.57 us 1.66 us 0.2747 1.13 KB
JwtSecurityTokenHandler_ValidateTokenAsync 10.711 us 0.2103 us 0.3083 us 10.68 us 11.15 us 11.35 us 0.3204 1.34 KB

Fixes #2410

Copy link
Collaborator

@jennyf19 jennyf19 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

awesome!
:shipit:


namespace Microsoft.IdentityModel.Benchmarks
{
// Define custom columns for P50 and P99 latency
public class P50Column : IColumn
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://benchmarkdotnet.org/articles/features/statistics.html

Most of these look like they already exist in StatisticColumn?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. The columns explicitly get the ones we want and not the others

Copy link
Contributor

@keegan-caruso keegan-caruso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few comments are off, otherwise LGTM

@jmprieur jmprieur merged commit e25592a into dev Nov 21, 2023
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Benchmarks should display P90 and P99,9 percentiles
3 participants