-
Notifications
You must be signed in to change notification settings - Fork 101
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Browse files
Browse the repository at this point in the history
Update with NFR test results for NGF version 1.6.0 ["oss","plus"] Co-authored-by: nginx-bot <68849795+nginx-bot@users.noreply.github.com>
- Loading branch information
Showing
89 changed files
with
1,958 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,92 @@ | ||
# Results | ||
|
||
## Test environment | ||
|
||
NGINX Plus: false | ||
|
||
NGINX Gateway Fabric: | ||
|
||
- Commit: b61c61d3f9ca29c6eb93ce9b44e652c9a521b3a4 | ||
- Date: 2025-01-13T16:47:24Z | ||
- Dirty: false | ||
|
||
GKE Cluster: | ||
|
||
- Node count: 12 | ||
- k8s version: v1.30.6-gke.1596000 | ||
- vCPUs per node: 16 | ||
- RAM per node: 65853984Ki | ||
- Max pods per node: 110 | ||
- Zone: us-west1-b | ||
- Instance Type: n2d-standard-16 | ||
|
||
## Summary: | ||
|
||
- Performance stayed consistent with 1.5.0 results. Average latency slightly increased across all routing methods. | ||
- Errors that occurred are consistent with errors that occurred in the previous results. | ||
|
||
## Test1: Running latte path based routing | ||
|
||
```text | ||
Requests [total, rate, throughput] 30000, 1000.04, 998.38 | ||
Duration [total, attack, wait] 30s, 29.999s, 844.157µs | ||
Latencies [min, mean, 50, 90, 95, 99, max] 309.6µs, 718.534µs, 681.308µs, 786.633µs, 827.114µs, 971.115µs, 18.386ms | ||
Bytes In [total, mean] 4741568, 158.05 | ||
Bytes Out [total, mean] 0, 0.00 | ||
Success [ratio] 99.84% | ||
Status Codes [code:count] 200:29951 503:49 | ||
Error Set: | ||
503 Service Temporarily Unavailable | ||
``` | ||
|
||
## Test2: Running coffee header based routing | ||
|
||
```text | ||
Requests [total, rate, throughput] 30000, 1000.01, 999.98 | ||
Duration [total, attack, wait] 30.001s, 30s, 708.18µs | ||
Latencies [min, mean, 50, 90, 95, 99, max] 519.443µs, 728.205µs, 716.283µs, 820.709µs, 859.918µs, 962.843µs, 6.974ms | ||
Bytes In [total, mean] 4770000, 159.00 | ||
Bytes Out [total, mean] 0, 0.00 | ||
Success [ratio] 100.00% | ||
Status Codes [code:count] 200:30000 | ||
Error Set: | ||
``` | ||
|
||
## Test3: Running coffee query based routing | ||
|
||
```text | ||
Requests [total, rate, throughput] 30000, 1000.02, 1000.00 | ||
Duration [total, attack, wait] 30s, 29.999s, 746.102µs | ||
Latencies [min, mean, 50, 90, 95, 99, max] 533.22µs, 735.075µs, 722.549µs, 830.432µs, 871.714µs, 973.911µs, 6.9ms | ||
Bytes In [total, mean] 5010000, 167.00 | ||
Bytes Out [total, mean] 0, 0.00 | ||
Success [ratio] 100.00% | ||
Status Codes [code:count] 200:30000 | ||
Error Set: | ||
``` | ||
|
||
## Test4: Running tea GET method based routing | ||
|
||
```text | ||
Requests [total, rate, throughput] 30000, 1000.02, 1000.00 | ||
Duration [total, attack, wait] 30s, 29.999s, 737.741µs | ||
Latencies [min, mean, 50, 90, 95, 99, max] 528.445µs, 724.715µs, 711.435µs, 816.76µs, 859.214µs, 967.474µs, 11.985ms | ||
Bytes In [total, mean] 4680000, 156.00 | ||
Bytes Out [total, mean] 0, 0.00 | ||
Success [ratio] 100.00% | ||
Status Codes [code:count] 200:30000 | ||
Error Set: | ||
``` | ||
|
||
## Test5: Running tea POST method based routing | ||
|
||
```text | ||
Requests [total, rate, throughput] 30000, 1000.03, 1000.01 | ||
Duration [total, attack, wait] 30s, 29.999s, 643.191µs | ||
Latencies [min, mean, 50, 90, 95, 99, max] 538.368µs, 728.96µs, 714.974µs, 818.991µs, 860.142µs, 971.866µs, 11.543ms | ||
Bytes In [total, mean] 4680000, 156.00 | ||
Bytes Out [total, mean] 0, 0.00 | ||
Success [ratio] 100.00% | ||
Status Codes [code:count] 200:30000 | ||
Error Set: | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
# Results | ||
|
||
## Test environment | ||
|
||
NGINX Plus: true | ||
|
||
NGINX Gateway Fabric: | ||
|
||
- Commit: b61c61d3f9ca29c6eb93ce9b44e652c9a521b3a4 | ||
- Date: 2025-01-13T16:47:24Z | ||
- Dirty: false | ||
|
||
GKE Cluster: | ||
|
||
- Node count: 12 | ||
- k8s version: v1.30.6-gke.1596000 | ||
- vCPUs per node: 16 | ||
- RAM per node: 65853984Ki | ||
- Max pods per node: 110 | ||
- Zone: us-west1-b | ||
- Instance Type: n2d-standard-16 | ||
|
||
## Summary: | ||
|
||
- Performance stayed consistent with 1.5.0 results. Average latency slightly increased across all routing methods. | ||
|
||
## Test1: Running latte path based routing | ||
|
||
```text | ||
Requests [total, rate, throughput] 30000, 1000.03, 1000.00 | ||
Duration [total, attack, wait] 30s, 29.999s, 744.047µs | ||
Latencies [min, mean, 50, 90, 95, 99, max] 535.49µs, 722.768µs, 702.708µs, 807.78µs, 849.575µs, 981.854µs, 21.041ms | ||
Bytes In [total, mean] 4770000, 159.00 | ||
Bytes Out [total, mean] 0, 0.00 | ||
Success [ratio] 100.00% | ||
Status Codes [code:count] 200:30000 | ||
Error Set: | ||
``` | ||
|
||
## Test2: Running coffee header based routing | ||
|
||
```text | ||
Requests [total, rate, throughput] 30000, 1000.01, 999.98 | ||
Duration [total, attack, wait] 30s, 30s, 718.788µs | ||
Latencies [min, mean, 50, 90, 95, 99, max] 558.587µs, 766.304µs, 750.921µs, 866.313µs, 907.422µs, 1.022ms, 10.872ms | ||
Bytes In [total, mean] 4800000, 160.00 | ||
Bytes Out [total, mean] 0, 0.00 | ||
Success [ratio] 100.00% | ||
Status Codes [code:count] 200:30000 | ||
Error Set: | ||
``` | ||
|
||
## Test3: Running coffee query based routing | ||
|
||
```text | ||
Requests [total, rate, throughput] 30000, 1000.02, 999.99 | ||
Duration [total, attack, wait] 30s, 30s, 733.649µs | ||
Latencies [min, mean, 50, 90, 95, 99, max] 572.624µs, 771.492µs, 758.449µs, 867.491µs, 907.997µs, 1.032ms, 11.906ms | ||
Bytes In [total, mean] 5040000, 168.00 | ||
Bytes Out [total, mean] 0, 0.00 | ||
Success [ratio] 100.00% | ||
Status Codes [code:count] 200:30000 | ||
Error Set: | ||
``` | ||
|
||
## Test4: Running tea GET method based routing | ||
|
||
```text | ||
Requests [total, rate, throughput] 30000, 1000.03, 1000.00 | ||
Duration [total, attack, wait] 30s, 29.999s, 712.155µs | ||
Latencies [min, mean, 50, 90, 95, 99, max] 549.224µs, 760.423µs, 746.75µs, 853.877µs, 894.554µs, 1.008ms, 8.12ms | ||
Bytes In [total, mean] 4710000, 157.00 | ||
Bytes Out [total, mean] 0, 0.00 | ||
Success [ratio] 100.00% | ||
Status Codes [code:count] 200:30000 | ||
Error Set: | ||
``` | ||
|
||
## Test5: Running tea POST method based routing | ||
|
||
```text | ||
Requests [total, rate, throughput] 30000, 1000.00, 999.98 | ||
Duration [total, attack, wait] 30.001s, 30s, 778.666µs | ||
Latencies [min, mean, 50, 90, 95, 99, max] 544.486µs, 762.077µs, 748.375µs, 852.722µs, 893.014µs, 1.009ms, 9.632ms | ||
Bytes In [total, mean] 4710000, 157.00 | ||
Bytes Out [total, mean] 0, 0.00 | ||
Success [ratio] 100.00% | ||
Status Codes [code:count] 200:30000 | ||
Error Set: | ||
``` |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,97 @@ | ||
# Results | ||
|
||
## Test environment | ||
|
||
NGINX Plus: false | ||
|
||
NGINX Gateway Fabric: | ||
|
||
- Commit: 8be03e1fc5161a2b1bc0962fb0d8732114a9093d | ||
- Date: 2025-01-14T18:57:38Z | ||
- Dirty: true | ||
|
||
GKE Cluster: | ||
|
||
- Node count: 3 | ||
- k8s version: v1.30.6-gke.1596000 | ||
- vCPUs per node: 2 | ||
- RAM per node: 4018128Ki | ||
- Max pods per node: 110 | ||
- Zone: us-central1-c | ||
- Instance Type: e2-medium | ||
|
||
## Traffic | ||
|
||
HTTP: | ||
|
||
```text | ||
Running 5760m test @ http://cafe.example.com/coffee | ||
2 threads and 100 connections | ||
Thread Stats Avg Stdev Max +/- Stdev | ||
Latency 189.49ms 147.10ms 2.00s 78.44% | ||
Req/Sec 293.54 193.84 1.95k 66.59% | ||
198532845 requests in 5760.00m, 67.91GB read | ||
Socket errors: connect 0, read 309899, write 63, timeout 2396 | ||
Requests/sec: 574.46 | ||
Transfer/sec: 206.05KB | ||
``` | ||
|
||
HTTPS: | ||
|
||
```text | ||
Running 5760m test @ https://cafe.example.com/tea | ||
2 threads and 100 connections | ||
Thread Stats Avg Stdev Max +/- Stdev | ||
Latency 179.59ms 121.50ms 1.99s 67.56% | ||
Req/Sec 292.54 193.88 2.39k 66.47% | ||
197890521 requests in 5760.00m, 66.57GB read | ||
Socket errors: connect 176, read 303560, write 0, timeout 7 | ||
Requests/sec: 572.60 | ||
Transfer/sec: 201.98KB | ||
``` | ||
|
||
### Logs | ||
|
||
No error logs in nginx-gateway. | ||
|
||
No error logs in nginx. | ||
|
||
|
||
### Key Metrics | ||
|
||
#### Containers memory | ||
|
||
![oss-memory.png](oss-memory.png) | ||
|
||
#### NGF Container Memory | ||
|
||
![oss-ngf-memory.png](oss-ngf-memory.png) | ||
|
||
### Containers CPU | ||
|
||
![oss-cpu.png](oss-cpu.png) | ||
|
||
### NGINX metrics | ||
|
||
![oss-stub-status.png](oss-stub-status.png) | ||
|
||
### Reloads | ||
|
||
Rate of reloads - successful and errors: | ||
|
||
![oss-reloads.png](oss-reloads.png) | ||
|
||
Reload spikes correspond to 1 hour periods of backend re-rollouts. | ||
|
||
No reloads finished with an error. | ||
|
||
Reload time distribution - counts: | ||
|
||
![oss-reload-time.png](oss-reload-time.png) | ||
|
||
|
||
## Comparison with previous results | ||
|
||
Graphs look similar to 1.5.0 results. There is a color change swap in a few graphs which is a little confusing. | ||
NGINX container memory decreased dramatically. NGINX Stub Status graph is confusing to interpret, which can make it seem | ||
quite different to the 1.5.0 results, but it is similar, only with an increase in requests. |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.