Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NFR Test Results for NGF version edge #2408

Merged
merged 3 commits into from
Aug 15, 2024
Merged

NFR Test Results for NGF version edge #2408

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
43f829e
NFR Test Results for NGF version edge
nginx-bot Aug 15, 2024
12078fc
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 15, 2024
c15d3a0
Merge branch 'main' into tests/nfr-tests-edge
kate-osborn Aug 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 24 additions & 24 deletions tests/results/dp-perf/edge/edge-oss.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ NGINX Plus: false

NGINX Gateway Fabric:

- Commit: 9a85dbcc0797e31557a3731688795aa166ee0f96
- Date: 2024-08-13T21:12:05Z
- Commit: 3c029b1417c1f89f2a29aeef07f47078640e28b2
- Date: 2024-08-15T00:04:25Z
- Dirty: false

GKE Cluster:
Expand All @@ -23,24 +23,24 @@ GKE Cluster:
## Test1: Running latte path based routing

```text
Requests [total, rate, throughput] 30000, 1000.01, 999.79
Duration [total, attack, wait] 30s, 30s, 701.442µs
Latencies [min, mean, 50, 90, 95, 99, max] 341.042µs, 737.394µs, 718.76µs, 827.697µs, 866.432µs, 962.831µs, 16.145ms
Bytes In [total, mean] 4769946, 159.00
Requests [total, rate, throughput] 30000, 1000.03, 999.91
Duration [total, attack, wait] 30s, 29.999s, 586.041µs
Latencies [min, mean, 50, 90, 95, 99, max] 459.105µs, 652.462µs, 640.858µs, 731.955µs, 766.916µs, 868.446µs, 5.683ms
Bytes In [total, mean] 4829967, 161.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 99.98%
Status Codes [code:count] 200:29994 502:6
Success [ratio] 99.99%
Status Codes [code:count] 200:29997 502:3
Error Set:
502 Bad Gateway
```

## Test2: Running coffee header based routing

```text
Requests [total, rate, throughput] 30000, 1000.02, 1000.00
Duration [total, attack, wait] 30s, 29.999s, 687.856µs
Latencies [min, mean, 50, 90, 95, 99, max] 554.445µs, 759.558µs, 740.252µs, 848.701µs, 889.401µs, 997.689µs, 22.657ms
Bytes In [total, mean] 4800000, 160.00
Requests [total, rate, throughput] 30000, 1000.03, 1000.01
Duration [total, attack, wait] 30s, 29.999s, 637.326µs
Latencies [min, mean, 50, 90, 95, 99, max] 474.92µs, 694.704µs, 678.367µs, 786.23µs, 826.769µs, 934.323µs, 12.252ms
Bytes In [total, mean] 4860000, 162.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand All @@ -50,10 +50,10 @@ Error Set:
## Test3: Running coffee query based routing

```text
Requests [total, rate, throughput] 30000, 1000.04, 1000.02
Duration [total, attack, wait] 29.999s, 29.999s, 726.197µs
Latencies [min, mean, 50, 90, 95, 99, max] 550.722µs, 769.327µs, 747.389µs, 857.742µs, 898.394µs, 1.015ms, 13.026ms
Bytes In [total, mean] 5040000, 168.00
Requests [total, rate, throughput] 30000, 1000.02, 999.99
Duration [total, attack, wait] 30s, 30s, 676.679µs
Latencies [min, mean, 50, 90, 95, 99, max] 525.067µs, 708.22µs, 693.208µs, 804.086µs, 849.031µs, 958.899µs, 10.449ms
Bytes In [total, mean] 5100000, 170.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand All @@ -63,10 +63,10 @@ Error Set:
## Test4: Running tea GET method based routing

```text
Requests [total, rate, throughput] 30000, 1000.00, 999.98
Duration [total, attack, wait] 30.001s, 30s, 705.904µs
Latencies [min, mean, 50, 90, 95, 99, max] 539.272µs, 750.546µs, 734.78µs, 841.798µs, 880.351µs, 979.455µs, 16.567ms
Bytes In [total, mean] 4710000, 157.00
Requests [total, rate, throughput] 30000, 1000.01, 999.99
Duration [total, attack, wait] 30s, 30s, 637.156µs
Latencies [min, mean, 50, 90, 95, 99, max] 512.28µs, 691.498µs, 675.513µs, 780.523µs, 817.834µs, 910.375µs, 9.619ms
Bytes In [total, mean] 4770000, 159.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand All @@ -76,10 +76,10 @@ Error Set:
## Test5: Running tea POST method based routing

```text
Requests [total, rate, throughput] 30000, 1000.04, 1000.01
Duration [total, attack, wait] 30s, 29.999s, 772.597µs
Latencies [min, mean, 50, 90, 95, 99, max] 541.485µs, 752.07µs, 740.31µs, 847.409µs, 886.955µs, 981.786µs, 12.398ms
Bytes In [total, mean] 4710000, 157.00
Requests [total, rate, throughput] 30000, 1000.00, 999.98
Duration [total, attack, wait] 30.001s, 30s, 773.157µs
Latencies [min, mean, 50, 90, 95, 99, max] 503.29µs, 669.335µs, 655.455µs, 754.428µs, 792.559µs, 892.505µs, 10.652ms
Bytes In [total, mean] 4770000, 159.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand Down
42 changes: 21 additions & 21 deletions tests/results/dp-perf/edge/edge-plus.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ NGINX Plus: true

NGINX Gateway Fabric:

- Commit: 9a85dbcc0797e31557a3731688795aa166ee0f96
- Date: 2024-08-13T21:12:05Z
- Commit: 3c029b1417c1f89f2a29aeef07f47078640e28b2
- Date: 2024-08-15T00:04:25Z
- Dirty: false

GKE Cluster:
Expand All @@ -23,10 +23,10 @@ GKE Cluster:
## Test1: Running latte path based routing

```text
Requests [total, rate, throughput] 30000, 1000.03, 1000.01
Duration [total, attack, wait] 30s, 29.999s, 682.261µs
Latencies [min, mean, 50, 90, 95, 99, max] 515.366µs, 729.019µs, 712.033µs, 843.484µs, 895.476µs, 1.019ms, 9.154ms
Bytes In [total, mean] 4800000, 160.00
Requests [total, rate, throughput] 30000, 1000.01, 999.99
Duration [total, attack, wait] 30s, 30s, 673.045µs
Latencies [min, mean, 50, 90, 95, 99, max] 487.307µs, 681.853µs, 664.969µs, 758.583µs, 798.223µs, 915.644µs, 11.35ms
Bytes In [total, mean] 4740000, 158.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand All @@ -37,9 +37,9 @@ Error Set:

```text
Requests [total, rate, throughput] 30000, 1000.03, 1000.01
Duration [total, attack, wait] 30s, 29.999s, 599.51µs
Latencies [min, mean, 50, 90, 95, 99, max] 540.423µs, 740.62µs, 724.16µs, 857.72µs, 911.755µs, 1.042ms, 7.517ms
Bytes In [total, mean] 4830000, 161.00
Duration [total, attack, wait] 30s, 29.999s, 687.489µs
Latencies [min, mean, 50, 90, 95, 99, max] 508.353µs, 694.959µs, 683.138µs, 780.78µs, 818.701µs, 943.447µs, 7.212ms
Bytes In [total, mean] 4770000, 159.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand All @@ -49,10 +49,10 @@ Error Set:
## Test3: Running coffee query based routing

```text
Requests [total, rate, throughput] 30000, 1000.04, 1000.02
Duration [total, attack, wait] 29.999s, 29.999s, 683.488µs
Latencies [min, mean, 50, 90, 95, 99, max] 523.729µs, 756.497µs, 740.664µs, 870.743µs, 921.816µs, 1.062ms, 12.337ms
Bytes In [total, mean] 5070000, 169.00
Requests [total, rate, throughput] 30000, 1000.01, 999.99
Duration [total, attack, wait] 30s, 30s, 607.004µs
Latencies [min, mean, 50, 90, 95, 99, max] 525.289µs, 702.559µs, 688.194µs, 791.113µs, 832.688µs, 952.855µs, 8.97ms
Bytes In [total, mean] 5010000, 167.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand All @@ -62,10 +62,10 @@ Error Set:
## Test4: Running tea GET method based routing

```text
Requests [total, rate, throughput] 30000, 1000.01, 999.98
Duration [total, attack, wait] 30.001s, 30s, 762.485µs
Latencies [min, mean, 50, 90, 95, 99, max] 499.61µs, 744.201µs, 726.564µs, 862.287µs, 914.928µs, 1.054ms, 7.531ms
Bytes In [total, mean] 4740000, 158.00
Requests [total, rate, throughput] 30000, 1000.02, 999.99
Duration [total, attack, wait] 30s, 30s, 682.661µs
Latencies [min, mean, 50, 90, 95, 99, max] 520.536µs, 700.459µs, 686.666µs, 790.333µs, 830.005µs, 947.376µs, 13.797ms
Bytes In [total, mean] 4680000, 156.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand All @@ -75,10 +75,10 @@ Error Set:
## Test5: Running tea POST method based routing

```text
Requests [total, rate, throughput] 30000, 1000.01, 999.99
Duration [total, attack, wait] 30s, 30s, 651.893µs
Latencies [min, mean, 50, 90, 95, 99, max] 532.673µs, 751.908µs, 734.466µs, 872.179µs, 927.874µs, 1.068ms, 6.327ms
Bytes In [total, mean] 4740000, 158.00
Requests [total, rate, throughput] 30000, 1000.02, 1000.00
Duration [total, attack, wait] 30s, 29.999s, 734.519µs
Latencies [min, mean, 50, 90, 95, 99, max] 517.019µs, 687.341µs, 674.327µs, 775.376µs, 819.395µs, 936.514µs, 8.43ms
Bytes In [total, mean] 4680000, 156.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:30000
Expand Down
28 changes: 14 additions & 14 deletions tests/results/ngf-upgrade/edge/edge-oss.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ NGINX Plus: false

NGINX Gateway Fabric:

- Commit: 9a85dbcc0797e31557a3731688795aa166ee0f96
- Date: 2024-08-13T21:12:05Z
- Commit: 3c029b1417c1f89f2a29aeef07f47078640e28b2
- Date: 2024-08-15T00:04:25Z
- Dirty: false

GKE Cluster:
Expand All @@ -20,32 +20,32 @@ GKE Cluster:
- Zone: us-west1-b
- Instance Type: n2d-standard-16

## Test: Send https /tea traffic
## Test: Send http /coffee traffic

```text
Requests [total, rate, throughput] 6000, 100.02, 100.02
Duration [total, attack, wait] 59.991s, 59.99s, 763.57µs
Latencies [min, mean, 50, 90, 95, 99, max] 631.935µs, 834.342µs, 818.106µs, 916.549µs, 953.194µs, 1.06ms, 12.108ms
Bytes In [total, mean] 918000, 153.00
Requests [total, rate, throughput] 6000, 100.02, 100.01
Duration [total, attack, wait] 59.991s, 59.99s, 879.288µs
Latencies [min, mean, 50, 90, 95, 99, max] 624.858µs, 833.765µs, 811.136µs, 926.889µs, 971.561µs, 1.103ms, 12.54ms
Bytes In [total, mean] 962028, 160.34
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:6000
Error Set:
```

![https-oss.png](https-oss.png)
![http-oss.png](http-oss.png)

## Test: Send http /coffee traffic
## Test: Send https /tea traffic

```text
Requests [total, rate, throughput] 6000, 100.02, 100.02
Duration [total, attack, wait] 59.991s, 59.99s, 761.784µs
Latencies [min, mean, 50, 90, 95, 99, max] 605.288µs, 828.183µs, 816.906µs, 927.903µs, 968.164µs, 1.089ms, 3.097ms
Bytes In [total, mean] 960000, 160.00
Requests [total, rate, throughput] 6000, 100.02, 100.01
Duration [total, attack, wait] 59.991s, 59.99s, 877.076µs
Latencies [min, mean, 50, 90, 95, 99, max] 630.112µs, 888.373µs, 853.502µs, 995.531µs, 1.046ms, 1.215ms, 12.537ms
Bytes In [total, mean] 918000, 153.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:6000
Error Set:
```

![http-oss.png](http-oss.png)
![https-oss.png](https-oss.png)
24 changes: 12 additions & 12 deletions tests/results/ngf-upgrade/edge/edge-plus.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ NGINX Plus: true

NGINX Gateway Fabric:

- Commit: 9a85dbcc0797e31557a3731688795aa166ee0f96
- Date: 2024-08-13T21:12:05Z
- Commit: 3c029b1417c1f89f2a29aeef07f47078640e28b2
- Date: 2024-08-15T00:04:25Z
- Dirty: false

GKE Cluster:
Expand All @@ -20,32 +20,32 @@ GKE Cluster:
- Zone: us-west1-b
- Instance Type: n2d-standard-16

## Test: Send https /tea traffic
## Test: Send http /coffee traffic

```text
Requests [total, rate, throughput] 6000, 100.02, 100.01
Duration [total, attack, wait] 59.991s, 59.99s, 921.041µs
Latencies [min, mean, 50, 90, 95, 99, max] 616.005µs, 851.643µs, 821.167µs, 986.363µs, 1.055ms, 1.241ms, 12.227ms
Bytes In [total, mean] 921995, 153.67
Duration [total, attack, wait] 59.992s, 59.991s, 994.033µs
Latencies [min, mean, 50, 90, 95, 99, max] 618.031µs, 867.451µs, 853.111µs, 978.461µs, 1.029ms, 1.281ms, 3.592ms
Bytes In [total, mean] 960000, 160.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:6000
Error Set:
```

![https-plus.png](https-plus.png)
![http-plus.png](http-plus.png)

## Test: Send http /coffee traffic
## Test: Send https /tea traffic

```text
Requests [total, rate, throughput] 6000, 100.02, 100.01
Duration [total, attack, wait] 59.991s, 59.99s, 902.051µs
Latencies [min, mean, 50, 90, 95, 99, max] 483.51µs, 832.015µs, 814.172µs, 976.685µs, 1.031ms, 1.193ms, 12.788ms
Bytes In [total, mean] 966000, 161.00
Duration [total, attack, wait] 59.992s, 59.991s, 1.001ms
Latencies [min, mean, 50, 90, 95, 99, max] 699.522µs, 977.274µs, 952.901µs, 1.125ms, 1.185ms, 1.338ms, 7.794ms
Bytes In [total, mean] 918000, 153.00
Bytes Out [total, mean] 0, 0.00
Success [ratio] 100.00%
Status Codes [code:count] 200:6000
Error Set:
```

![http-plus.png](http-plus.png)
![https-plus.png](https-plus.png)
Binary file modified tests/results/ngf-upgrade/edge/http-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/ngf-upgrade/edge/http-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/ngf-upgrade/edge/https-oss.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified tests/results/ngf-upgrade/edge/https-plus.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Loading