diff --git a/Modules/III-k6-Intermediate/08-Setting-load-profiles-with-executors/Constant-Arrival-Rate-Exercises.md b/Modules/III-k6-Intermediate/08-Setting-load-profiles-with-executors/Constant-Arrival-Rate-Exercises.md index 1bc6cb0..2a76a22 100644 --- a/Modules/III-k6-Intermediate/08-Setting-load-profiles-with-executors/Constant-Arrival-Rate-Exercises.md +++ b/Modules/III-k6-Intermediate/08-Setting-load-profiles-with-executors/Constant-Arrival-Rate-Exercises.md @@ -2,9 +2,11 @@ As noted in [Setting load profiles with executors](../08-Setting-load-profiles-with-executors.md#Constant-Arrival-Rate), the _Constant Arrival Rate_ executor has a primary focus on the _iteration rate_ being applied over a specified timeframe. +This is a scenario typically seen with [load testing an API](https://k6.io/docs/testing-guides/api-load-testing/) when there is a need to simulate a constant request rate for a particular API endpoint. + ## Exercises -For our exercises, we're going to start by using a very basic script that simply performs an HTTP request and then waits one second before completing the test iteration. We're providing some console output as things change. +Our exercises start with a basic script. It runs an HTTP request on each test iteration and tries to achieve 50 requests per time unit (which defaults to 1 second). We provide some console output as things change. ### Creating our script @@ -12,15 +14,14 @@ Let's begin by implementing our test script with the minimally required configur ```js import http from 'k6/http'; -import { sleep } from 'k6'; export const options = { scenarios: { k6_workshop: { executor: 'constant-arrival-rate', - rate: 10, + rate: 50, duration: '30s', - preAllocatedVUs: 5, + preAllocatedVUs: 2, }, }, }; @@ -28,13 +29,14 @@ export const options = { export default function () { console.log(`[VU: ${__VU}, iteration: ${__ITER}] Starting iteration...`); http.get('https://test.k6.io/contacts.php'); - sleep(1); } ``` We're starting with the bare minimum to use the executor. Compared to previous executors, this has a bit more required configuration beyond the usual `executor` itself. -Reviewing the additional options, we see that the `rate` and `duration` are now required. This makes sense given the focus of this executor is achieving and maintaining the specified _iteration rate_ over the provided timeframe. Let's defer the discussion of `preAllocatedVUs` for the moment until we get our initial test execution completed. +As noted in the [`constant-arrival-rate` documentation](https://k6.io/docs/using-k6/scenarios/executors/constant-arrival-rate/), `rate` and `duration` are now required. This makes sense given the focus of this executor is to achieve and maintain the specified _iteration rate_ over the provided timeframe. + +Let's defer the discussion of `preAllocatedVUs` for the moment until our initial test finishes. ### Initial test run @@ -47,58 +49,78 @@ k6 run test.js Your test will now start printing results to your terminal... ```bash -INFO[0000] [VU: 4, iteration: 0] Starting iteration... source=console -INFO[0000] [VU: 5, iteration: 0] Starting iteration... source=console -WARN[0000] Insufficient VUs, reached 5 active VUs and cannot initialize more executor=constant-arrival-rate scenario=k6_workshop -INFO[0001] [VU: 2, iteration: 1] Starting iteration... source=console -INFO[0002] [VU: 1, iteration: 1] Starting iteration... source=console -... -INFO[0029] [VU: 4, iteration: 26] Starting iteration... source=console -INFO[0029] [VU: 5, iteration: 26] Starting iteration... source=console + execution: local + script: test.js + output: - -running (0m30.7s), 0/5 VUs, 135 complete and 0 interrupted iterations -k6_workshop ✓ [======================================] 0/5 VUs 30s 10 iters/s + scenarios: (100.00%) 1 scenario, 2 max VUs, 1m0s max duration (incl. graceful stop): + * k6_workshop: 50.00 iterations/s for 30s (maxVUs: 2, gracefulStop: 30s) + +INFO[0000] [VU: 1, iteration: 0] Starting iteration... source=console +INFO[0000] [VU: 2, iteration: 0] Starting iteration... source=console +WARN[0000] Insufficient VUs, reached 2 active VUs and cannot initialize more executor=constant-arrival-rate scenario=k6_workshop +INFO[0000] [VU: 1, iteration: 1] Starting iteration... source=console +INFO[0000] [VU: 2, iteration: 1] Starting iteration... source=console ... - iterations.....................: 135 4.402609/s - vus............................: 5 min=5 max=5 +INFO[0030] [VU: 2, iteration: 204] Starting iteration... source=console +INFO[0030] [VU: 1, iteration: 204] Starting iteration... source=console + +running (0m30.1s), 0/2 VUs, 410 complete and 0 interrupted iterations +k6_workshop ✓ [======================================] 0/2 VUs 30s 50.00 iters/s + + data_received..................: 325 kB 11 kB/s + data_sent......................: 47 kB 1.6 kB/s + dropped_iterations.............: 1091 36.235918/s + http_req_blocked...............: avg=3.48ms min=1µs med=6µs max=256.93ms p(90)=10µs p(95)=12µs + http_req_connecting............: avg=1.73ms min=0s med=0s max=136.79ms p(90)=0s p(95)=0s + http_req_duration..............: avg=132.35ms min=103.06ms med=115.38ms max=827.03ms p(90)=143.56ms p(95)=165.43ms + { expected_response:true }...: avg=132.35ms min=103.06ms med=115.38ms max=827.03ms p(90)=143.56ms p(95)=165.43ms + http_req_failed................: 0.00% ✓ 0 ✗ 410 + http_req_receiving.............: avg=86.77µs min=19µs med=80µs max=524µs p(90)=142µs p(95)=153.54µs + http_req_sending...............: avg=26.88µs min=7µs med=24.5µs max=194µs p(90)=39.1µs p(95)=43.54µs + http_req_tls_handshaking.......: avg=1.74ms min=0s med=0s max=126.36ms p(90)=0s p(95)=0s + http_req_waiting...............: avg=132.23ms min=102.98ms med=115.28ms max=826.93ms p(90)=143.43ms p(95)=165.28ms + http_reqs......................: 410 13.617531/s + iteration_duration.............: avg=136.06ms min=103.26ms med=115.77ms max=827.24ms p(90)=144.98ms p(95)=183ms + iterations.....................: 410 13.617531/s + vus............................: 2 min=2 max=2 + vus_max........................: 2 min=2 max=2 ``` -Our test ran successfully, but closer inspection of the results shows that our actual results are not as intended. + +Our test ran successfully. However, closer inspections show that our results are not as intended. Looking at the output, we see the following: ```bash -WARN[0000] Insufficient VUs, reached 5 active VUs and cannot initialize more executor=constant-arrival-rate scenario=k6_workshop +WARN[0000] Insufficient VUs, reached 2 active VUs and cannot initialize more executor=constant-arrival-rate scenario=k6_workshop ``` -What happened here? Remember the `preAllocatedVUs` setting we glossed over earlier? With this setting, we simply told k6 to start the test with 5 virtual users; after a few iterations, k6 was able to determine that it would **not** be able to achieve our desired iteration rate. -This fact is evident by looking at the `iterations` value in the test summary; our test was only able to attain a rate of 4.4 iterations per second---we wanted 10. +What happened here? + +Remember the `preAllocatedVUs` setting we glossed over earlier? With this setting, we told k6 to start the test with 2 virtual users. With such few allocated VUs, k6 could **not** achieve our desired iteration rate (50 iterations/s). + +You can confirm this fact in the `iterations` value in the test summary; our test attained a rate of only 13.61 iterations per second and we wanted 50. -We _could_ simply double the number of `preAllocatedVUs`, or better yet, we could allow k6 to automatically control the number of VUs to achieve our rate. +### Adjusting preallocated virtual users -### Autoscaling of virtual users +To restate, _Constant Arrival Rate_ focuses on the _iteration rate_. k6 aims to eliminate the need to be overly concerned about the actual number of users required to achieve such a rate. However, we still must give our script enough VUs to achieve that rate. -Restating once again, the _Constant Arrival Rate_ executor is focused on the _iteration rate_ over a period of time. k6 aims to eliminate the need to be overly concerned about the actual number of users required to achieve such a rate. As a user of the executor, our script can simply specify the minimum and maximum number of VUs allowed, letting k6 handle the actual details. This can be referred to as _autoscaling_. +In the preceding example, the request duration or latency is, on average, 132.35ms, and the 95 percentile is around 165.43ms. With just 2 VUs (`preAllocatedVUs`), in a very optimistic scenario, we cannot expect more than `2 VUs / 0.132 s = 15.15 iterations/s`. -With the `preAllocatedVUs`, we're basically providing the starting number of virtual users. Now we will provide the `maxVUs` to denote our upper bounds. Update the `options` as follows: +Playing a bit with the number of `preAllocatedVUs`, we can update your script to a higher value, e.g. 25. ```js export const options = { - scenarios: { - k6_workshop: { - executor: 'constant-arrival-rate', - rate: 10, - duration: '30s', - preAllocatedVUs: 5, - maxVUs: 50, + scenarios: { + k6_workshop: { + executor: 'constant-arrival-rate', + rate: 50, + duration: '30s', + preAllocatedVUs: 25, + }, }, - }, }; ``` -To show us when the autoscaling is taking place, let's include the following javascript before the `options` block. -```js -// This will be executed for each VU when initialized -console.log(`Hello, VU #${__VU} has entered the test!`); -``` Let's run our test once again: ```bash @@ -106,33 +128,32 @@ k6 run test.js ``` ```bash -INFO[0000] Hello, VU #1 has entered the test! source=console -INFO[0000] Hello, VU #0 has entered the test! source=console -INFO[0000] [VU: 2, iteration: 0] Starting iteration... source=console -INFO[0000] [VU: 5, iteration: 0] Starting iteration... source=console -INFO[0000] [VU: 3, iteration: 0] Starting iteration... source=console -INFO[0000] [VU: 1, iteration: 0] Starting iteration... source=console -INFO[0000] [VU: 4, iteration: 0] Starting iteration... source=console -INFO[0000] Hello, VU #6 has entered the test! source=console -INFO[0000] [VU: 6, iteration: 0] Starting iteration... source=console -INFO[0001] Hello, VU #7 has entered the test! source=console -... -INFO[0030] [VU: 6, iteration: 22] Starting iteration... source=console -INFO[0030] [VU: 8, iteration: 22] Starting iteration... source=console -INFO[0031] Hello, VU #0 has entered the test! source=console - -running (0m31.1s), 00/13 VUs, 293 complete and 0 interrupted iterations -k6_workshop ✓ [======================================] 00/13 VUs 30s 10 iters/s -INFO[0031] Hello, VU #0 has entered the test! source=console -... - iterations.....................: 293 9.432192/s - vus............................: 13 min=8 max=13 +running (0m30.5s), 00/25 VUs, 1501 complete and 0 interrupted iterations +k6_workshop ✓ [======================================] 00/25 VUs 30s 50.00 iters/s + + data_received..................: 1.2 MB 40 kB/s + data_sent......................: 174 kB 5.7 kB/s + http_req_blocked...............: avg=4.26ms min=1µs med=4µs max=332.88ms p(90)=8µs p(95)=14µs + http_req_connecting............: avg=2.05ms min=0s med=0s max=167.98ms p(90)=0s p(95)=0s + http_req_duration..............: avg=136.87ms min=101.78ms med=114.56ms max=862.77ms p(90)=146.68ms p(95)=184.36ms + { expected_response:true }...: avg=136.87ms min=101.78ms med=114.56ms max=862.77ms p(90)=146.68ms p(95)=184.36ms + http_req_failed................: 0.00% ✓ 0 ✗ 1501 + http_req_receiving.............: avg=46.57µs min=11µs med=39µs max=503µs p(90)=82µs p(95)=96µs + http_req_sending...............: avg=17.21µs min=5µs med=15µs max=151µs p(90)=28µs p(95)=30µs + http_req_tls_handshaking.......: avg=2.19ms min=0s med=0s max=203.88ms p(90)=0s p(95)=0s + http_req_waiting...............: avg=136.81ms min=101.73ms med=114.5ms max=862.74ms p(90)=146.57ms p(95)=184.31ms + http_reqs......................: 1501 49.213783/s + iteration_duration.............: avg=141.27ms min=101.88ms med=114.83ms max=862.9ms p(90)=150.38ms p(95)=407.84ms + iterations.....................: 1501 49.213783/s + vus............................: 25 min=25 max=25 + vus_max........................: 25 min=25 max=25 ``` -> :point_up: Feel free to ignore the entries about `VU #0`...it's a bit of a special case. -Reviewing the output now, we see that the desired rate was more closely achieved at 9.43 iterations per second (for the overall test), and that k6 ultimately scaled up to 13 VUs to achieve the desired rate. That's autoscaling! +Reviewing the output now, we see that the desired rate is more closely achieved at 49.21 iterations per second (for the overall test). + +You might be tempted to set a lower value for [`preAllocatedVUs` and use `maxVUs` option](https://k6.io/docs/using-k6/scenarios/executors/constant-arrival-rate/) to autoscale the test. `maxVUs` is the maximum number of VUs to allow during the test run, which defaults to `preAllocatedVUs` when not set. However, it's usually best to leave the default value for `maxVUs` and increase the `preAllocatedVUs`. In this way, the VUs are allocated before the test starts and will be used when (and only if) necessary. -> :point_up: By **not** specifying `maxVUs` in our first test, we essentially disabled autoscaling of VUs. +Allocating VUs in the middle of the test can be costly in terms of CPU resources in the load generator instance, and can skew the tests. With preallocating VUs, the test starts with all the VUs available―no need to wait to allocate more when needed in the middle of the test run. ### Other rate options @@ -149,22 +170,22 @@ export const options = { timeUnit: '1h', duration: '30s', preAllocatedVUs: 5, - maxVUs: 50, }, }, }; ``` -Running the script shows a pace of 3 iterations per second: +Running the script shows a pace of 2.78 iterations per second, and we can get there with 5 VUs. ```bash -INFO[0029] [VU: 1, iteration: 20] Starting iteration... source=console -INFO[0030] [VU: 2, iteration: 20] Starting iteration... source=console -INFO[0031] Hello, VU #0 has entered the test! source=console + scenarios: (100.00%) 1 scenario, 5 max VUs, 1m0s max duration (incl. graceful stop): + * k6_workshop: 2.78 iterations/s for 30s (maxVUs: 5, gracefulStop: 30s) +... +INFO[0029] [VU: 1, iteration: 16] Starting iteration... source=console +INFO[0030] [VU: 5, iteration: 16] Starting iteration... source=console -running (0m30.9s), 00/04 VUs, 83 complete and 0 interrupted iterations -k6_workshop ✓ [======================================] 00/04 VUs 30s 3 iters/s -INFO[0031] Hello, VU #0 has entered the test! source=console +running (0m30.0s), 0/5 VUs, 84 complete and 0 interrupted iterations +k6_workshop ✓ [======================================] 0/5 VUs 30s 2.78 iters/s ... iterations.....................: 83 2.682393/s vus............................: 4 min=3 max=4 diff --git a/Modules/III-k6-Intermediate/08-Setting-load-profiles-with-executors/Ramping-Arrival-Rate-Exercises.md b/Modules/III-k6-Intermediate/08-Setting-load-profiles-with-executors/Ramping-Arrival-Rate-Exercises.md index 5836e5a..3e6d41e 100644 --- a/Modules/III-k6-Intermediate/08-Setting-load-profiles-with-executors/Ramping-Arrival-Rate-Exercises.md +++ b/Modules/III-k6-Intermediate/08-Setting-load-profiles-with-executors/Ramping-Arrival-Rate-Exercises.md @@ -2,9 +2,11 @@ As noted in [Setting load profiles with executors](Setting-load-profiles-with-executors.md#Ramping-Arrival-Rate), the _Ramping Arrival Rate_ executor has a primary focus on the _iteration rate_ being applied over a specified duration within _stages_. +This is a scenario typically seen with [stress or spike testing](https://k6.io/docs/test-types/stress-testing/) when there is a need to gradually push an API beyond its breaking point or to simulate spikes to extreme load over a very short period of time. + ## Exercises -For our exercises, we're going to start by using a very basic script that simply performs an HTTP request and then waits one second before completing the test iteration. We're providing some console output as things change. +For our exercises, let's start with a basic script that runs an HTTP request and tries to achieve 30 requests per time unit (which defaults to 1 second) during the only _stage_ we define. We're providing some console output as things change. ### Creating our script @@ -12,16 +14,15 @@ Let's begin by implementing our test script. Create a file named _test.js_ with ```js import http from 'k6/http'; -import { sleep } from 'k6'; export const options = { scenarios: { k6_workshop: { executor: 'ramping-arrival-rate', stages: [ - { target: 10, duration: "30s" }, + { target: 30, duration: "30s" }, ], - preAllocatedVUs: 5, + preAllocatedVUs: 2, }, }, }; @@ -29,7 +30,6 @@ export const options = { export default function () { console.log(`[VU: ${__VU}, iteration: ${__ITER}] Starting iteration...`); http.get('https://test.k6.io/contacts.php'); - sleep(1); } ``` @@ -46,26 +46,46 @@ k6 run test.js Your test will now start printing results to your terminal... ```bash -INFO[0002] [VU: 4, iteration: 0] Starting iteration... source=console -INFO[0003] [VU: 5, iteration: 0] Starting iteration... source=console -INFO[0004] [VU: 3, iteration: 0] Starting iteration... source=console -INFO[0005] [VU: 2, iteration: 0] Starting iteration... source=console -INFO[0005] [VU: 1, iteration: 0] Starting iteration... source=console -... -INFO[0014] [VU: 2, iteration: 6] Starting iteration... source=console -INFO[0014] [VU: 4, iteration: 7] Starting iteration... source=console -WARN[0015] Insufficient VUs, reached 5 active VUs and cannot initialize more executor=ramping-arrival-rate scenario=k6_workshop -INFO[0015] [VU: 1, iteration: 6] Starting iteration... source=console -INFO[0015] [VU: 5, iteration: 7] Starting iteration... source=console -... -INFO[0029] [VU: 1, iteration: 19] Starting iteration... source=console -INFO[0030] [VU: 5, iteration: 20] Starting iteration... source=console + execution: local + script: test.js + output: - + + scenarios: (100.00%) 1 scenario, 2 max VUs, 1m0s max duration (incl. graceful stop): + * k6_workshop: Up to 30.00 iterations/s for 30s over 1 stages (maxVUs: 2, gracefulStop: 30s) -running (0m30.8s), 0/5 VUs, 102 complete and 0 interrupted iterations -k6_workshop ✓ [======================================] 0/5 VUs 30s 10 iters/s +INFO[0001] [VU: 1, iteration: 0] Starting iteration... source=console +INFO[0002] [VU: 2, iteration: 0] Starting iteration... source=console +INFO[0002] [VU: 1, iteration: 1] Starting iteration... source=console ... - iterations.....................: 102 3.315689/s - vus............................: 5 min=5 max=5 +INFO[0015] [VU: 1, iteration: 55] Starting iteration... source=console +INFO[0015] [VU: 2, iteration: 56] Starting iteration... source=console +WARN[0015] Insufficient VUs, reached 2 active VUs and cannot initialize more executor=ramping-arrival-rate scenario=k6_workshop +INFO[0015] [VU: 1, iteration: 56] Starting iteration... source=console +INFO[0015] [VU: 2, iteration: 57] Starting iteration... source=console +... +INFO[0030] [VU: 2, iteration: 157] Starting iteration... source=console +INFO[0030] [VU: 1, iteration: 156] Starting iteration... source=console + +running (0m30.1s), 0/2 VUs, 315 complete and 0 interrupted iterations +k6_workshop ✓ [======================================] 0/2 VUs 30s 29.95 iters/s + + data_received..................: 246 kB 8.2 kB/s + data_sent......................: 36 kB 1.2 kB/s + dropped_iterations.............: 134 4.455953/s + http_req_blocked...............: avg=2.96ms min=2µs med=8µs max=280.3ms p(90)=12µs p(95)=16µs + http_req_connecting............: avg=1.37ms min=0s med=0s max=121.04ms p(90)=0s p(95)=0s + http_req_duration..............: avg=116.57ms min=100.21ms med=112.53ms max=423.4ms p(90)=133.78ms p(95)=151.35ms + { expected_response:true }...: avg=116.57ms min=100.21ms med=112.53ms max=423.4ms p(90)=133.78ms p(95)=151.35ms + http_req_failed................: 0.00% ✓ 0 ✗ 315 + http_req_receiving.............: avg=93.73µs min=25µs med=94µs max=244µs p(90)=146µs p(95)=158.29µs + http_req_sending...............: avg=31.07µs min=8µs med=30µs max=470µs p(90)=42.6µs p(95)=47µs + http_req_tls_handshaking.......: avg=1.49ms min=0s med=0s max=133.12ms p(90)=0s p(95)=0s + http_req_waiting...............: avg=116.45ms min=100.05ms med=112.39ms max=423.24ms p(90)=133.68ms p(95)=151.19ms + http_reqs......................: 315 10.474815/s + iteration_duration.............: avg=119.81ms min=100.54ms med=112.83ms max=423.76ms p(90)=140.04ms p(95)=154.62ms + iterations.....................: 315 10.474815/s + vus............................: 2 min=2 max=2 + vus_max........................: 2 min=2 max=2 ``` While _successful_, a closer inspection of our results shows that our test behavior was not as intended. @@ -73,18 +93,22 @@ While _successful_, a closer inspection of our results shows that our test behav Looking at the output, we see the following: ```bash -WARN[0015] Insufficient VUs, reached 5 active VUs and cannot initialize more executor=ramping-arrival-rate scenario=k6_workshop +WARN[0015] Insufficient VUs, reached 2 active VUs and cannot initialize more executor=ramping-arrival-rate scenario=k6_workshop ``` -What happened? With the `preAllocatedVUs` setting we glossed over earlier, we told k6 to start the test with 5 virtual users; after a few iterations, k6 was able to determine that it would **not** be able to achieve our desired iteration rate within the stage. This fact is evident by looking at the `iterations` value in the test summary; our test was only able to attain a rate of 3.32 iterations per second---we wanted 10. +What happened? + +With the `preAllocatedVUs` setting we glossed over earlier, we told k6 to start the test with 2 virtual users. With this pre-allocation, k6 could **not** achieve the desired iteration rate within the stage. + +This fact is evident by looking at the `iterations` value in the test summary; our test was only able to attain a rate of 10.47 iterations per second and we wanted 30. -We _could_ simply double the number of `preAllocatedVUs`, or better yet, we could allow k6 to automatically control the number of VUs to achieve our desired rate. +### Adjusting preallocated virtual users -### Autoscaling of virtual users +Restating once again, the _Ramping Arrival Rate_ executor is focused on the _iteration rate_ over a time frame within each stage. k6 aims to eliminate the need to be overly concerned about the actual number of users required to achieve such a rate. However, we still need to give our script enough VUs to achieve that rate. -Restating once again, the _Ramping Arrival Rate_ executor is focused on the _iteration rate_ over a period of time within stages. k6 aims to eliminate the need to be overly concerned about the actual number of users required to achieve such a rate. As a user of the executor, our script can simply specify the maximum number of VUs allowed, letting k6 handle the actual details. This can be referred to as _autoscaling_. +From our example above, we have that our request duration -- or latency -- is, on average, 116.57ms, and the 95 percentile is around 151.35ms. With just 2 VUs (`preAllocatedVUs`), in a very optimistic scenario, we cannot expect much more than `2 VUs / 0.116 s = 17.24 iterations/s`. Even if this was the only factor at play, which we will see is not in the [next section](#ramping-effect). -With the `preAllocatedVUs`, we're basically providing the starting number of virtual users. Now we will provide the `maxVUs` to denote our upper bounds. Update the `options` as follows: +Playing a bit with the number of `preAllocatedVUs`, we can update your script to a higher value, e.g. 10. ```js export const options = { @@ -92,64 +116,55 @@ export const options = { k6_workshop: { executor: 'ramping-arrival-rate', stages: [ - { target: 10, duration: "30s" }, + { target: 30, duration: "30s" }, ], - preAllocatedVUs: 5, - maxVUs: 50, + preAllocatedVUs: 10, }, }, }; ``` -To show us when the autoscaling is taking place, let's include the following javascript before the `options` block. - -```js -// This will be executed for each VU when initialized -console.log(`Hello, VU #${__VU} has entered the test!`); -``` - Let's run our test once again: ```bash k6 run test.js ``` ```bash -INFO[0014] [VU: 5, iteration: 6] Starting iteration... source=console -INFO[0014] [VU: 3, iteration: 6] Starting iteration... source=console -INFO[0015] Hello, VU #6 has entered the test! source=console -INFO[0015] [VU: 4, iteration: 7] Starting iteration... source=console -INFO[0015] [VU: 6, iteration: 0] Starting iteration... source=console -INFO[0015] [VU: 1, iteration: 7] Starting iteration... source=console -... -INFO[0030] [VU: 4, iteration: 19] Starting iteration... source=console -INFO[0030] [VU: 10, iteration: 4] Starting iteration... source=console -INFO[0031] Hello, VU #0 has entered the test! source=console - -running (0m31.0s), 00/11 VUs, 143 complete and 0 interrupted iterations -k6_workshop ✓ [======================================] 00/11 VUs 30s 10 iters/s -INFO[0031] Hello, VU #0 has entered the test! source=console -... - iterations.....................: 143 4.61819/s - vus............................: 11 min=5 max=11 +running (0m30.1s), 00/10 VUs, 449 complete and 0 interrupted iterations +k6_workshop ✓ [======================================] 00/10 VUs 30s 29.95 iters/s + + data_received..................: 374 kB 12 kB/s + data_sent......................: 53 kB 1.8 kB/s + http_req_blocked...............: avg=5.19ms min=2µs med=8µs max=254.28ms p(90)=12µs p(95)=13µs + http_req_connecting............: avg=2.46ms min=0s med=0s max=122.56ms p(90)=0s p(95)=0s + http_req_duration..............: avg=115.77ms min=101.27ms med=110.42ms max=182.21ms p(90)=134.9ms p(95)=143.31ms + { expected_response:true }...: avg=115.77ms min=101.27ms med=110.42ms max=182.21ms p(90)=134.9ms p(95)=143.31ms + http_req_failed................: 0.00% ✓ 0 ✗ 449 + http_req_receiving.............: avg=87.06µs min=23µs med=85µs max=942µs p(90)=130µs p(95)=150.19µs + http_req_sending...............: avg=31.32µs min=9µs med=29µs max=386µs p(90)=43µs p(95)=49.19µs + http_req_tls_handshaking.......: avg=2.64ms min=0s med=0s max=131.24ms p(90)=0s p(95)=0s + http_req_waiting...............: avg=115.65ms min=101.19ms med=110.29ms max=182.12ms p(90)=134.75ms p(95)=143.19ms + http_reqs......................: 449 14.929648/s + iteration_duration.............: avg=121.23ms min=101.49ms med=110.94ms max=367.86ms p(90)=138.57ms p(95)=150.07ms + iterations.....................: 449 14.929648/s + vus............................: 10 min=10 max=10 + vus_max........................: 10 min=10 max=10 ``` -> :point_up: Feel free to ignore the entries about `VU #0`...it's a bit of a special case. - -Reviewing the output, we now see that k6 automatically added VUs to increase the _iteration rate_ ultimately reaching 11 VUs from our starting point of 5 VUs. -> :point_up: By **not** specifying `maxVUs` in our first test, we essentially disabled autoscaling of VUs potentially eliminating the ability to reach the desire iteration rate. +And we can see the iterations/s increased. However, not enough. ### Ramping effect -Looking at the previous summary, it also seems that our _iteration rate_ ended at 4.62 iterations per second...we wanted 10! Scaling VUs is only part of what the executor is using to control the _iteration rate_. +Looking at the previous summary, it also seems that our _iteration rate_ ended at 14.92 iterations per second, and we wanted 30! The number of VUs is not the only factor at play when controlling the _iteration rate_. -This brings up the _ramping_ aspect of the executor: k6 will linearly scale up or down the iteration rate to achieve the `target` rate within the stage. The `duration` will determine how long the ramping up/down will take place. +This brings up the _ramping_ aspect of this executor: k6 will linearly scale up or down the iteration rate to achieve the `target` rate within the stage. The `duration` will determine how long the ramping up/down will take place. -Because we did not specify a `startRate` option, k6 used the default value of 0 iterations per second. Therefore, our test started from 0, then _ramped up_ to 10 iterations per second over a period of 30 seconds. +Because we did not specify a `startRate` option, k6 used the default value of 0 iterations per second. Therefore, our test started from 0, then _ramped up_ to 30 iterations per second over a period of 30 seconds. ```text Rate -10/s | ....... +30/s | ....... | ......./ | ......./ | ......./ @@ -158,7 +173,7 @@ Rate S T A G E # 1 ``` -Because we have this linear progression, it makes sense that our overall rate was 4.62/s as this is roughly equal the midpoint of 0 - 10. +Because we have this linear progression, it makes sense that our overall rate was 14.92 iterations/s as this is roughly equal to the midpoint of 0 - 30. ### Altering the slope @@ -166,58 +181,48 @@ With _ramping_, each stage defines the `target` to be achieved at the end of the ```js export const options = { - scenarios: { - k6_workshop: { - executor: 'ramping-arrival-rate', - startRate: 10, - stages: [ - { target: 10, duration: "30s" }, - ], - preAllocatedVUs: 5, - maxVUs: 50, + scenarios: { + k6_workshop: { + executor: 'ramping-arrival-rate', + startRate: 30, + stages: [ + { target: 30, duration: "30s" }, + ], + preAllocatedVUs: 10, + }, }, - }, }; ``` Running our test once again using `k6 run test.js` produces the following: ```bash -INFO[0000] [VU: 5, iteration: 0] Starting iteration... source=console -INFO[0000] [VU: 4, iteration: 0] Starting iteration... source=console -INFO[0000] [VU: 3, iteration: 0] Starting iteration... source=console -INFO[0000] Hello, VU #6 has entered the test! source=console -INFO[0001] [VU: 6, iteration: 0] Starting iteration... source=console -INFO[0001] Hello, VU #7 has entered the test! source=console -INFO[0001] [VU: 7, iteration: 0] Starting iteration... source=console -INFO[0001] Hello, VU #8 has entered the test! source=console -INFO[0001] [VU: 8, iteration: 0] Starting iteration... source=console -... -INFO[0030] [VU: 4, iteration: 22] Starting iteration... source=console -INFO[0030] [VU: 6, iteration: 22] Starting iteration... source=console -INFO[0031] Hello, VU #0 has entered the test! source=console +running (0m30.8s), 00/10 VUs, 897 complete and 0 interrupted iterations +k6_workshop ✓ [======================================] 00/10 VUs 30s 30.00 iters/s -running (0m31.0s), 00/13 VUs, 291 complete and 0 interrupted iterations -k6_workshop ✓ [======================================] 00/13 VUs 30s 10 iters/s ... - iterations.....................: 291 9.397613/s - vus............................: 13 min=7 max=13 + iteration_duration.............: avg=130.68ms min=101.77ms med=112.27ms max=1.54s p(90)=145.2ms p(95)=165.15ms + iterations.....................: 897 29.089145/s + vus............................: 10 min=10 max=10 + vus_max........................: 10 min=10 max=10 ``` -This time we requested the test to start with an iteration rate of 10, using 5 VUs. k6 quickly determined that 5 VUs could not support that rate, so it scaled out VUs in order to attain the desired rate. +This time we requested the test to start with an iteration rate of 30, using 10 VUs, and this was enough get us close to that rate. + ```text Rate -10/s | .................................. - | ../ - |./ +30/s |........................................ | | 0/s +---------------------------------------+ 30s S T A G E # 1 ``` -The scaling was within the first second or two so that our graph would look like the above for the configured stage. -> If we wanted a single stage with a flat---or _constant_---rate we'd use the _Constant Arrival Rate_ executor instead! +You might be tempted to set a lower value for [`preAllocatedVUs` and use `maxVUs` option](https://k6.io/docs/using-k6/scenarios/executors/ramping-arrival-rate/) to autoscale the test. `maxVUs` is the maximum number of VUs to allow during the test run, which defaults to `preAllocatedVUs` when not set. However, it's usually best to leave the default value for `maxVUs` and increase the `preAllocatedVUs`. In this way, the VUs are allocated before the test starts and will be used when (and only if) necessary. + +Allocating VUs in the middle of the test can be costly in terms of CPU resources in the load generator instance, and can skew the tests. Preallocating VUs will allow the test to start with all the VUs available with no need to wait to allocate more when needed in the middle of the test run. + +If we wanted a single stage with a flat---or _constant_---rate we'd use the _Constant Arrival Rate_ executor instead! This executor can be useful for [stress or spike testing](https://k6.io/docs/test-types/stress-testing/) as we'll see in the next section. ### Adding stages to simulate spikes @@ -255,8 +260,7 @@ export const options = { // Leveled off at 30 iters/s for remainder { target: 30, duration: "8s" }, ], - preAllocatedVUs: 5, - maxVUs: 50, + preAllocatedVUs: 50, }, }, }; @@ -288,8 +292,7 @@ export const options = { // Leveled off at 30 iters/minute for remainder { target: 30, duration: "8s" }, ], - preAllocatedVUs: 5, - maxVUs: 50, + preAllocatedVUs: 50, }, }, }; @@ -297,4 +300,4 @@ export const options = { ### Wrapping up -With this exercise, you should be able to see the power in being able to ramp up and down the iteration rate to more realistically model your test activity. +With this exercise, you should be able to see the power of being able to ramp up and down the iteration rate to model your test activity more realistically.