Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Misc scenario edits #936

Merged
merged 17 commits into from
Dec 19, 2022
Merged

Misc scenario edits #936

Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
9808150
scenarios, clean up arrival rate doc
MattDodsonEnglish Dec 6, 2022
d8c1b6f
Scenarios, clean up example text
MattDodsonEnglish Dec 6, 2022
9d1753c
Scenario example, improve intro
MattDodsonEnglish Dec 6, 2022
aa09357
Add restricted comma phrase for smoother reading
MattDodsonEnglish Dec 6, 2022
a78d686
scenario example, shorten text
MattDodsonEnglish Dec 6, 2022
fe8c619
Scenario examples, language clean up
MattDodsonEnglish Dec 13, 2022
e64aaa9
Favor character over encoding
MattDodsonEnglish Dec 13, 2022
aa79e86
Delete redundant text and old reference
MattDodsonEnglish Dec 13, 2022
18f5b76
Scenario examples, de-emphasize sequencing and composablilty
MattDodsonEnglish Dec 13, 2022
4aae4e6
scenarios example, say "combine" instead of "sequence"
MattDodsonEnglish Dec 13, 2022
ea8a943
scenarios, language trimming
MattDodsonEnglish Dec 14, 2022
bfb7dcc
typo fix
MattDodsonEnglish Dec 15, 2022
150d401
Update src/data/markdown/translated-guides/en/02 Using k6/14 Scenario…
MattDodsonEnglish Dec 15, 2022
3c8c0f3
Scenario, more precise description
MattDodsonEnglish Dec 16, 2022
79e817b
Formatting fixes
MattDodsonEnglish Dec 16, 2022
2f0a52c
Improve description of splitting logic
MattDodsonEnglish Dec 16, 2022
a2d3c45
formatting fix
MattDodsonEnglish Dec 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 30 additions & 13 deletions .../markdown/translated-guides/en/02 Using k6/14 Scenarios/02 Advanced Examples.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,19 @@ title: 'Advanced Examples'
excerpt: 'Advanced Examples using the k6 Scenario API - Using multiple scenarios, different environment variables and tags per scenario.'
---

## Using multiple scenarios
You can use multiple scenarios in one script, and these scenarios can be run in sequence or in parallel.
Some ways that you can combine scenarios include the following:
- Have different start times to sequence workloads
- Add per-scenario tags and environment variables
- Make scenario-specific thresholds.
- Run scenarios in different VU [lifecycle functions](/using-k6/test-lifecycle).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what you mean here, sorry 😕 You can't have different setup() or teardown() per scenario (see grafana/k6#1638, though it will probably be solved by actually implementing grafana/k6#1342)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now it's not specific enough. Thanks for pointing it out. What about this?

 Export multiple scenarios as functions in [VU code](/using-k6/test-lifecycle)

About the strange terminology: a month ago we defined lifecycle function as

a function called in a specific sequence in the k6 runtime.

We changed "callbacks" to "lifecycle functions" because callback sounded like a JS callback. It wasn't so obvious that it was to the Go runtime. Discussion here: #845.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to communicate what is done in this example, by the way.
Different scenarios run different VU logic.

export function contacts() {
  http.get('https://test.k6.io/contacts.php', {
    tags: { my_custom_tag: 'contacts' },
  });
}

export function news() {
  http.get('https://test.k6.io/news.php', { tags: { my_custom_tag: 'news' } });
}

MattDodsonEnglish marked this conversation as resolved.
Show resolved Hide resolved

This configuration will first execute a scenario where 50 VUs will try to run as many iterations
as possible for 30 seconds. It will then transition to the next scenario, executing 100 iterations
per VU for a maximum duration of 1 minute.

## Combine scenarios

With the `startTime` property, you can configure your script to start some scenarios later than others. If you combine this with the executor's duration options, you can sequence your scenarios (this is easiest to do with executors with set durations, like the arrival-rate executors.).

This configuration first executes a scenario where 50 VUs try to run as many iterations as possible for 30 seconds. It then runs the next scenario, which executes 100 iterations per VU for a maximum duration of 1 minute.

Note the use of `startTime`, and different `exec` functions for each scenario.

Expand Down Expand Up @@ -49,10 +57,10 @@ export function news() {

</CodeGroup>

## Different environment variables and tags per scenario.
## Use different environment variables and tags per scenario.

In the previous example we set tags on individual HTTP request metrics, but this
can also be done per scenario, which would apply them to other
The previous example sets tags on individual HTTP request metrics.
But, you can also set tags per scenario, which applies them to other
[taggable](/using-k6/tags-and-groups#tags) objects as well.

<CodeGroup labels={[ "multiple-scenarios-env-tags.js" ]} lineNumbers={[true]}>
Expand Down Expand Up @@ -98,14 +106,23 @@ export function news() {

</CodeGroup>

Note that by default a `scenario` tag with the name of the scenario as value is
applied to all metrics in each scenario, which can be used in thresholds and
simplifies filtering metrics when using [result outputs](/get-started/results-output).
This can be disabled with the [`--system-tags` option](/using-k6/options#system-tags).
<Blockquote mod="note" title="">

By default, k6 applies a `scenario` tag to all metrics in each scenario (the value is the scenario name.
You can combine these tags with thresholds, or use them to simplify results filtering.

To disable scenario tags, use the [`--system-tags` option](/using-k6/options#system-tags).

</Blockquote>

## Run multiple scenario functions, with different thresholds

## Multiple exec functions, tags, environment variables, and thresholds
You can also set different thresholds for different scenario functions.
To do this:
1. Set scenario-specific tags
1. Set thresholds for these tags.

A test with 3 scenarios, each with different `exec` functions, tags and environment variables, and thresholds:
This test has 3 scenarios, each with different `exec` functions, tags and environment variables, and thresholds:

<CodeGroup labels={[ "multiple-scenarios-complex.js" ]} lineNumbers={[true]}>

Expand Down
47 changes: 27 additions & 20 deletions src/data/markdown/translated-guides/en/02 Using k6/14 Scenarios/04 Arrival Rate.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,20 @@ title: 'Arrival rate'
excerpt: 'In k6, we have implemented this open model with our two arrival rate executors: constant-arrival-rate and ramping-arrival-rate.'
---

Different k6 executors have different ways of scheduling VUs.
Some executors use the _closed model_, while the arrival rate executors use the _open model_.

In short, in the closed model, VU iterations start only when the last iteration finishes.
In the open model, on the other hand, VUs arrive independently of iteration completion.
Different models suit different test aims.

## Closed Model

> In a closed model, the execution time of each iteration dictates the actual
> number of iterations executed in your test, as the next iteration won't be started
> until the previous one is completed.
In a closed model, the execution time of each iteration dictates the
number of iterations executed in your test.
The next iteration doesn't start until the previous one finishes.

Prior to v0.27.0, k6 only supported a closed model for the simulation of new VU arrivals.
In this closed model, a new VU iteration only starts when a VU's previous iteration has
completed its execution. Thus, in a closed model, the start rate, or arrival rate, of
Thus, in a closed model, the start or arrival rate of
new VU iterations is tightly coupled with the iteration duration (that is, time from start
to finish of the VU's `exec` function, by default the `export default function`):

Expand Down Expand Up @@ -53,32 +58,34 @@ closed_model ✓ [======================================] 1 VUs 1m0s

```

## Drawbacks of using the closed model
### Drawbacks of using the closed model

This tight coupling between the VU iteration duration and start of new VU iterations
in effect means that the target system can influence the throughput of the test, via
its response time. Slower response times means longer iterations and a lower arrival
rate of new iterations, and vice versa for faster response times.
When the duration of the VU iteration is tightly coupled to the start of new VU iterations,
the target system's response time can influence the throughput of the test.
Slower response times means longer iterations and a lower arrival rate of new iterations―and vice versa for faster response times.
In some testing literature, this problem is known as _coordinated omission._

In other words, when the target system is being stressed and starts to respond more
slowly a closed model load test will play "nice" and wait, resulting in increased
In other words, when the target system is stressed and starts to respond more
slowly, a closed model load test will wait, resulting in increased
iteration durations and a tapering off of the arrival rate of new VU iterations.

This is not ideal when the goal is to simulate a certain arrival rate of new VUs,
This effect is not ideal when the goal is to simulate a certain arrival rate of new VUs,
or more generally throughput (e.g. requests per second).

## Open model

> Compared to the closed model, the open model decouples VU iterations from
> the actual iteration duration. The response times of the target system are no longer
> influencing the load being put on the target system.
Compared to the closed model, the open model decouples VU iterations from
the iteration duration.
The response times of the target system no longer
influence the load on the target system.

To fix this problem we use an open model, decoupling the start of new VU iterations
from the iteration duration and the influence of the target system's response time.
To fix this problem of coordination, you can use an open model,
which decouples the start of new VU iterations from the iteration duration.
This reduces the influence of the target system's response time.

![Arrival rate closed/open models](../images/Scenarios/arrival-rate-open-closed-model.png)

In k6, we've implemented this open model with our two "arrival rate" executors:
k6 implements the open model with two _arrival rate_ executors:
[constant-arrival-rate](/using-k6/scenarios/executors/constant-arrival-rate) and [ramping-arrival-rate](/using-k6/scenarios/executors/ramping-arrival-rate):

<CodeGroup labels={[ "open-model.js" ]} lineNumbers={[true]}>
Expand Down