Scenarios documentation #49

imiric · 2020-06-26T16:25:36Z

This is an initial draft of the scenarios / new executors documentation, with likely many things missing, so just to get the ball rolling.

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

mstoykov · 2020-06-29T12:34:10Z

~~I would prefer if we move the default column commit in a separate PR that gets merged earlier and then this get rebased on it?~~ saw the PR right after I commented, sorry ;) good job :P

src/data/markdown/docs/01 guides/02 Using k6/05 Options.md

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

mstoykov

Looks good, still needs more work :)
Things that I think are important:

have a section explaining new concepts

arrival rate
gracefulStop/RampDown

Have examples with probably what each executors is good for.

"If you want and then using this is good idea."

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

mstoykov · 2020-06-29T12:52:42Z

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

+Note that in order to reliably achieve a fixed request rate, it's recommended to keep
+the function being executed very simple, with preferably only a single request call,
+and no additional processing or `sleep()` calls.


While I agree having sleep is contra-productive in this case. Having complex functions with multiple requests isn't the problem - the problem is that the user needs to understand that a VU still needs to execute the iteration and they still can only execute 1 at time. So if there is a complex iteration taking 30seconds that isn't a problem if the requested iterations per second are 1 and we have 30-40 VUs.

So IMO we need to have a section on arrival rate and how it differs specifically from the other executors

👍 for the separate arrival rate section.

Would you or @robingustafsson volunteer for that? 😄 I could copy or paraphrase what Robin wrote in grafana/k6#550, but you guys have a better grasp of this and will probably explain it better.

I added a placeholder for the arrival rate section in 95f1187.

mstoykov · 2020-06-29T12:54:31Z

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

+    constant_arr_rate: {  // arbitrary scenario name
+      executor: 'constant-arrival-rate',
+      rate: 200,
+      timeUnit: '1s',
+      duration: '1m',
+      preAllocatedVUs: 50,
+      maxVUs: 100,


can you drop the timeUnit as this is its the default value (maybe add a comment next to the rate)

can we give the scenario an arbitrary name? and then still have the comment. This way it is more obvious :)

Hah I thought this one is obviously arbitrary but maybe not :)

Let me know if f99fbf3 is an improvement.

mstoykov · 2020-06-29T12:55:39Z

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

+      executor: 'constant-vus',
+      exec: 'scenario1',
+      vus: 50,
+      duration: '30s',


There will need to be gracefulStop:0s here in order for the 50VUs to stop executing at exactly 30s

I would rather not introduce gracefulStop here since this example already showcases a couple of features, and gracefulStop would need a more thorough explanation. And it wasn't meant to stop at exactly 30s. :)

So let's use gracefulStop in a separate example in that new gracefulStop/RampDown section you mentioned.

See the gracefulStop section added in 72d17e4.

mstoykov · 2020-06-29T13:00:02Z

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

+      tags: { scenario: '2' },
+      env: { SCENARIO: '2' },


if we are going to be setting those in k6 it's probably not a good idea to give examples like that ;)

Hah true, though I wrote this before the default scenario tag feature, and well, because of lack of imagination :)

I changed it slightly in f99fbf3, but let me know if you have a better idea.

Based on PR feedback by Mihail.

To avoid confusion with the shared-iterations executor, which is mentioned right after.

robingustafsson

This is a really good start @imiric! As I wrote in one of my comments I can take a stab at the arrival rate section.

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

robingustafsson · 2020-07-07T15:05:00Z

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

+
+## Arrival rate
+
+TODO (expand on https://github.com/loadimpact/k6/issues/550)


I can take a stab at writing this section intro if you want.

Thanks, I think that would be best as you have more knowledge about the topic. I'd just be paraphrasing :)

Ok, have pushed a first draft of content for the arrival rate section now :)

That looks great, the examples make it very easy to understand. Just fixed a couple of typos :)

robingustafsson · 2020-07-07T15:50:01Z

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

@@ -0,0 +1,535 @@
+---
+title: Executors


I think we should either call this guide "Scenarios" and rework the page a little bit to first talk about scenarios as a concept and explaining multi-scenario tests before going into the details of each executor which is a setting for a scenario, or we should introduce a new guide called "Scenarios" that is introduced before this guide that talks about scenarios as a concept etc. linking to this guide for more details.

Makes sense. The feature as a whole is "scenarios", and "executors" are a (crucial) part that make them possible, so we should mention and list them, but not make it the focal point of the article.

I'll see what I can add in the intro, and we can expand it to a standalone article if needed.

@robingustafsson Let me know if 17b520a is sufficient for what you had in mind. I think this is a gentler introduction, focusing on the benefits of scenarios and a general configuration example before going into details with executors. I'm not sure what else we can mention there, so maybe it doesn't need a separate article?

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

mstoykov · 2020-07-10T07:21:34Z

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

+
+</div>
+
+Note that by default a `scenario` tag with the name of the scenario as value is


This should probably be added to the list of tags somewhere else in the documentation

I added it to the default list of --system-tags on the Options page. Or do you mean somewhere else?

in the tags and groups ;)

Done, see 8c0ee99.

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

mstoykov · 2020-07-10T07:42:31Z

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md

+k6 v0.27.0 introduces the concept of _executors_: configurable modes of executing a
+JavaScript function that can model diverse traffic scenarios in load tests.
+
+Multiple executors can be used in the same script, they can be scheduled to run


Suggested change

Multiple executors can be used in the same script, they can be scheduled to run

Multiple executors can be used in the same script, they can be scheduled to run in

I reworked the intro somewhat to focus on scenarios, and now it's "run in sequence or parallel". Not sure if a second "in" is needed before "parallel", but DRY? 😄

See #49 (comment)

See #49 (review)

Resolves #49 (comment)

imiric · 2020-07-10T11:47:46Z

@mstoykov See the "When to use" sections added in ee1e8aa, to address your original suggestion, which is a good idea. Not sure if it's detailed enough, but let me know.

- #49 (comment) - #49 (comment) - #49 (comment)

This is already explained in the first example.

na--

I have quite a bit of minor nitpicks as inline comments and code suggestions, but overall I think the new docs are awesome ❤️

src/data/markdown/docs/01 guides/02 Using k6/05 Options.md

src/data/markdown/docs/01 guides/02 Using k6/14 Scenarios.md

src/data/markdown/docs/01 guides/02 Using k6/05 Options.md

src/data/markdown/docs/01 guides/02 Using k6/14 Scenarios.md

Co-authored-by: na-- <n@andreev.sh>

Resolves #49 (comment)

imiric · 2020-07-10T17:04:17Z

I think we're missing an article about the --execution-segment/--execution-segment-sequence options (or "Distributed execution" in general) in addition to what's mentioned on the Options page, with more examples, etc. Unless we shouldn't advertise that much? 😄

@na-- Would you volunteer to write that? :) It can probably wait after v0.27.0 is released, don't think a lot of people will notice or use it at first.

na-- · 2020-07-13T10:30:29Z

I can write it, but it's definitely something for after 0.27.0. It will probably only matter when we also have data segmentation for most people, so for now the brief mention in the options page should be enough. If anyone wants more details, they can read through the issues and PRs, for now 😅

na-- · 2020-07-14T06:37:32Z

Making a final skimming read through the docs, so far LGTM, but noticed we lack dropped_iterations in https://github.com/loadimpact/k6-docs/blob/master/src/data/markdown/docs/01%20guides/02%20Using%20k6/02%20Metrics.md

Its description should be something like "Introduced in k6 v0.27.0, the number of iterations that could not be started due to lack of VUs (for the arrival-rate executors) or lack of time (due to expired maxDuration in the iteration-based executors)."

na--

LGTM. We probably should include more (and more varied) examples and maybe some more advice to users, but that can come in the form of blog posts, or future PRs here.

Resolves #49 (comment)

imiric requested review from mstoykov and na-- June 26, 2020 16:25

imiric commented Jun 26, 2020

View reviewed changes

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md Outdated Show resolved Hide resolved

mstoykov reviewed Jun 29, 2020

View reviewed changes

src/data/markdown/docs/01 guides/02 Using k6/05 Options.md Outdated Show resolved Hide resolved

mstoykov reviewed Jun 29, 2020

View reviewed changes

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md Outdated Show resolved Hide resolved

mstoykov reviewed Jun 29, 2020

View reviewed changes

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md Outdated Show resolved Hide resolved

mstoykov reviewed Jun 29, 2020

View reviewed changes

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md Outdated Show resolved Hide resolved

mstoykov reviewed Jun 29, 2020

View reviewed changes

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md Outdated Show resolved Hide resolved

mstoykov reviewed Jun 29, 2020

View reviewed changes

Ivan Mirić added 3 commits July 1, 2020 09:56

Add initial executors documentation

0ad1017

Add execution segment docs to Options page

9df07f3

Reorder and reword some sections, more realistic examples

f99fbf3

Based on PR feedback by Mihail.

imiric force-pushed the docs/new-executors branch from 4b34af0 to f99fbf3 Compare July 1, 2020 10:00

Ivan Mirić added 5 commits July 2, 2020 11:21

Add gracefulStop/gracefulRampDown section

72d17e4

Mention default scenario tag

3c07b3d

Rename Shared to Common configuration

6819aca

To avoid confusion with the shared-iterations executor, which is mentioned right after.

Add WIP Arrival rate section

95f1187

Restructure executors examples

6666d5d

imiric force-pushed the docs/new-executors branch from bf41ee7 to 6666d5d Compare July 2, 2020 16:43

imiric mentioned this pull request Jul 3, 2020

Write documentation for v0.27.0 release grafana/k6#1450

Closed

3 tasks

Document executor name option

0d97053

robingustafsson reviewed Jul 7, 2020

View reviewed changes

Arrival rate section

b3abc15

mstoykov reviewed Jul 10, 2020

View reviewed changes

src/data/markdown/docs/01 guides/02 Using k6/14 Executors.md Outdated Show resolved Hide resolved

mstoykov reviewed Jul 10, 2020

View reviewed changes

Ivan Mirić added 4 commits July 10, 2020 10:18

Fix typos

7b25f28

Rename Executors article to Scenarios, rework introduction

17b520a

See #49 (comment)

Use single quotes in JS examples for consistency

6ab5bff

Separate executor options, add 'When to use' section

ee1e8aa

See #49 (review)

Use indefinite article when explaining gracefulRampDown

26911c0

Resolves #49 (comment)

imiric changed the title ~~[WIP] New executors documentation~~ [WIP] Scenarios documentation Jul 10, 2020

Ivan Mirić added 2 commits July 10, 2020 16:27

Address some PR comments

2d07d72

- #49 (comment) - #49 (comment) - #49 (comment)

Remove repeated comment

b665e51

This is already explained in the first example.

na-- reviewed Jul 10, 2020

View reviewed changes

Ivan Mirić and others added 6 commits July 10, 2020 17:12

Apply suggestions from code review

49a5532

Co-authored-by: na-- <n@andreev.sh>

Document scenario tag on the Tags and Groups page

8c0ee99

Document scenarios on the Options page

52a4d4c

Update iterations option doc

8753826

Resolves #49 (comment)

Link to executor sections when listing possible values

a57a3c1

Resolves #49 (comment)

Fix scenarios ID on the Options page

04e5713

imiric requested review from na--, mstoykov and robingustafsson July 10, 2020 16:57

mstoykov approved these changes Jul 13, 2020

View reviewed changes

robingustafsson approved these changes Jul 13, 2020

View reviewed changes

imiric changed the title ~~[WIP] Scenarios documentation~~ Scenarios documentation Jul 13, 2020

imiric marked this pull request as ready for review July 13, 2020 13:17

Add complex scenario example, authored by Ned

415d8a8

na-- approved these changes Jul 14, 2020

View reviewed changes

Mention dropped_iterations metric

b3b38bb

Resolves #49 (comment)

imiric merged commit cb7c3ea into develop Jul 14, 2020

imiric deleted the docs/new-executors branch July 14, 2020 09:13

na-- mentioned this pull request Jul 14, 2020

Document how tests are executed (closed vs open system model, VUs etc.) grafana/k6#551

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scenarios documentation #49

Scenarios documentation #49

imiric commented Jun 26, 2020 •

edited

Loading

mstoykov commented Jun 29, 2020 •

edited

Loading

mstoykov left a comment

mstoykov Jun 29, 2020

imiric Jul 1, 2020

imiric Jul 2, 2020

mstoykov Jun 29, 2020

imiric Jul 1, 2020

mstoykov Jun 29, 2020

imiric Jul 1, 2020

imiric Jul 2, 2020

mstoykov Jun 29, 2020

imiric Jul 1, 2020

robingustafsson left a comment

robingustafsson Jul 7, 2020

imiric Jul 8, 2020

robingustafsson Jul 9, 2020

imiric Jul 10, 2020

robingustafsson Jul 7, 2020

imiric Jul 8, 2020

imiric Jul 10, 2020

mstoykov Jul 10, 2020

imiric Jul 10, 2020

mstoykov Jul 10, 2020

imiric Jul 10, 2020

mstoykov Jul 10, 2020

imiric Jul 10, 2020

imiric commented Jul 10, 2020

na-- left a comment

imiric commented Jul 10, 2020

na-- commented Jul 13, 2020

na-- commented Jul 14, 2020

na-- left a comment


		## Arrival rate

		TODO (expand on https://github.com/loadimpact/k6/issues/550)


		</div>

		Note that by default a `scenario` tag with the name of the scenario as value is

	Multiple executors can be used in the same script, they can be scheduled to run
	Multiple executors can be used in the same script, they can be scheduled to run in

Scenarios documentation #49

Scenarios documentation #49

Conversation

imiric commented Jun 26, 2020 • edited Loading

mstoykov commented Jun 29, 2020 • edited Loading

mstoykov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

robingustafsson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

imiric commented Jul 10, 2020

na-- left a comment

Choose a reason for hiding this comment

imiric commented Jul 10, 2020

na-- commented Jul 13, 2020

na-- commented Jul 14, 2020

na-- left a comment

Choose a reason for hiding this comment

imiric commented Jun 26, 2020 •

edited

Loading

mstoykov commented Jun 29, 2020 •

edited

Loading