v0.30.0.md

k6 v0.30.0 is here! 🎉 It was a bit of a slow after-holiday release, but it still packs a few major new features and improvements that users have been requesting for a long time!

New features

Share memory between VUs using read-only arrays (#1739)

k6 has long had an issue with the handling of big data files with test fixtures. For example, if you have a huge users.json file with test users for your application:

[
  {"username": "user1", "password": "password1", "other": "some-long-data-....-1"},
  {"username": "user2", "password": "password2", "other": "some-long-data-....-2"},
  // ... ~1 million more users or more... :D
  {"username": "user999999", "password": "password999999", "other": "some-long-data-....-999999"}
]

If you just use JSON.parse(open('users.json')) in your script, then every VU will have a copy of the whole huge data set. Every VU in k6 is a separate JavaScript runtime, so there wasn't a thread-safe way to share data between them. Until now, that is!

We've added a new built-in SharedArray object in the new k6/data module that allows VUs to share read-only data:

import { SharedArray } from 'k6/data';
import { sleep } from 'k6';
import http from 'k6/http';

let users = new SharedArray('someName', function () {
    // This function will be called only once, in the first init context
    // execution. Every other VU will just get a memory-safe read-only reference
    // to the already loaded data.
    console.log('Loading users.json, this happens only once...');
    // You are not restricted to JSON, you can do anything - parse a CSV or XML
    // file, generate random data, etc. - as long as you return an array.
    return JSON.parse(open('users.json'));
});

export let options = { vus: 10, duration: '30s' };
export default function () {
    let randomUserID = Math.floor(Math.random() * users.length);
    let user = users[randomUserID]; // alternatively, we can also use __VU and/or __ITER
    console.log(`VU ${__VU} is running iteration ${__ITER} with user ${user.username}...`);
    http.post('https://httpbin.test.k6.io/post', JSON.stringify(user));
    sleep(Math.random() * 2); // or, better yet, use arrival-rate
}

Notice how Loading users.json is logged only once, but each VU uses the users variable like a normal JS array. The data is read only once and we have just a single copy of the huge array in memory! Behind the scenes, k6 uses a JS Proxy to transparently copy only the row each VU requests in users[randomUserID] to it. This on-demand copying is a bit inefficient, but it's still leagues better than having a copy of the huge array in every VU! And you can avoid the copying in every iteration by pinning the data used by every VU and having let user = users[randomUserID] in the init context!

And yes, you can have multiple SharedArray objects in the same script, just make sure to give them unique names - this is what someName in the script above was for. Because VUs are independent JS runtimes, we need some way to differentiate between the different shared memory objects, so we require them to have unique names. These names are also the IDs that any xk6 extensions would need to use to access them.

This works both locally and in the cloud. We advise everyone who deals with large data files to wrap them in a SharedArray and give this new feature a try. The required script changes should be minimal, while the memory usage should be significantly lower. Hopefully, we can finally consider one of the biggest blockers k6 users have had for a long time solved! 🎉

Support a handleSummary() callback at the end of the test (#1768)

You can now export a function called handleSummary() and k6 will call it at the end of the test run, after even teardown(). handleSummary() will be called with a JS object containing the same information that is used to generate the end-of-test summary and --summary-export, and allows users to completely customize how the end-of-test summary looks like.

Besides customizing the end-of-test CLI summary (if handleSummary() is exported, k6 will not print the default), you can also transform the summary data to various machine or human-readable formats and save it to files. This allows the creation of JS helper functions that generate JSON, CSV, XML (JUnit/xUnit/etc.), HTML, etc. files from the summary data. Even binary formats like PDF are not out of reach, potentially, with an appropriate JS library that works in k6! You can also send the generated reports to a remote server by making an HTTP request with them (or using any of the other protocols k6 already supports)! Here's a simple example:

import http from 'k6/http';
import k6example from 'https://raw.githubusercontent.com/loadimpact/k6/master/samples/thresholds_readme_example.js';
export default k6example; // use some predefined example to generate some data
export const options = { vus: 5, iterations: 10 };

// These are still very much WIP and untested, but you can use them as is or write your own!
import { jUnit, textSummary } from 'https://jslib.k6.io/k6-summary/0.0.1/index.js';

export function handleSummary(data) {
    console.log('Preparing the end-of-test summary...');

    // Send the results to some remote server or trigger a hook
    let resp = http.post('https://httpbin.test.k6.io/anything', JSON.stringify(data));
    if (resp.status != 200) {
        console.error('Could not send summary, got status ' + resp.status);
    }

    return {
        'stdout': textSummary(data, { indent: ' ', enableColors: true}), // Show the text summary to stdout...
        'junit.xml': jUnit(data), // but also transform it and save it as a JUnit XML...
        'summary.json': JSON.stringify(data), // and a JSON with all the details...
        // And any other JS transformation of the data you can think of,
        // you can write your own JS helpers to transform the summary data however you like!
    }
}

k6 expects handleSummary() to return a {key1: value1, key2: value2, ...} map. The values can be string or ArrayBuffer and represent the generated summary report contents. The keys should be strings and determine where the contents will be displayed or saved: stdout for standard output, stderr for standard error, or a path to a file on the system (which will be overwritten).

The format of the data parameter is similar but not identical to the data format of --summary-export. The format of --summary-export remains unchanged, for backwards compatibility, but the data format for this new k6 feature was made more extensible and had some of the ambiguities and issues from the previous format fixed. We can't cover the new format in the release notes, though you can easily see what it contains by using return { 'stdout': JSON.stringify(data)}; in handleSummary()! 😄

This feature is only available for local k6 run tests for now, though we plan to support k6 cloud tests eventually. And, as mentioned in the snippet above, the JS helper functions that transform the summary in various formats are far from final, so keep an eye on jslib.k6.io for updates. Or, better yet, submit PRs with improvements and more transformations at https://github.com/loadimpact/jslib.k6.io 😄

Other enhancements and UX improvements

• CI: k6 releases for Windows will now be digitally signed, which should reduce the number and severity of warnings Windows users see; the warnings would hopefully disappear altogether once Microsoft sees enough usage of the signed k6 releases to trust us (#1746). The installer and binary were also enhanced with more metadata and had their look updated with the new k6 logo and styling (#1727).
• JS: goja, the JS runtime k6 uses, was updated to its latest master version. This includes a few bugfixes and support for several new features, so --compatibility-mode=base is even more feature-rich at no additional runtime cost. We are contributing patches to goja in an effort to completely drop core.js and have the benefit of lower CPU and memory usage per VU even with --compatibility-mode=extended in the next k6 version! 🎉
• Config: integer values for duration and similar time values in the exported script options and environment variables are now treated as milliseconds. Similarly, the timeout option in http.Params can now be "stringy", e.g. "30s", "1m10s", etc. (#1738).
• HTTP: k6 now accepts ArrayBuffer values for the HTTP request body (#1776). This is a prelude/MVP for us gradually adopting ArrayBuffer for all binary data in k6 (#1020).
• Docker: We've added WORKDIR /home/k6 to our official Dockerfile (#1794).

Bugs fixed!

• HTTP: updated the golang.org/x/crypto and golang.org/x/net dependencies, which should have resolved some corner case issues with HTTP/2 connections, since k6 depends on golang.org/x/net/http2 (#1734).
• HTTP: fixed a couple of issues with blockHostnames that prevented zero-length matches for wildcards, as well as the explicit blocking of a domain and its sub-domain at the same time (#1723).
• Logs: if logs are streamed to a loki instance, k6 will now wait for them to finish being pushed before it exits - this will specifically mean that logs and errors in the init context will be propagated (#1694).
• HTTP: fixed the missing host value from http.Response.request.headers when it was explicitly set in the HTTP request params. (#1744). Thanks, @noelzubin!
• UI: fixed the lack of newline after k6 login password inputs (#1749). Thanks, @paroar!
• HTML: fixed a panic in the html.Selection.slice() method (#1756). Thanks, @asettouf!
• Summary: fixed random ordering of groups and checks in the end-of-test summary, they should now be shown in the order of their occurrence (#1788).
• Summary: the value for Rate metrics in the --summary-export JSON file was was always 0, regardless of the pass/(pass+fail) ratio (#1768).

Internals

• JS: Added automated tc39/test262 tests in our CI pipeline, so we have greater assurance that we're not breaking things when we update our JS runtime or when we finally drop core.js (#1747).
• CI: We've enabled more CI tests on Windows, now that we've switched to GitHub Actions (#1720).

Breaking changes

Some of the changes above deserve a special second mention, since they either slightly break previously documented k6 behavior, or fix previously undefined behavior and bugs that someone might have inadvertently relied on:

• Summary: --no-summary now also disables --summary-export (#1768). You can recreate the previous behavior of k6 run --no-summary --summary-export=summary.json script.js by having an empty exported handleSummary() function in your script (so that the default text summary is not shown by k6) and executing only k6 run --summary-export=summary.json script.js. Or omitting --summary-export as well and using handleSummary() as shown above.
• Config: integer values for duration and similar time values in the exported script options and environment variables are now treated as milliseconds. This was previously undefined behavior, but instead of k6 erroring out, it silently accepted and treated such values as nanoseconds (#1738).
• Docker: We've added WORKDIR /home/k6 to our official Dockerfile (#1794).