Output plugin: kcs metrics streaming

[ragnarlonn]

The interface for streaming metrics to the cloud services seems to match the intended use of output plugins. It might not be a 100% match, which is why we need to discuss it further (here). Some points to clarify:

• Authentication. I'm thinking the plugin can look for credentials in a file or in environment variables, like tools interacting with AWS do.

• How do we transport the data most effectively? Client-to-cloud streaming (chunking in HTTP/1 parlance), periodic batching with POST requests or something I don't know about in HTTP/2?

Imported from https://trello.com/c/0Rzly0xL/90-output-plugin-kcs-metrics-streaming

[ragnarlonn]

Emily Ekberg
It's gonna be a bit tough to design, but I think we can do it with output plugins, just by changing the interface a bit.

Firstly, output plugins need some way to store JSON blobs for authentication, and a hook into the login command (eg. k6 login). I can think of a couple of ways that could work, need to think more about what the best way to handle this would be.

Second, an Init() hook or something of the sort, which… I'm not sure how to design at all, or even what information it'd need. I do agree that output plugins feel a little iffy right now, but I can't think of a better way to do it than to simply reshape what output plugins actually do.
Dec 13 at 10:33 PM - Reply - Delete

Daniel Brandt
There are other problems with my proposed solution of using an output plugin though. We're not interested in having to parse the JS test code server-side to get a hold of test configurations, which means a lot more is required in regards to API calls and test setup from k6 before we even get to the result data streaming. Output plugins is probably not a good match for that. Waiting for @uppfinnarn for final verdict, but I'm pretty sure she'll side with this sentiment.
Dec 13 at 12:25 PM - Reply - Delete

Daniel Brandt
@robinegustafsson The problem regarding authentication is no state is shared with the collector API AFAICT, but if the k6 login command produces a file or modifies the environment variables in a predetermined way, you can of course pick that up in the collector as well. In the end, as long as you can augment the request headers with the needed tokens, all will be good.

There is no contradiction in keeping the API an open spec and implementing it in an output plugin. The only issue is where the code makes most sense. To me, if it's to be an open source project, we have to isolate our vendor specific code in some clear way instead of baking it into the core. The login command will likely be separated from the rest as well, and I se it mostly as a convenience and to make the user experience similar to other tools.
Dec 13 at 11:54 AM - Reply - Delete

Robin Gustafsson
(added labels to make board filterable per group)

Authentication, yes looking in the usual places like env vars and a config file seems sensible. @uppfinnarn and I talked previously of using JSON Web Tokens for auth after login. Thoughts on that?

Also if the login is a command like k6 login it feels like the protocol/API should still be an open spec. What are your thoughts on having this an open spec (https://loadimpact.quip.com/08WVAPpFVbfG) vs it just being an output plugin, would it matter to you how you perceived a tool like k6 if there were vendor specific stuff like this in the repo vs doing something more open like Docker client and the registry API?

Regarding transport of data I think periodic batching sounds like a good first iteration to get going. Feels like the simplest approach to me.
Dec 13 at 11:44 AM - Reply - Add Link as Attachment - Delete