SFTP protocol support

[sagikazarmark]

Feature Description

This is probably a very long shot, but I thought I would get my use case out there anyway, maybe it's interesting for someone.

I'm building a custom SFTP server that stores files in S3 that I need to load test somehow.

The subject of the test is to asses the upload performance (with local and S3 filesystems), identify and potential improvements and monitor the performance over time (with automated tests).

k6 is obviously not capable of doing that at the moment (or is it?), but after looking at loads of load testing tools (pun not intended), k6 is the one that I would preferably use for all my needs. Most of the other tools are not extensible at all, usually supporting only HTTP (sadly 1.0/1.1 for most of the time). "Use the best tool for the job" is the saying, but I usually prefer if it's one tool and not a dozen for the different load testing scenarios.

So ideally, I'd implement the "sftp" protocol myself (and probably contribute it back too), except it seems to be a lot of work which I can't invest in k6 at the moment. 😞

So my suggestion would be: make it easier to implement new protocols in k6 (if at all possible). Some documentation, dummy examples would probably help a lot. Whether the core should have support for protocols like sftp or they should be some sort of extensions is an entirely different question.

As I mentioned at the beginning, this might not be valuable at all for the majority of users, so it's a long shot, I don't really expect any positive answer for adding support to the core, but some pointers to adding a new protocol would be nice, maybe I can find some time to implement sftp.

Thanks!

[na--]

Thanks for making this issue! You've hit a lot of nails on the head - we basically want to do most of the things you propose! 🎉 There is a lot of work to be done before that though, which can be roughly split like this:

1. Make it possible for advanced users to add new protocols to k6, without forking it, even some that are unlikely to be added to the k6 core. The initial attempt at this was to add plugins with the Go plugin API (Plugin support #1396), but we hit some pretty serious compatibility limitations (Plugin support #1353 (comment) among others), so the current plan (Plugin support #1353 (comment)) is to adopt an extension approach similar to the one caddy uses with xcaddy. This would allow you and everyone else to build k6 extensions with minimal effort and reasonable UX, and we plan to get started on xk6 (or however we decide to call it) in the next few weeks! Hopefully it will be ready by k6 v0.29.0 (mid-November) or soon after that.

2. Make it easy to contribute new protocols/outputs or fixes in the existing ones - this means better documentation, examples, code comments, etc.

3. Make it actually possible to add any protocol, either to the k6 core or as an xk6 plugin, easily and cleanly. This means fixing some core architectural limitations in k6 that make it difficult or impossible to support certain protocols, and even make the currently supported ones a bit subpar. Some examples:

   • The current k6 VU architecture is very synchronous, we don't have event loops (Global JS event loops in every VU #882), so a big chunk of asynchronous protocols can only be added in a very subpar way (see the current ws API, for example).
   • Another limitation of the current k6 architecture is that each VU is its own separate JS runtime, so when you open() a big file with 300 VUs, you get 300 copies of that file in memory. This has a few possible solutions, depending on the use case (Shared state/memory between VUs #532, Proposal for a K6 stream API #592, Data segmentation API framework #1539).
   • Better support for binary data (Better support for binary data #1020)
     (and a lot of other more minor/less blocking "inner core" issues)

4. Add more mainstream protocols in the k6 core. I can't promise that we'll ever support sftp load testing in k6 core, but there are plenty of protocols we want to support natively - gRPC (initial support for which should land in k6 v0.29.0, Support for gRPC protocol  #1623), various databases, raw TCP/UDP, DNS, etc. If there are popular xk6 plugins of high quality, there is a good chance we might adopt them and bring them into the core.

As I said, we plan to have xk6 soon, and the current plan is to have some support for shared data in k6 v0.30.0 (mid-January). I'm not so certain about event loops, given the complexity, but hopefully they will land soon after...

[sagikazarmark]

Thanks for the detailed response! It sounds really great.

It's really great you moved away from Go plugins. They are just terrible for a lot of reasons. 😞

[na--]

Probably worth mentioning that we've since released https://github.com/k6io/xk6 and you can use it for pretty much any protocol you want. Here's a short tutorial on how to write a k6 extension: https://k6.io/blog/extending-k6-with-xk6

I'll leave this issue open to gauge interest in an sftp support in the k6 core, though at this point such a thing doesn't seem very likely.

[codebien]

As the previous comment mentions, a solution for creating protocol extensions is available. However, considering the low amount of reactions, the k6 team doesn't plan to implement it directly.

If anyone is interested in adding SFTP as a new supported protocol should start creating an extension. From there, we may reconsider our plans based on the impact.