Better handling of multipart/form-data requests

[na--]

A user asked a question about multipart/form-data requests in the slack chat and I found out that users cannot directly make multipart/form-data POST requests. The following script produces an invalid request:

import http from 'k6/http';

export default function () {
   let resp = http.post(
       "https://httpbingo.org/post",
       { username: "testuser", password: "testpass" },
       { headers: {"Content-Type": "multipart/form-data"} }
   );
   console.log(resp.body);
};

It seems like it sends multipart/form-data as the Content-Type header, but sends the actual POST data like the request was a normal application/x-www-form-urlencoded HTTP request.

We only correctly use send multipart/form-data requests when there's a file involved, so as I mentioned in the slack chat, this works as an ugly workaround:

import http from 'k6/http';

export default function () {
   let resp = http.post(
       "https://httpbingo.org/post",
       { username: "testuser", password: "testpass", _dummyFile: http.file("", "placeholder") },
   );
   console.log(resp.json().headers["Content-Type"]);
};

It would output something like multipart/form-data; boundary=some_random_boundary.... It seems like k6 automatically switches to multipart/form-data content-type when there are files involved, so when we try to send a dummy file as well as the username and password it works. And of course users can also manually build the HTTP request body, but it's a huge PITA:

import http from 'k6/http';

export default function () {
    let resp = http.post(
        "https://httpbingo.org/post",
        "--my-awesome-handcrafted-boundary\r\nContent-Disposition: form-data; name=\"username\"\r\n\r\ntestuser\r\n--my-awesome-handcrafted-boundary\r\nContent-Disposition: form-data; name=\"password\"\r\n\r\ntestpass\r\n--my-awesome-handcrafted-boundary--\r\n",
        { headers: { "Content-Type": "multipart/form-data; boundary=my-awesome-handcrafted-boundary" } }
    );
    let respJson = resp.json();
    console.log(respJson.headers["Content-Type"]);
    console.log(respJson.form.username);
    console.log(respJson.form.password);
};

Also, when I decided to see how the HAR converter deals with multipart/form-data, I saw that it just skips them, so we should fix that as well...

[na--]

#843 is connected to this, we should probably fix it together with this issue.

[na--]

New report of the same issues, with attached examples: #984

[stevegraykcf]

Would be nice if this were fixed. Worked around it with the dummy file, noticed you also have to NOT sully the content-type header when you do this, so I commented out that bit and got this post to execute

var systemName = "LoadTestCreatedSystem-"+(Math.random()*0xFFFFFF<<0).toString(8);
    res = http.post("https://"+__ENV.HOST+"/api2/accounts/"+locationId+"/systems",
        {Name : systemName, GroupCreationCount:"1", TemplateId:"-1", Type:"1", _dummyFile: http.file("", "placeholder")}//,
        //{ headers: { "Content-Type": "multipart/form-data" } }
    )

[na--]

Thanks, @stevegraykcf, we know that this issue is very annoying and we'll try to fix it soon. It's a bit tricky to fix it so k6 behaves consistently and doesn't break users' expectations, but we had a recent discussion about it in this somewhat related pull request, so we now have a somewhat clearer idea how we will proceed with this issue.

[imiric]

An update about this: multipart requests should be simpler to make now with the FormData polyfill and k6 v0.30.0 (make sure you're using at least this version). See the documentation.

Should we close this issue?

[na--]

I think so... Though please open an issue in https://github.com/k6io/har-to-k6 to have the converter also use the new FormData polyfill

[imiric]

Forgot to create the issue, but it's being addressed in grafana/har-to-k6#67, so closing this.