request with multipart/form-data is not being accepted at endpoint

[Korysam15]

All,

I am having an issue where I am trying to upload a file with size 36M and it is not reaching my endpoint at all. However, if I send another file with the same format that is of size 57K it hits the endpoint and succeeds. I've been reading into the stream_threshold() and how it is set to a default value of 1MB. I increased this value to 45MB by using the code below:

app.stream_threshold(45 * 1024 * 1024);

but I'm not having any luck.

This is what the endpoint definition looks like:

CROW_ROUTE(app, "/v1/job/<string>/event/<string>/upload") .methods(crow::HTTPMethod::POST) ([this](const crow::request& req, string job_id, string id) { }

I am sending the following request:

curl <ip addr>:5002/v1/job/${job_id}/event/${event_id}/upload -i -F 'file=@cases/1.8.0/test.nc;'

When I use the verbose flag I receive the following output:

> POST /v1/job/5cccf134-4fa7-413d-9e55-25ab70131c71/event/82bbc650-bcb9-45e1-b7a2-41ca01cdf89b/upload HTTP/1.1
> Host: <host name>
> User-Agent: curl/7.87.0
> Accept: */*
> Content-Length: 37751200
> Content-Type: application/netcdf; boundary=------------------------a9c1e72da9dd7fc2
> Expect: 100-continue
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 100 Continue
HTTP/1.1 100 Continue

* Recv failure: Connection reset by peer
* Closing connection 0
curl: (56) Recv failure: Connection reset by peer

Here is the log output from the CROW Server when I make this request:

(2023-04-28 17:40:37) [DEBUG   ] 0x7fe870000f80 timer added: 0x7fe8b4ba0410 2
(2023-04-28 17:40:37) [DEBUG   ] 0x7fe870000f80 timer cancelled: 0x7fe8b4ba0410 2
(2023-04-28 17:40:37) [DEBUG   ] task_timer cancelled: 0x7fe8b4ba0410 2
(2023-04-28 17:40:37) [DEBUG   ] 0x7fe870000f80 from read(1) with description: "invalid HTTP method"
(2023-04-28 17:40:37) [DEBUG   ] 0x7fe870000f80 is_reading 0 is_writing 0
(2023-04-28 17:40:37) [DEBUG   ] 0x7fe870000f80 delete (idle) (queue length: 0)
(2023-04-28 17:40:37) [DEBUG   ] 0x7fe870000f80 timer cancelled: 0x7fe8b4ba0410 2
(2023-04-28 17:40:37) [DEBUG   ] task_timer cancelled: 0x7fe8b4ba0410 2

Note I took out the IP address information from the output of the curl request.

I have also tried increasing the max time to allow the transfer to take and that didn't help either.

Any help is appreciated.

[kiner-shah]

This looks similar to issue #558 #607

[Korysam15]

@kiner-shah it is very similar. I didn't see a solution to that one besides downgrading crow. I downgraded to v1.0+3 and that seemed to fix the issue.

[lkoeller]

I think the issue is due to HTTP header "Expect: 100-continue" sent by curl automatically if uploaded file is bigger than some fixed size.

I can reproduce with :

curl -k -v --header "Content-Type: application/json" --header "Transfer-Encoding: chunked" --header "Accept: application/json" --header "Expect: 100-continue" -X POST https://myserver/api/xxx -d @data.json

By forcing Expect:100-continue, I have the same issue : from read(1) with description: "invalid HTTP method"

When I remove the header, and send a small file, it works !

curl -k -v --header "Content-Type: application/json" --header "Transfer-Encoding: chunked" --header "Accept: application/json" -X POST https://myserver/api/xxx -d "{}"

[lkoeller]

I just revert the commit

0aff1bc Prevent HTTP pipelining which Crow doesn't support.

And the issue is solved !

It seems that fixing this vulnerability leads to critical regression

[Korysam15]

@lkoeller did you make a merge request for this fix? If not could you?

[lkoeller]

Hi No, not yet. I just revert the commit. I could do a merge request in a few day after vacation Le lun. 26 juin 2023 à 20:22, Kory ***@***.***> a écrit :

…

@lkoeller <https://github.com/lkoeller> did you make a merge request for this fix? If not could you? — Reply to this email directly, view it on GitHub <#626 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB26BSUVZO2FCLPMFO7U6XTXNHHPTANCNFSM6AAAAAAXPQY7ZU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[german1608]

Hello!

I'm facing this issue. Any updates on a fix?

[lkoeller]

Hello You have just to revert this commit : 0aff1bc Prevent HTTP pipelining which Crow doesn't support. And it will work again. Best Regards Le ven. 4 août 2023 à 21:26, German Robayo ***@***.***> a écrit :

…

Hello! I'm facing this issue. Any updates on a fix? — Reply to this email directly, view it on GitHub <#626 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB26BSU4RUTGSBNG3PIYGY3XTVEE7ANCNFSM6AAAAAAXPQY7ZU> . You are receiving this because you were mentioned.Message ID: ***@***.***>