request with multipart/form-data is parsed only when its size is very small

[shk755]

Hello,

I am making a file uploading server. the simplified code is like this.

CROW_ROUTE(app, "/upload").methods(crow::HTTPMethod::POST)([&](const crow::request &req){
    crow::multipart::message msg(req);
    std::string filecont = msg.get_part_by_name("file").body;
    std::string userid = msg.get_part_by_name("user_id").body;
    CROW_LOG_DEBUG << filecont;
    CROW_LOG_DEBUG << userid;
    crow::json::wvalue x({{"message", "success"}});
    return x; 
});

When the Content-length exceeds 1024, http parser raises an error "invalid HTTP method". I tested with varying sizes and file types but it always stops to work when the content-length is 1025 or over

• server log on error

(2023-03-23 07:25:50) [DEBUG   ] 0x564305ed41b0 {1} queue length: 1
(2023-03-23 07:25:50) [DEBUG   ] 0x7f61b4002550 timer cancelled: 0x7f62837fd690 0
(2023-03-23 07:25:50) [DEBUG   ] task_timer cancelled: 0x7f62837fd690 0
(2023-03-23 07:25:50) [DEBUG   ] task_timer scheduled: 0x7f62837fd690 1
(2023-03-23 07:25:50) [DEBUG   ] 0x7f61b4002550 timer added: 0x7f62837fd690 1
(2023-03-23 07:25:50) [DEBUG   ] 0x7f61b4002550 timer cancelled: 0x7f62837fd690 1
(2023-03-23 07:25:50) [DEBUG   ] task_timer cancelled: 0x7f62837fd690 1
(2023-03-23 07:25:50) [DEBUG   ] task_timer scheduled: 0x7f62837fd690 2
(2023-03-23 07:25:50) [DEBUG   ] 0x7f61b4002550 timer added: 0x7f62837fd690 2
(2023-03-23 07:25:50) [DEBUG   ] 0x7f61b4002550 timer cancelled: 0x7f62837fd690 2
(2023-03-23 07:25:50) [DEBUG   ] task_timer cancelled: 0x7f62837fd690 2
(2023-03-23 07:25:50) [DEBUG   ] 0x7f61b4002550 from read(1) with description: "invalid HTTP method"
(2023-03-23 07:25:50) [DEBUG   ] 0x7f61b4002550 is_reading 0 is_writing 0
(2023-03-23 07:25:50) [DEBUG   ] 0x7f61b4002550 delete (idle) (queue length: 0)
(2023-03-23 07:25:50) [DEBUG   ] 0x7f61b4002550 timer cancelled: 0x7f62837fd690 2
(2023-03-23 07:25:50) [DEBUG   ] task_timer cancelled: 0x7f62837fd690 2

What should I do to make it work with bigger files?
Thanks

[kiner-shah]

Very strange, I don't get this error when I try to upload a file with Content-Length 2593701.
This is my code:

CROW_ROUTE(app, "/")
   .methods(crow::HTTPMethod::Post)
([](const crow::request& req)
{
    for (auto [headerKey, headerValue] : req.headers)
        std::cout << headerKey << ' ' << headerValue << '\n';
    // std::cout << req.body << '\n';

    crow::multipart::message file_message(req);
    std::ofstream out_file("input_file.txt");
    for (auto& part : file_message.parts)
        out_file << part.body;
        out_file.close();
    return crow::response(200);
});

[shk755]

Strange indeed... it must be a silly mistake but I cannot find it. I copied your code and it does not work for me either. maybe I included libraries in a bad way since i'm using standalone ASIO and manually including Crow, but the code works just fine other than this specific case. Thanks for checking it out @kiner-shah

[kiner-shah]

Similar to #558 #626

[lkoeller]

Probably curl inserts a HTTP header "Expect: 100-continue" when uploaded file is greater than some threshold.

I have a similar issue when POSTING json data with curl :

curl -k -v --header "Content-Type: application/json" --header "Transfer-Encoding: chunked" --header "Accept: application/json" --header "Expect: 100-continue" -X POST https://myserver/api/xxx -d @data.json

By forcing Expect:100-continue, I have the same issue : from read(1) with description: "invalid HTTP method"

When I remove the header, and send a small file, it works !

curl -k -v --header "Content-Type: application/json" --header "Transfer-Encoding: chunked" --header "Accept: application/json" -X POST https://myserver/api/xxx -d "{}"