Rest Server Hanging Dead When Requested Concurrently

[wangtao9]

When I send POST requests concurrently with >=200 threads, the rest server just stops response.
The handle_post is:

void handler::handle_post(http_request message)
{
    ucout <<  message.to_string() << endl;
    json::value body;
    try {
        body = message.extract_json().get();
    }
    catch (std::exception&) {}
     message.reply(status_codes::OK,message.to_string());
    return ;
};

Please help me on this.

[Dimon4eg]

Similar to #1144

[garethsb]

You're doing a blocking task get() inside an async task so when the threadpool is loaded it can block.

Do this instead:

void handler::handle_post(http_request message)
{
    ucout << message.to_string() << endl;
    message.extract_json().then([message](pplx::task<json::value> value)
    {
        json::value body;
        try {
            body = value.get();
        }
        catch (std::exception&) {}
        message.reply(status_codes::OK, message.to_string());
    });
}

[yyzhang123]

I have the same problem as you， but my code as follow:

first， I set the receive and send timeouts via http_listener_config like following：
http_listener_config listenconfig;
listenconfig.set_timeout(utility::seconds(3));//设置收发超时时间为3s，及backlog为0

this is my httpserver class as follow：
HttpServerBase(utility::string_t url, http_listener_config& listenconfig, std::string reqcmdid, eRedisListKey redislistkey, RedisConnectPool* connpool, eMapType reqcmdtype, bool bsyncprocess = false)
:m_Listener(url, listenconfig)
{
m_RedisConnPool = std::move(connpool);

	m_ReqCommandId = std::move(reqcmdid);

	m_RedisListKey = std::move(redislistkey);

	m_SyncProcess = bsyncprocess;
	
	m_ReqCmdMapType =  std::move(reqcmdtype);
	
	m_Listener.support(methods::POST, std::bind(&HttpServerBase::Handle_Process_Json, this, std::placeholders::_1));
	SPDLOG_DEBUG("create HttpServerBase succeed listener url:{}, m_ReqCommandId:{}, m_RedisListKey:{}, m_SyncProcess:{}", url, m_ReqCommandId, m_RedisListKey, m_SyncProcess);

};

//json parse
virtual void Handle_Process_Json(http_request message)
{
try
{
SPDLOG_DEBUG("recv request:{} from remote_address:{}, command:{}", http::uri::decode(message.to_string()), message.remote_address(), m_ReqCommandId);

		//解析任务
		auto parsetask = message.extract_json().then([&](pplx::task<json::value> previousTask)
		{
			try  
			{ 
				SPDLOG_DEBUG("after extract_json");
				const json::value& cjsonreq = previousTask.get();
				SPDLOG_DEBUG("after previousTask.get()");
				message.reply(status_codes::OK);
			} 
			catch (const std::exception& sysexp)
			{
				SPDLOG_ERROR("occured std::exception:{} when recv:{}", sysexp.what(), m_ReqCommandId);
			}
		}).wait();
	}
	catch (const std::exception& sysexp)
	{
		SPDLOG_ERROR("occured std::exception:{} when recv:{}", sysexp.what(), m_ReqCommandId);
	}
	catch(...)
	{
		SPDLOG_ERROR("occured exception");
	}
};

Does this cpprest sdk support the centos7.4 operating system? Why is this?I found that after the program runs for a while, multiple threads will enter each other and wait for each other to compete with std::unique_lock and can't quit, When this happens, httpserver will not be able to receive new messages.

[garethsb]

Same problem. You have a blocking wait inside a handler. If the system is loaded, all threads in the task pool will be blocked.

[yyzhang123]

Same problem. You have a blocking wait inside a handler. If the system is loaded, all threads in the task pool will be blocked.

Hi,
Thanks for your reply, but Is that right?

first， I set the receive and send timeouts via http_listener_config like following：
http_listener_config listenconfig;
listenconfig.set_timeout(utility::seconds(3));//set timeout

this is my httpserver class as follow：
HttpServerBase(utility::string_t url, http_listener_config& listenconfig, std::string reqcmdid, eRedisListKey redislistkey, RedisConnectPool* connpool, eMapType reqcmdtype, bool bsyncprocess = false)
:m_Listener(url, listenconfig)
{
m_RedisConnPool = std::move(connpool);

	m_ReqCommandId = std::move(reqcmdid);

	m_RedisListKey = std::move(redislistkey);

	m_SyncProcess = bsyncprocess;
	
	m_ReqCmdMapType =  std::move(reqcmdtype);
	
	m_Listener.support(methods::POST, std::bind(&HttpServerBase::Handle_Process_Json, this, std::placeholders::_1));
	SPDLOG_DEBUG("create HttpServerBase succeed listener url:{}, m_ReqCommandId:{}, m_RedisListKey:{}, m_SyncProcess:{}", url, m_ReqCommandId, m_RedisListKey, m_SyncProcess);

};

//json parse
virtual void Handle_Process_Json(http_request message)
{
try
{
SPDLOG_DEBUG("recv request:{} from remote_address:{}, command:{}", http::uri::decode(message.to_string()), message.remote_address(), m_ReqCommandId);

		auto cts = pplx::cancellation_token_source();
		auto parsetask = message.extract_json().then([&](pplx::task<json::value> previousTask)
		{
			try  
			{ 
				SPDLOG_DEBUG("after extract_json");
				const json::value& cjsonreq = previousTask.get(); //I want to get the json content in the http body, and then start processing
				SPDLOG_DEBUG("after previousTask.get()");
				message.reply(status_codes::OK);
			} 
			catch (const std::exception& sysexp)
			{
				SPDLOG_ERROR("occured std::exception:{} when recv:{}", sysexp.what(), m_ReqCommandId);
			}
		}).wait();
	}
	catch (const std::exception& sysexp)
	{
		SPDLOG_ERROR("occured std::exception:{} when recv:{}", sysexp.what(), m_ReqCommandId);
	}
	catch(...)
	{
		SPDLOG_ERROR("occured exception");
	}
};

I found that after the program runs for a while, multiple threads will enter each other and wait for each other to compete with std::unique_lock and can't quit, When this happens, httpserver will not be able to receive new messages.

Can you answer some useful suggestions from me? what shout i do?

[mengzhangjian]

I also met the same problem, any way to solve it ?

[garethsb]

Do not perform blocking wait inside a listener handler, always use continuations.

[yyzhang123]

I also met the same problem, any way to solve it ?

yes， Do not perform blocking wait inside a listener handler，you should use Asynchronous task chain when recv or send like this :
try
{
auto parsetask = message.extract_json().then([=](pplx::taskjson::value previousTask)
{
try
{
const json::value& cjsonreq = previousTask.get();
ProcessJson(cjsonreq, message);
}
catch (const http_exception& e)
{
SPDLOG_ERROR("occured http_exception:{}, errorcode:{} when recv:{}", e.what(), e.error_code().message(), m_ReqCommandId);
}
});
}

          ....

//-------------------------------------

m_http_client->request(mtd, path_query_fragment, req_body_data).then([=](pplx::task<http_response> task)
{
try
{
//parse response
http_response response = task.get();
if(response.status_code() == status_codes::OK)
{
response.extract_json().then([=](pplx::taskjson::value jsontask){
try
{
auto rspbody = jsontask.get();
if(rspbody.is_null())
{
SPDLOG_ERROR("recv NULL rsp in after SendRequest uuid:{}, body:{}", uniqueid_from_dc, req_body_data.serialize());
return ;
}
processjson(rspbody, uniqueid_from_dc);
}catch(const std::exception& sysexp)
{
SPDLOG_ERROR("occured std::exception:{} in after SendRequest uuid:{}, body:{}", sysexp.what(), uniqueid_from_dc, req_body_data.serialize());
}
});
}
else
SPDLOG_ERROR("recv response msg from media that rspcode:{}, uuid:{}", response.status_code(), uniqueid_from_dc);
}
....

[ahmedyarub]

Spring Webflux detects blocking calls inside its threads and throws a run-time error. Can this be implemented in this project or generally in C++?

[sahnnu]

If i had to wait for some response and perform further steps only after the request is completed, then what to do in that case . Also many cpprest blogs demonstrate to wait on the request object.

https://mariusbancila.ro/blog/2017/11/19/revisited-full-fledged-client-server-example-with-c-rest-sdk-2-10/

Below is my code snippet and it seems to work for first few requests and then hangs. I dnt have access to server side code and seems server side code is there for many years. I am just writing the client side code

pplx::task<web::http::http_response> request_task = client.request(request);
        try
        {        
            web_response = request_task.get();
            result = true;

        }
        catch (const std::exception& ex)
        {
            string_t errorMessage;
            ErrorMsg = L"Can't reach the confighub agent. " + utility::conversions::to_string_t(ex.what());
        }
        catch (...)
        {
            return result;
        }

[sahnnu]

If i had to wait for some response and perform further steps only after the request is completed, then what to do in that case . Also many cpprest blogs demonstrate to wait on the request object.

https://mariusbancila.ro/blog/2017/11/19/revisited-full-fledged-client-server-example-with-c-rest-sdk-2-10/

Below is my code snippet and it seems to work for first few requests and then hangs and it retrieves after few minutes . I dnt have access to server side code and seems server side code is there for many years. I am just writing the client side code

pplx::task<web::http::http_response> request_task = client.request(request);
        try
        {        
            web_response = request_task.get();
            result = true;

        }
        catch (const std::exception& ex)
        {
            string_t errorMessage;
            ErrorMsg = L"Can't reach the confighub agent. " + utility::conversions::to_string_t(ex.what());
        }
        catch (...)
        {
            return result;
        }

@garethsb can you pls mention if the issue for blocking is in server side or client side

[sahnnu]

If i had to wait for some response and perform further steps only after the request is completed, then what to do in that case . Also many cpprest blogs demonstrate to wait on the request object.

https://mariusbancila.ro/blog/2017/11/19/revisited-full-fledged-client-server-example-with-c-rest-sdk-2-10/

Below is my code snippet and it seems to work for first few requests and then hangs and it retrieves after few minutes . I dnt have access to server side code and seems server side code is there for many years. I am just writing the client side code

pplx::task<web::http::http_response> request_task = client.request(request);
        try
        {        
            web_response = request_task.get();
            result = true;

        }
        catch (const std::exception& ex)
        {
            string_t errorMessage;
            ErrorMsg = L"Can't reach the confighub agent. " + utility::conversions::to_string_t(ex.what());
        }
        catch (...)
        {
            return result;
        }

@garethsb can you pls tell you changed the code at client side or server side.