Provide better json support for servers don't support chunked request body

[huoyaoyuan]

Background

JsonContent and PostAsJsonAsync provided good support for calling json web apis. To save buffer usage, it serializes directly into send buffer, which doesn't calculate content length before serialization, and uses chunked encoding.

However, there are certain amount of web servers don't support chunked encoding in request body, namely PHP. They would see chunked encoding as empty request. Discovering such issue is not straight.

The current workaround is to use LoadIntoBufferAsync:

var content = JsonContent.Create<T>(json);
await content.LoadIntoBufferAsync();
var response = await httpClient.PostAsync(url, content);

This is hard to discover, and prevents the usage of PostAsJsonAsync.

Possible solutions

• Add new overloads to JsonContent.Create and PostAsJsonAsync, with a parameter indicating whether to use chunked encoding.
  This can add a large amount of overloads.
• Serialize into a buffer eagerly
  Can downgrade performance for servers do support chunked request
• Change the http pipeline to calculate content length before sending
  The change is too broad

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background

JsonContent and PostAsJsonAsync provided good support for calling json web apis. To save buffer usage, it serializes directly into send buffer, which doesn't calculate content length before serialization, and uses chunked encoding.

However, there are certain amount of web servers don't support chunked encoding in request body, namely PHP. They would see chunked encoding as empty request. Discovering such issue is not straight.

The current workaround is to use LoadIntoBufferAsync:

var content = JsonContent.Create<T>(json);
await content.LoadIntoBufferAsync();
var response = await httpClient.PostAsync(url, content);

This is hard to discover, and prevents the usage of PostAsJsonAsync.

Possible solutions

• Add new overloads to JsonContent.Create and PostAsJsonAsync, with a parameter indicating whether to use chunked encoding.
  This can add a large amount of overloads.
• Serialize into a buffer eagerly
  Can downgrade performance for servers do support chunked request
• Change the http pipeline to calculate content length before sending
  The change is too broad

Author:	huoyaoyuan
Assignees:	-
Labels:	area-System.Net.Http, untriaged
Milestone:	-

[scalablecory]

Are you sure this is not some PHP (or Apache/nginx etc.) configuration that you're missing? It seems very weird that it would straight up not support chunked encoding.

If you are sure, has an issue been filed with PHP? It lacking support for a major part of the HTTP/1.1 standard is very strange and should be looked at.

I'm generally against adding workarounds when we're spec compliant. But, PHP being a large platform does make this more compelling. If we can verify that this is a limitation there, it might make sense to add an API to always buffer request content.

[karelz]

I agree that compensating for missing HTTP/1.1 standard implementation feels weird.
Given there is no detail about wide-spread server usage with chunked encoding disabled / not working, we should close this until more evidence appears.

[davidfowl]

Can we improve the docs to show the above and not recommend buffering for large JSON payloads

[huoyaoyuan]

Are you sure this is not some PHP (or Apache/nginx etc.) configuration that you're missing?

The answer is Yes, but it not I'm missing. The developers of certain popular sites/rpc peers misses it.

Search for "php chunked request", the first issue from StackOverflow indicates that it needs user-side explicit handling in PHP.
I've also mentioned that aria2 doesn't support chunked request.

Given that the popular browser doesn't use it, and popular tools like curl may not be using it, it's hard to say that this HTTP standard is widely obeyed.