ASP.NET Core 9 sends response content for HTTP HEAD requests

[GREsau]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

In .NET 8 (and I think earlier versions), returning an IResult that writes a response body would not actually write the body in response to HTTP HEAD requests, but this has changed in .NET 9. I couldn't find anything in the published change notes indicating that this is an intended change.

For example, with the program:

var app = WebApplication.Create();

app.Map("/", () => Results.Ok(new { message = "ok!" }));

app.Run();

When running with .NET 8:

>curl --head http://localhost:5087
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Thu, 02 Jan 2025 11:59:38 GMT
Server: Kestrel

>curl -X HEAD http://localhost:5087 -m 1
Warning: Setting custom HTTP method to HEAD with -X/--request may not work the way you want. Consider using -I/--head instead.
curl: (28) Operation timed out after 1001 milliseconds with 0 bytes received

The second invocation uses -X HEAD to make curl ignore all HTTP semantics regarding the HEAD method, and so times out waiting for a response body that never arrives (hence curl's big warning!)

But when running with .NET 9:

>curl --head http://localhost:5087
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Thu, 02 Jan 2025 12:02:19 GMT
Server: Kestrel
Transfer-Encoding: chunked

>curl -X HEAD http://localhost:5087 -m 1
Warning: Setting custom HTTP method to HEAD with -X/--request may not work the way you want. Consider using -I/--head instead.
{"message":"ok!"}

Note that the second invocation now succeeds, because the server included the response content. This violates RFC 9110 9.3.2.: The HEAD method is identical to GET except that the server MUST NOT send content in the response.

Expected Behavior

No response

Steps To Reproduce

No response

Exceptions (if any)

No response

.NET Version

9.0.101

Anything else?

No response

[BrennanConroy]

Could you test with Results.Text("ok!") as well? I'm guessing this unintentional change might be due to writing json as chunked now and there might be a missing check for writing the body in that case.

[GREsau]

With Results.Text("ok!"), both .NET 8 and 9 are identical in that they omit the response content:

>curl --head http://localhost:5087
HTTP/1.1 200 OK
Content-Length: 3
Content-Type: text/plain; charset=utf-8
Date: Thu, 02 Jan 2025 21:12:48 GMT
Server: Kestrel

>curl -X HEAD http://localhost:5087 -m 1
Warning: Setting custom HTTP method to HEAD with -X/--request may not work the way you want. Consider using -I/--head instead.
curl: (28) Operation timed out after 1001 milliseconds with 0 out of 3 bytes received

So your guess looks likely!

[BrennanConroy]

Could you tell us how you found this issue? Was it in a real-world scenario and does it have more side-effects than additional bandwidth for HEAD responses?

[GREsau]

It wasn't a real-world scenario, it was caught by integration tests. The HTTP client used by the tests would reuse the connection after the HEAD request (unlike most clients which would close the connection after noticing the extra data), and try to process the response content as a second response

[quentez]

I've also noticed this as it brakes other tools that don't expect this behavior:

• The Vite server proxy throws Error: Parse Error: Data after "Connection: close".
• Postman fails with Parse Error: The server returned a malformed response.

Both only happen on HEAD requests that return an IResult with JSON content.

[BrennanConroy]

How common is it to do a HEAD request with these tools?

[quentez]

@BrennanConroy We just happen to use HEAD requests extensively in the product we're building. Vite relays requests when running the frontend locally and Postman performs the same requests for debugging.

[GREsau]

Until the fix is available (in ASP.NET 10?), presumably we can work-around the issue by swapping out the response body for HEAD requests for a null stream?

e.g. based on some quick local testing, adding this early in the middleware pipeline appears to resolve the issue:

app.Use((ctx, next) =>
{
    if (HttpMethods.IsHead(ctx.Request.Method))
    {
        ctx.Response.Body = Stream.Null;
    }
    return next();
});