TryAddWithoutValidation works differently for HttpClient.DefaultRequestHeaders and HttpRequestMessage.Headers

[AbeniMatteo]

Description

Header in HttpClient.DefaultRequestHeaders is formatted and sent as en-US, en; q=0.5 (note the spaces).

var client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Get, new Uri("http://localhost/"));
client.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Language", "en-US,en;q=0.5");
client.Send(request);

Header in HttpRequestMessage.Headers is untouched and sent as en-US,en;q=0.5.

var client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Get, new Uri("http://localhost/"));
request.Headers.TryAddWithoutValidation("Accept-Language", "en-US,en;q=0.5");
client.Send(request);

Configuration

.NET 5

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

Header in HttpClient.DefaultRequestHeaders is formatted and sent as en-US, en; q=0.5 (note the spaces).

var client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Get, new Uri("http://localhost/"));
client.DefaultRequestHeaders.TryAddWithoutValidation("Accept-Language", "en-US,en;q=0.5");
client.Send(request);

Header in HttpRequestMessage.Headers is untouched and sent as en-US,en;q=0.5.

var client = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Get, new Uri("http://localhost/"));
request.Headers.TryAddWithoutValidation("Accept-Language", "en-US,en;q=0.5");
client.Send(request);

Configuration

.NET 5

Author:	AbeniMatteo
Assignees:	-
Labels:	area-System.Net.Http, untriaged
Milestone:	-

[stephentoub]

The code here:

runtime/src/libraries/System.Net.Http/src/System/Net/Http/Headers/HttpHeaders.cs

Line 557 in 89fbdca

if (!sourceHeaders.ParseRawHeaderValues(header.Key, info, removeEmptyHeader: false))

is parsing the data from the default headers being added. Looking back through history, it appears this was done when the handler would enumerate the headers in a way that would parse/validate them, in which case it was desirable to forcible do that parsing/validation just once for the default header collection rather than on every send. But as of dotnet/corefx#41640 in .NET 5, we're no longer doing that parsing/validation forcibly as part of actually sending the request, which means that up-front parsing/validation of the default header collection is now not only unnecessary work but also leads to the behavioral discrepancy highlighted in this issue. Seems like we should just stop doing that.

[geoffkizer]

Seems like we should just stop doing that.

By "that", you mean:

up-front parsing/validation of the default header collection

? Seems reasonable to me.

[stephentoub]

Yup :)

[ManickaP]

Triage: we should investigate here and if we parse the headers we should not.

[AbeniMatteo]

This is getting interesting. 😄
DebuggerDisplayAttribute works the opposite way. Now HttpRequestMessage.Headers is the one being formatted.

[stephentoub]

DebuggerDisplayAttribute works the opposite way

There's no DebuggerDisplayAttribute here: the debugger is just calling ToString().

I also don't see the behavior you're seeing... what .NET version is that? ToString() shouldn't be parsing the values (and I don't see it doing so in the code).

[AbeniMatteo]

The problem appears after hovering .Headers then request and again .Headers.
Apparently calling request.ToString() changes the value, you can also trigger it programmatically:

var a = request.Headers.ToString(); // Accept-Language: en-US,en;q=0.5
request.ToString();
var b = request.Headers.ToString(); // Accept-Language: en-US, en; q=0.5

I'm using .NET 5.0.201 with VS 2019 Version 16.9.1.

[stephentoub]

Apparently calling request.ToString() changes the value

Yes. I thought you were just looking at headers, but HttpRequestMessage.ToString() enumerates HttpHeaders (rather than using its ToString()), and such enumerating will parse.