Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HTTP 400 ("Invalid Host header") for link-local IPv6 addresses #59341

Closed
skrysmanski opened this issue Sep 20, 2021 · 5 comments · Fixed by #69203
Closed

HTTP 400 ("Invalid Host header") for link-local IPv6 addresses #59341

skrysmanski opened this issue Sep 20, 2021 · 5 comments · Fixed by #69203
Assignees
@wfurt
Labels
area-System.Net bug
Milestone
7.0.0

Comments

@skrysmanski
Copy link

Describe the bug

On Linux, it doesn't seem to be possible to do a requests through a link-local IPv6 address.

Link-local IPv6 addresses have the following format: fe80::...%... (e.g. fe80::20d:3aff:fe6d:4068%2).

If you want to make a request to this address, you basically have two options: with or without the %2 part.

If you keep the %2 part, you'll get a HTTP 400 (Bad request) both on Windows and on Linux. In this case, Kestrel logs:

Connection id "0HMBS0U5EM1IC" bad request data: "Invalid Host header: '[fe80::20d:3aff:fe6d:4068%2]:1234'"

If you remove the %2 part, it works on Windows but on Linux you'll get a HttpRequestException saying:

Invalid argument ([fe80::20d:3aff:fe6d:4068]:1237)

From what I could gather, the %2 is required on Linux but (apparently) not on Windows (see: https://stackoverflow.com/q/12260003/614177).

I'm not sure who the "culprit" is here - but it seems to me that Kestrel should allow [fe80::20d:3aff:fe6d:4068%2]:1234 as host header.

To Reproduce

I've created a reproducer here: https://github.com/skrysmanski/reproducer-kestel-ipv6-github-bug

The primary test code is here (method TestConnection()): https://github.com/skrysmanski/reproducer-kestel-ipv6-github-bug/blob/main/Reproducer/ReproducerTest.cs

You can find a Linux GitHub Actions run here: https://github.com/skrysmanski/reproducer-kestel-ipv6-github-bug/actions/runs/1252686642

This run contains the .trx files and a test report.

Exceptions (if any)

Complete test log for run with %2:

[07:45:30.864] [dbug] Microsoft.Extensions.Hosting.Internal.Host
      Hosting starting
[07:45:30.867] [info] Microsoft.Hosting.Lifetime
      Now listening on: http://[::]:1235
[07:45:30.868] [dbug] Microsoft.AspNetCore.Hosting.Diagnostics
      Loaded hosting startup assembly Reproducer
[07:45:30.868] [info] Microsoft.Hosting.Lifetime
      Application started. Press Ctrl+C to shut down.
[07:45:30.868] [info] Microsoft.Hosting.Lifetime
      Hosting environment: Production
[07:45:30.869] [info] Microsoft.Hosting.Lifetime
      Content root path: /home/runner/work/reproducer-kestel-ipv6-github-bug/reproducer-kestel-ipv6-github-bug/Reproducer/bin/Release
[07:45:30.870] [dbug] Microsoft.Extensions.Hosting.Internal.Host
      Hosting started

[CLIENT] Target host: fe80::20d:3aff:fe97:a10f%2
[CLIENT] Running query against: http://[fe80::20d:3aff:fe97:a10f]:1235/api/ping

[07:45:30.871] [dbug] Microsoft.AspNetCore.Server.Kestrel
      Connection id "0HMBS1AFEIM7M" accepted.
[07:45:30.871] [dbug] Microsoft.AspNetCore.Server.Kestrel
      Connection id "0HMBS1AFEIM7M" started.
[07:45:30.874] [dbug] Microsoft.AspNetCore.Server.Kestrel
      Connection id "0HMBS1AFEIM7M" bad request data: "Invalid Host header: '[fe80::20d:3aff:fe97:a10f%2]:1235'"
[07:45:30.874] [dbug] Microsoft.AspNetCore.Server.Kestrel
      Connection id "0HMBS1AFEIM7M" disconnecting.
[07:45:30.874] [dbug] Microsoft.AspNetCore.Server.Kestrel
      Connection id "0HMBS1AFEIM7M" stopped.
[07:45:30.875] [dbug] Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets
      Connection id "0HMBS1AFEIM7M" sending FIN because: "The Socket transport's send loop completed gracefully."

[CLIENT] Error: Response status code does not indicate success: 400 (Bad Request).

[07:45:30.875] [info] Microsoft.Hosting.Lifetime
      Application is shutting down...
[07:45:30.875] [dbug] Microsoft.Extensions.Hosting.Internal.Host
      Hosting stopping
[07:45:30.876] [dbug] Microsoft.Extensions.Hosting.Internal.Host
      Hosting stopped

Exception log for test run without %2:

Exception Type: System.Net.Http.HttpRequestException

Message: Invalid argument ([fe80::20d:3aff:fe97:a10f]:1236)

┌────────────────────────────────────────────────────────────────────┐
│ Exception Properties                                               │
└────────────────────────────────────────────────────────────────────┘

HResult: 0x80004005 (E_FAIL)

Source: System.Net.Http

┌────────────────────────────────────────────────────────────────────┐
│ StackTrace                                                         │
└────────────────────────────────────────────────────────────────────┘

at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
at Reproducer.ReproducerTests.ExecuteRequest(String targetHostIpAddress) in /home/runner/work/reproducer-kestel-ipv6-github-bug/reproducer-kestel-ipv6-github-bug/Reproducer/ReproducerTest.cs:line 91

	╔════════════════════════════════════════════════════════════════════╗
	║ Inner Exception                                                    ║
	╚════════════════════════════════════════════════════════════════════╝

	Exception Type: System.Net.Sockets.SocketException

	Message: Invalid argument

	┌────────────────────────────────────────────────────────────────────┐
	│ Exception Properties                                               │
	└────────────────────────────────────────────────────────────────────┘

	ErrorCode: 22

	HResult: 0x80004005 (E_FAIL)

	NativeErrorCode: 22

	SocketErrorCode: InvalidArgument

	Source: System.Net.Sockets

	┌────────────────────────────────────────────────────────────────────┐
	│ StackTrace                                                         │
	└────────────────────────────────────────────────────────────────────┘

	at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
	at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(short token)
	at System.Net.Http.HttpConnectionPool.DefaultConnectAsync(SocketsHttpConnectionContext context, CancellationToken cancellationToken)
	at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)

Further technical details

  • ASP.NET Core version: 5.0.0
  • .NET Core version: 5.0.401
@Tratcher
Copy link
Member

Tratcher commented Sep 20, 2021 •
edited
Loading

https://datatracker.ietf.org/doc/html/rfc7230#section-5.4

Host = uri-host [ ":" port ]

https://datatracker.ietf.org/doc/html/rfc7230#section-2.7.1
https://datatracker.ietf.org/doc/html/rfc3986#section-3.2.2

This syntax does not support
IPv6 scoped addressing zone identifiers.

https://datatracker.ietf.org/doc/html/rfc6874#section-2

IPv6addrz = IPv6address "%25" ZoneID

https://datatracker.ietf.org/doc/html/rfc6874#section-4

An HTTP client, proxy, or other intermediary MUST remove any ZoneID
attached to an outgoing URI, as it has only local significance at the
sending host.

This would appear to be a HttpClient bug. It needs the zone id to identify the right interface, but then it must remove that from the outgoing host header.

@Tratcher Tratcher transferred this issue from dotnet/aspnetcore Sep 20, 2021
@dotnet-issue-labeler dotnet-issue-labeler bot added area-System.Net untriaged New issue has not been triaged by the area owner labels Sep 20, 2021
@ghost
Copy link

ghost commented Sep 20, 2021

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

Describe the bug

On Linux, it doesn't seem to be possible to do a requests through a link-local IPv6 address.

Link-local IPv6 addresses have the following format: fe80::...%... (e.g. fe80::20d:3aff:fe6d:4068%2).

If you want to make a request to this address, you basically have two options: with or without the %2 part.

If you keep the %2 part, you'll get a HTTP 400 (Bad request) both on Windows and on Linux. In this case, Kestrel logs:

Connection id "0HMBS0U5EM1IC" bad request data: "Invalid Host header: '[fe80::20d:3aff:fe6d:4068%2]:1234'"

If you remove the %2 part, it works on Windows but on Linux you'll get a HttpRequestException saying:

Invalid argument ([fe80::20d:3aff:fe6d:4068]:1237)

From what I could gather, the %2 is required on Linux but (apparently) not on Windows (see: https://stackoverflow.com/q/12260003/614177).

I'm not sure who the "culprit" is here - but it seems to me that Kestrel should allow [fe80::20d:3aff:fe6d:4068%2]:1234 as host header.

To Reproduce

I've created a reproducer here: https://github.com/skrysmanski/reproducer-kestel-ipv6-github-bug

The primary test code is here (method TestConnection()): https://github.com/skrysmanski/reproducer-kestel-ipv6-github-bug/blob/main/Reproducer/ReproducerTest.cs

You can find a Linux GitHub Actions run here: https://github.com/skrysmanski/reproducer-kestel-ipv6-github-bug/actions/runs/1252686642

This run contains the .trx files and a test report.

Exceptions (if any)

Complete test log for run with %2:

[07:45:30.864] [dbug] Microsoft.Extensions.Hosting.Internal.Host
      Hosting starting
[07:45:30.867] [info] Microsoft.Hosting.Lifetime
      Now listening on: http://[::]:1235
[07:45:30.868] [dbug] Microsoft.AspNetCore.Hosting.Diagnostics
      Loaded hosting startup assembly Reproducer
[07:45:30.868] [info] Microsoft.Hosting.Lifetime
      Application started. Press Ctrl+C to shut down.
[07:45:30.868] [info] Microsoft.Hosting.Lifetime
      Hosting environment: Production
[07:45:30.869] [info] Microsoft.Hosting.Lifetime
      Content root path: /home/runner/work/reproducer-kestel-ipv6-github-bug/reproducer-kestel-ipv6-github-bug/Reproducer/bin/Release
[07:45:30.870] [dbug] Microsoft.Extensions.Hosting.Internal.Host
      Hosting started

[CLIENT] Target host: fe80::20d:3aff:fe97:a10f%2
[CLIENT] Running query against: http://[fe80::20d:3aff:fe97:a10f]:1235/api/ping

[07:45:30.871] [dbug] Microsoft.AspNetCore.Server.Kestrel
      Connection id "0HMBS1AFEIM7M" accepted.
[07:45:30.871] [dbug] Microsoft.AspNetCore.Server.Kestrel
      Connection id "0HMBS1AFEIM7M" started.
[07:45:30.874] [dbug] Microsoft.AspNetCore.Server.Kestrel
      Connection id "0HMBS1AFEIM7M" bad request data: "Invalid Host header: '[fe80::20d:3aff:fe97:a10f%2]:1235'"
[07:45:30.874] [dbug] Microsoft.AspNetCore.Server.Kestrel
      Connection id "0HMBS1AFEIM7M" disconnecting.
[07:45:30.874] [dbug] Microsoft.AspNetCore.Server.Kestrel
      Connection id "0HMBS1AFEIM7M" stopped.
[07:45:30.875] [dbug] Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets
      Connection id "0HMBS1AFEIM7M" sending FIN because: "The Socket transport's send loop completed gracefully."

[CLIENT] Error: Response status code does not indicate success: 400 (Bad Request).

[07:45:30.875] [info] Microsoft.Hosting.Lifetime
      Application is shutting down...
[07:45:30.875] [dbug] Microsoft.Extensions.Hosting.Internal.Host
      Hosting stopping
[07:45:30.876] [dbug] Microsoft.Extensions.Hosting.Internal.Host
      Hosting stopped

Exception log for test run without %2:

Exception Type: System.Net.Http.HttpRequestException

Message: Invalid argument ([fe80::20d:3aff:fe97:a10f]:1236)

┌────────────────────────────────────────────────────────────────────┐
│ Exception Properties                                               │
└────────────────────────────────────────────────────────────────────┘

HResult: 0x80004005 (E_FAIL)

Source: System.Net.Http

┌────────────────────────────────────────────────────────────────────┐
│ StackTrace                                                         │
└────────────────────────────────────────────────────────────────────┘

at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
at System.Net.Http.HttpClient.SendAsyncCore(HttpRequestMessage request, HttpCompletionOption completionOption, Boolean async, Boolean emitTelemetryStartStop, CancellationToken cancellationToken)
at Reproducer.ReproducerTests.ExecuteRequest(String targetHostIpAddress) in /home/runner/work/reproducer-kestel-ipv6-github-bug/reproducer-kestel-ipv6-github-bug/Reproducer/ReproducerTest.cs:line 91

	╔════════════════════════════════════════════════════════════════════╗
	║ Inner Exception                                                    ║
	╚════════════════════════════════════════════════════════════════════╝

	Exception Type: System.Net.Sockets.SocketException

	Message: Invalid argument

	┌────────────────────────────────────────────────────────────────────┐
	│ Exception Properties                                               │
	└────────────────────────────────────────────────────────────────────┘

	ErrorCode: 22

	HResult: 0x80004005 (E_FAIL)

	NativeErrorCode: 22

	SocketErrorCode: InvalidArgument

	Source: System.Net.Sockets

	┌────────────────────────────────────────────────────────────────────┐
	│ StackTrace                                                         │
	└────────────────────────────────────────────────────────────────────┘

	at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
	at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(short token)
	at System.Net.Http.HttpConnectionPool.DefaultConnectAsync(SocketsHttpConnectionContext context, CancellationToken cancellationToken)
	at System.Net.Http.ConnectHelper.ConnectAsync(Func`3 callback, DnsEndPoint endPoint, HttpRequestMessage requestMessage, CancellationToken cancellationToken)

Further technical details

  • ASP.NET Core version: 5.0.0
  • .NET Core version: 5.0.401
Author: skrysmanski
Assignees: -
Labels:

area-System.Net, untriaged
Milestone: -

@Tratcher
Copy link
Member

You can probably work around this by setting the request URI to http://[fe80::20d:3aff:fe6d:4068%2]:1234 and the request Host header to [fe80::20d:3aff:fe6d:4068]:1234.

@wfurt
Copy link
Member

wfurt commented Sep 20, 2021

Also for 5.0 you can use the connect callback.
The LLA should have link identification attached IMHO. Without it, the system would not know which interface to send to since the prefix is same an all. As @Tratcher mentioned this is probably HttpClient bug. Probably related to #25782.

@scalablecory scalablecory added this to the 7.0.0 milestone Sep 21, 2021
@scalablecory scalablecory added bug and removed untriaged New issue has not been triaged by the area owner labels Sep 21, 2021
@skrysmanski
Copy link
Author

I've verified that (for my limited scope) this is indeed the fix:

var requestUri = new Uri(requestUriAsString);
var requestMessage = new HttpRequestMessage(HttpMethod.Get, requestUri);

requestMessage.Headers.Host = requestUri.Authority; // the fix

var response = await s_httpClient.SendAsync(requestMessage);

Not sure whether Uri.Authority is always correct (or even the best property for the job). But as stated in #25782, Uri.IdnHost does not work for this.

The fix works on Windows, too.

@wfurt wfurt self-assigned this May 11, 2022
@wfurt wfurt mentioned this issue May 11, 2022
Merged
@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label May 11, 2022
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label May 12, 2022
@ghost ghost locked as resolved and limited conversation to collaborators Jun 11, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@wfurt wfurt

Labels
area-System.Net bug
Projects
None yet
Milestone
7.0.0
Development

Successfully merging a pull request may close this issue.

fix invalid Host header for link-local IPv6 wfurt/runtime
4 participants
@skrysmanski @Tratcher @scalablecory @wfurt