Connecting to IPAddress.Any via HTTP/3

[amcasey]

Description

This is more of a question than a bug, so please let me know if this is by design.

We did find #95487 (which points to microsoft/msquic#2704) but, frankly, it was hard to follow and apply to our scenario.

TL;DR: When a (kestrel) server is listening on IPAddress.Any, HttpClient can connect to localhost with HTTP/1.1 and HTTP/2.0, but not with HTTP/3.0. You seem to have to either use 127.0.0.1 or switch to IPv6Any (which also works for 1.1 and 2.0).

This seems to happen in both dotnet 8.0 and 9.0.

Reproduction Steps

Server

using Microsoft.AspNetCore.Server.Kestrel.Core;
using System.Net;

var builder = WebApplication.CreateBuilder(args);

builder.WebHost.ConfigureKestrel(options =>
{
    options.Listen(IPAddress.Any, 5001, listenOptions => // H3 to localhost doesn't work
    //options.Listen(IPAddress.IPv6Any, 5001, listenOptions => // H3 to localhost works
    {
        listenOptions.UseHttps();
        listenOptions.Protocols = HttpProtocols.Http1AndHttp2AndHttp3;
    });
});

var app = builder.Build();
 
app.MapGet("/", () => "Hello, world!");

app.Run();

Client

var handler = new SocketsHttpHandler(); 
handler.SslOptions.RemoteCertificateValidationCallback = (_, _, _, _) => true;

using var client = new HttpClient(handler)
{
    //BaseAddress = new Uri("https://127.0.0.1:5001"), // Works with ipv4 and ipv6
    //BaseAddress = new Uri("https://[::1]:5001"), // Works with ipv6
    BaseAddress = new Uri("https://localhost:5001"), // Works with ipv6
    DefaultRequestVersion = new Version(3, 0),
    DefaultVersionPolicy = HttpVersionPolicy.RequestVersionExact,
};

try
{
    Console.WriteLine("Requesting /");
    var response = await client.GetAsync("/");
    response.EnsureSuccessStatusCode();

    var data = await response.Content.ReadAsStringAsync();
    Console.WriteLine(data);
}
catch (HttpRequestException e)
{
    Console.WriteLine($"Request error: {e.Message}");
}

Expected behavior

Requesting /
Hello, world!

Actual behavior

Requesting /
Request error: Application layer protocol negotiation error was encountered. (localhost:5001)

Regression?

Not since 8.0, anyway. Possibly not even a bug.

Known Workarounds

In kestrel, it's more convenient to call ListenAnyIP anyway.

Configuration

Win11 x64. I appear to have dotnet 8.0.108 and 9.0.100-rc.1.24452.12, though the client may have been using whatever preview of 9.0 is in VS and the server would have been running aspnetcore main. I don't believe the specific dotnet versions matter, though the OS might.

Other information

No response

[amcasey]

FYI @BrennanConroy

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

[rzikm]

I expect that problem is that localhost resolves to IPv6 address at the DNS layer

var addresses = Dns.GetHostAddresses("localhost");
foreach (var addr in addresses)
{
    System.Console.WriteLine(addr);
}

prints on my machine

::1
127.0.0.1

We attempt a connection only via the first IP address returned from Dns.GetHostAddresses

runtime/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicConnection.cs

Lines 392 to 402 in 042cc95

	// Given just a ServerName to connect to, MsQuic would also use the first address after the resolution
	// (https://github.com/microsoft/msquic/issues/1181) and it would not return a well-known error code
	// for resolution failures we could rely on. By doing the resolution in managed code, we can guarantee
	// that a SocketException will surface to the user if the name resolution fails.
	IPAddress[] addresses = await Dns.GetHostAddressesAsync(host, cancellationToken).ConfigureAwait(false);
	cancellationToken.ThrowIfCancellationRequested();
	if (addresses.Length == 0)
	{
	throw new SocketException((int)SocketError.HostNotFound);
	}
	address = addresses[0];

Related: #82404

A possible workaround can be disabling IPv6, either system-wide, or for .NET only via AppCtx switch or env variable

[amcasey]

Assuming I understand correctly that #82404 tracks (indirectly) making HTTP/3.0 consistent with HTTP/2.0, that answers my question and we can close this issue. Thanks!

[wfurt]

while HTTP 1&2 can connect there is still probably performance penalty for trying IPv6 and falling back to IP4. You should probably investigate @amcasey. As mentioned above HTTP 3 does not have the fall-back at the moment.

[amcasey]

@wfurt Sorry, I'm not following. I agree that reaching ipv4 as a fallback is probably slower than reaching ipv6 directly, but I'm not sure what you want me to investigate.

[wfurt]

can you try packet capture for the case when it seems to work with HTTP 1? e.g. when you bind on IPAddress.Any does it connect on the first try or are there two separate attempts?

[amcasey]

Sure, though I did provide complete repro steps. 😛

[amcasey]

Two attempts:

[wfurt]

yes, this is what I expected. So even if it "works" the setup is not correct IMHO. And the fallback masks the misconfiguration.

[amcasey]

Are you proposing removing the fallback from the socket client?

[wfurt]

no. That is still useful IMHO in general case and it should work for HTTP3 as well at some point. But we should be careful with asp.net docs and examples, perhaps make some explicit comments. We had several cases in the past when people complained about slowness caused by this very issue.

[amcasey]

So the proposed guidance is that, if your server's OS supports ipv6, your server should listen on IPv6Any to reduce the number of retries?

[wfurt]

yes, that would be great. It will always work on Windows and since IPv6 is prevalent now it will likely work on most Unixes as well.

[amcasey]

Updated the comment and suggested an analyzer.