Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression in SPN generation in .NET 5.0 for non-standard HTTP ports #53193

Closed
mattpwhite opened this issue May 24, 2021 · 5 comments · Fixed by #57159
Closed

Regression in SPN generation in .NET 5.0 for non-standard HTTP ports #53193

mattpwhite opened this issue May 24, 2021 · 5 comments · Fixed by #57159
Assignees
Milestone

Comments

@mattpwhite
Copy link
Contributor

It appears that SocketsHttpHandler was changed in .NET 5.0 (#40860) to unconditionally include the port in the Kerberos service principal name when the HTTP service runs on a non-default port. The intent of that change appears to have been to emulate the behavior of .NET Framework, but it actually created a divergence from .NET Framework behavior, at least for environments that expect out-of-the-box default behavior (no port in SPN).

The inclusion of the port in the SPN is a thing IE could be configured to do (the original KB is lost to time, but it's referenced many places, e.g. https://techcommunity.microsoft.com/t5/ask-the-directory-services-team/internet-explorer-behaviors-with-kerberos-authentication/ba-p/396428). To the best of my knowledge, no browser includes the port by default. Chrome has an option (again, off-by-default) to include the port - https://www.chromium.org/developers/design-documents/http-authentication (see "Kerberos SPN Generation"). Perhaps there was also something that could be done to configure .NET Framework to do this also, but again, it is not the default.

My guess is that this change broke fewer things than one might expect because, in a pure Windows shop, assuming security hasn't been tightened, you get an auto-magic downgrade to NTLM after the KDC returns an error because a principal with a port number in it does not exist. If NTLM is unsupported for one reason or another (Linux stuff in the mix, you care about security and have disabled NTLM) the fallback won't happen and things just don't work.

Given the following test program, I observe consistent behavior in .NET 4.8 and .NET Core 3.1, but .NET 5.0 diverges. My test below only has Windows output, but this regression is also observed in Linux .NET Core clients. This is blocking our ability to port several legacy codebases to .NET Core or upgrade from .NET Core 3.1.

Client code:

using System;
using System.Diagnostics;
using System.Net.Http;
using System.Runtime.InteropServices;
using System.Threading.Tasks;

namespace SpnPortDemo
{
    class Program
    {
        static async Task Main(string[] args)
        {
            Console.WriteLine($"Running under: {RuntimeInformation.FrameworkDescription}");
            RunConsoleApp("klist", "purge");

            try
            {
                var client = new HttpClient(new HttpClientHandler {UseDefaultCredentials = true});
                Console.WriteLine("Sending HTTP GET to " + args[0]);
                using (var response = await client.GetAsync(args[0]))
                {
                    Console.WriteLine($"Status: {response.StatusCode}");
                    Console.WriteLine("Body:");
                    Console.WriteLine(await response.Content.ReadAsStringAsync());
                }
            }
            catch (HttpRequestException e)
            {
                Console.WriteLine(e);
            }

            RunConsoleApp("klist");
        }

        private static void RunConsoleApp(string file, string args = null)
        {
            var psi = new ProcessStartInfo
            {
                FileName = file,
                Arguments = args,
                CreateNoWindow = true,
                UseShellExecute = false,
                RedirectStandardOutput = true,
            };

            Console.WriteLine();
            Console.WriteLine($"Output from: {file} {args}");
            var proc = Process.Start(psi);
            Console.WriteLine(proc.StandardOutput.ReadToEnd());
            Console.WriteLine();
        }
    }
}

Dummy (Windows only) server code (run as SYSTEM using psexec -sid cmd)

using System;
using System.Net;
using System.Text;
using System.Threading.Tasks;

namespace ListenerTest
{
    class Program
    {
        static int Main(string[] args)
        {
            if (args.Length != 1 || !ushort.TryParse(args[0], out var port))
            {
                Console.WriteLine("Failed to parse port");
                return 1;
            }

            var listener = new HttpListener();
            listener.Prefixes.Add($"http://+:{port}/");
            listener.AuthenticationSchemes = AuthenticationSchemes.Negotiate;
            Console.WriteLine($"Starting listener on port {port}");
            listener.Start();

            Task.Run(() =>
            {
                try
                {
                    while (true)
                    {
                        var ctx = listener.GetContext();
                        var bytes = Encoding.UTF8.GetBytes($"{ctx.User.Identity.AuthenticationType}: {ctx.User.Identity.Name}");
                        ctx.Response.ContentLength64 = bytes.Length;
                        ctx.Response.OutputStream.Write(bytes, 0, bytes.Length);
                        ctx.Response.OutputStream.Close();
                    }
                }
                catch (Exception e)
                {
                    if (!listener.IsListening)
                        return;

                    Console.WriteLine($"Error: {e}");
                }
            });

            Console.ReadLine();
            return 0;
        }
    }
}

.NET 4.8 output - Kerberos works (note client and server must be separate boxes or you'll get NTLM anyway):

Running under: .NET Framework 4.8.4341.0

Output from: klist purge

Current LogonId is 0:0x576ffb0
        Deleting all tickets:
        Ticket(s) purged!


Sending HTTP GET to http://testhost.domain.com:12345
Status: OK
Body:
Kerberos: DOMAIN\testuser

Output from: klist

Current LogonId is 0:0x576ffb0

Cached Tickets: (2)

#0>     Client: testuser @ DOMAIN.COM
        Server: krbtgt/DOMAIN.COM @ DOMAIN.COM
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize
        Start Time: 5/24/2021 12:59:55 (local)
        End Time:   5/24/2021 22:59:55 (local)
        Renew Time: 5/24/2021 22:59:55 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96
        Cache Flags: 0x1 -> PRIMARY
        Kdc Called: dc1.domain.com

#1>     Client: testuser @ DOMAIN.COM
        Server: HTTP/testhost.domain.com @ DOMAIN.COM
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize
        Start Time: 5/24/2021 12:59:55 (local)
        End Time:   5/24/2021 22:59:55 (local)
        Renew Time: 5/24/2021 22:59:55 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96
        Cache Flags: 0
        Kdc Called: dc1.domain.com

.NET Core 3.1, again Kerberos works

Running under: .NET Core 3.1.15

Output from: klist purge

Current LogonId is 0:0x576ffb0
        Deleting all tickets:
        Ticket(s) purged!


Sending HTTP GET to http://testhost.domain.com:12345
Status: OK
Body:
Kerberos: DOMAIN\testuser

Output from: klist

Current LogonId is 0:0x576ffb0

Cached Tickets: (2)

#0>     Client: testuser @ DOMAIN.COM
        Server: krbtgt/DOMAIN.COM @ DOMAIN.COM
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize
        Start Time: 5/24/2021 13:01:49 (local)
        End Time:   5/24/2021 23:01:49 (local)
        Renew Time: 5/24/2021 23:01:49 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96
        Cache Flags: 0x1 -> PRIMARY
        Kdc Called: dc1.domain.com

#1>     Client: testuser @ DOMAIN.COM
        Server: HTTP/testhost.domain.com @ DOMAIN.COM
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize
        Start Time: 5/24/2021 13:01:49 (local)
        End Time:   5/24/2021 23:01:49 (local)
        Renew Time: 5/24/2021 23:01:49 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96
        Cache Flags: 0
        Kdc Called: dc1.domain.com

But in .NET 5.0, Kerberos fails. A packet capture shows the TGS-REQ for HTTP/testhost.domain.com:12345, which isn't found, which then triggers a fallback to NTLM. My dummy server/test environment here has NTLM enabled, so things "work". Our production environment does not support does not support NTLM.

Running under: .NET 5.0.6

Output from: klist purge

Current LogonId is 0:0x576ffb0
        Deleting all tickets:
        Ticket(s) purged!


Sending HTTP GET to http://testhost.domain.com:12345
Status: OK
Body:
NTLM: DOMAIN\testuser

Output from: klist

Current LogonId is 0:0x576ffb0

Cached Tickets: (1)

#0>     Client: testuser @ DOMAIN.COM
        Server: krbtgt/DOMAIN.COM @ DOMAIN.COM
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize
        Start Time: 5/24/2021 15:28:29 (local)
        End Time:   5/25/2021 1:28:29 (local)
        Renew Time: 5/25/2021 1:28:29 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96
        Cache Flags: 0x1 -> PRIMARY
        Kdc Called: dc2.domain.com

I'm sure there are environments that do depend on the port in the SPN, so simply reverting might cause problems for them. .NET Core should be changed to either do it conditionally in the same manner as .NET Framework (assuming the framework checked/inherited behavior from some system level setting), or to expose a configurable knob for people that need it to opt in.

@dotnet-issue-labeler dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label May 24, 2021
@dotnet-issue-labeler
Copy link

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

@ghost
Copy link

ghost commented May 25, 2021

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

It appears that SocketsHttpHandler was changed in .NET 5.0 (#40860) to unconditionally include the port in the Kerberos service principal name when the HTTP service runs on a non-default port. The intent of that change appears to have been to emulate the behavior of .NET Framework, but it actually created a divergence from .NET Framework behavior, at least for environments that expect out-of-the-box default behavior (no port in SPN).

The inclusion of the port in the SPN is a thing IE could be configured to do (the original KB is lost to time, but it's referenced many places, e.g. https://techcommunity.microsoft.com/t5/ask-the-directory-services-team/internet-explorer-behaviors-with-kerberos-authentication/ba-p/396428). To the best of my knowledge, no browser includes the port by default. Chrome has an option (again, off-by-default) to include the port - https://www.chromium.org/developers/design-documents/http-authentication (see "Kerberos SPN Generation"). Perhaps there was also something that could be done to configure .NET Framework to do this also, but again, it is not the default.

My guess is that this change broke fewer things than one might expect because, in a pure Windows shop, assuming security hasn't been tightened, you get an auto-magic downgrade to NTLM after the KDC returns an error because a principal with a port number in it does not exist. If NTLM is unsupported for one reason or another (Linux stuff in the mix, you care about security and have disabled NTLM) the fallback won't happen and things just don't work.

Given the following test program, I observe consistent behavior in .NET 4.8 and .NET Core 3.1, but .NET 5.0 diverges. My test below only has Windows output, but this regression is also observed in Linux .NET Core clients. This is blocking our ability to port several legacy codebases to .NET Core or upgrade from .NET Core 3.1.

Client code:

using System;
using System.Diagnostics;
using System.Net.Http;
using System.Runtime.InteropServices;
using System.Threading.Tasks;

namespace SpnPortDemo
{
    class Program
    {
        static async Task Main(string[] args)
        {
            Console.WriteLine($"Running under: {RuntimeInformation.FrameworkDescription}");
            RunConsoleApp("klist", "purge");

            try
            {
                var client = new HttpClient(new HttpClientHandler {UseDefaultCredentials = true});
                Console.WriteLine("Sending HTTP GET to " + args[0]);
                using (var response = await client.GetAsync(args[0]))
                {
                    Console.WriteLine($"Status: {response.StatusCode}");
                    Console.WriteLine("Body:");
                    Console.WriteLine(await response.Content.ReadAsStringAsync());
                }
            }
            catch (HttpRequestException e)
            {
                Console.WriteLine(e);
            }

            RunConsoleApp("klist");
        }

        private static void RunConsoleApp(string file, string args = null)
        {
            var psi = new ProcessStartInfo
            {
                FileName = file,
                Arguments = args,
                CreateNoWindow = true,
                UseShellExecute = false,
                RedirectStandardOutput = true,
            };

            Console.WriteLine();
            Console.WriteLine($"Output from: {file} {args}");
            var proc = Process.Start(psi);
            Console.WriteLine(proc.StandardOutput.ReadToEnd());
            Console.WriteLine();
        }
    }
}

Dummy (Windows only) server code (run as SYSTEM using psexec -sid cmd)

using System;
using System.Net;
using System.Text;
using System.Threading.Tasks;

namespace ListenerTest
{
    class Program
    {
        static int Main(string[] args)
        {
            if (args.Length != 1 || !ushort.TryParse(args[0], out var port))
            {
                Console.WriteLine("Failed to parse port");
                return 1;
            }

            var listener = new HttpListener();
            listener.Prefixes.Add($"http://+:{port}/");
            listener.AuthenticationSchemes = AuthenticationSchemes.Negotiate;
            Console.WriteLine($"Starting listener on port {port}");
            listener.Start();

            Task.Run(() =>
            {
                try
                {
                    while (true)
                    {
                        var ctx = listener.GetContext();
                        var bytes = Encoding.UTF8.GetBytes($"{ctx.User.Identity.AuthenticationType}: {ctx.User.Identity.Name}");
                        ctx.Response.ContentLength64 = bytes.Length;
                        ctx.Response.OutputStream.Write(bytes, 0, bytes.Length);
                        ctx.Response.OutputStream.Close();
                    }
                }
                catch (Exception e)
                {
                    if (!listener.IsListening)
                        return;

                    Console.WriteLine($"Error: {e}");
                }
            });

            Console.ReadLine();
            return 0;
        }
    }
}

.NET 4.8 output - Kerberos works (note client and server must be separate boxes or you'll get NTLM anyway):

Running under: .NET Framework 4.8.4341.0

Output from: klist purge

Current LogonId is 0:0x576ffb0
        Deleting all tickets:
        Ticket(s) purged!


Sending HTTP GET to http://testhost.domain.com:12345
Status: OK
Body:
Kerberos: DOMAIN\testuser

Output from: klist

Current LogonId is 0:0x576ffb0

Cached Tickets: (2)

#0>     Client: testuser @ DOMAIN.COM
        Server: krbtgt/DOMAIN.COM @ DOMAIN.COM
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize
        Start Time: 5/24/2021 12:59:55 (local)
        End Time:   5/24/2021 22:59:55 (local)
        Renew Time: 5/24/2021 22:59:55 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96
        Cache Flags: 0x1 -> PRIMARY
        Kdc Called: dc1.domain.com

#1>     Client: testuser @ DOMAIN.COM
        Server: HTTP/testhost.domain.com @ DOMAIN.COM
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize
        Start Time: 5/24/2021 12:59:55 (local)
        End Time:   5/24/2021 22:59:55 (local)
        Renew Time: 5/24/2021 22:59:55 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96
        Cache Flags: 0
        Kdc Called: dc1.domain.com

.NET Core 3.1, again Kerberos works

Running under: .NET Core 3.1.15

Output from: klist purge

Current LogonId is 0:0x576ffb0
        Deleting all tickets:
        Ticket(s) purged!


Sending HTTP GET to http://testhost.domain.com:12345
Status: OK
Body:
Kerberos: DOMAIN\testuser

Output from: klist

Current LogonId is 0:0x576ffb0

Cached Tickets: (2)

#0>     Client: testuser @ DOMAIN.COM
        Server: krbtgt/DOMAIN.COM @ DOMAIN.COM
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize
        Start Time: 5/24/2021 13:01:49 (local)
        End Time:   5/24/2021 23:01:49 (local)
        Renew Time: 5/24/2021 23:01:49 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96
        Cache Flags: 0x1 -> PRIMARY
        Kdc Called: dc1.domain.com

#1>     Client: testuser @ DOMAIN.COM
        Server: HTTP/testhost.domain.com @ DOMAIN.COM
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize
        Start Time: 5/24/2021 13:01:49 (local)
        End Time:   5/24/2021 23:01:49 (local)
        Renew Time: 5/24/2021 23:01:49 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96
        Cache Flags: 0
        Kdc Called: dc1.domain.com

But in .NET 5.0, Kerberos fails. A packet capture shows the TGS-REQ for HTTP/testhost.domain.com:12345, which isn't found, which then triggers a fallback to NTLM. My dummy server/test environment here has NTLM enabled, so things "work". Our production environment does not support does not support NTLM.

Running under: .NET 5.0.6

Output from: klist purge

Current LogonId is 0:0x576ffb0
        Deleting all tickets:
        Ticket(s) purged!


Sending HTTP GET to http://testhost.domain.com:12345
Status: OK
Body:
NTLM: DOMAIN\testuser

Output from: klist

Current LogonId is 0:0x576ffb0

Cached Tickets: (1)

#0>     Client: testuser @ DOMAIN.COM
        Server: krbtgt/DOMAIN.COM @ DOMAIN.COM
        KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
        Ticket Flags 0x40e10000 -> forwardable renewable initial pre_authent name_canonicalize
        Start Time: 5/24/2021 15:28:29 (local)
        End Time:   5/25/2021 1:28:29 (local)
        Renew Time: 5/25/2021 1:28:29 (local)
        Session Key Type: AES-256-CTS-HMAC-SHA1-96
        Cache Flags: 0x1 -> PRIMARY
        Kdc Called: dc2.domain.com

I'm sure there are environments that do depend on the port in the SPN, so simply reverting might cause problems for them. .NET Core should be changed to either do it conditionally in the same manner as .NET Framework (assuming the framework checked/inherited behavior from some system level setting), or to expose a configurable knob for people that need it to opt in.

Author: mattpwhite
Assignees: -
Labels:

area-System.Net, untriaged

Milestone: -

@karelz
Copy link
Member

karelz commented May 25, 2021

Triage: Technically duplicate of #51701, but we may want to rethink this one ...

@karelz karelz removed the untriaged New issue has not been triaged by the area owner label May 25, 2021
@karelz karelz added this to the 6.0.0 milestone May 25, 2021
@karelz karelz added the bug label May 25, 2021
@mattpwhite
Copy link
Contributor Author

Sorry, didn't notice that existing issue. That linked issue mentions that this is conforming to a spec. Which spec is that? https://datatracker.ietf.org/doc/html/rfc4559 makes no mention of including a port. That RFC is basically a retcon of IE behavior (came 6 years after IE shipped the feature), and IE doesn't do this by default. Chrome and Firefox don't do this, nor does curl.

@ghost ghost added the in-pr There is an active PR which will close this issue when it is merged label Aug 10, 2021
@ghost ghost removed the in-pr There is an active PR which will close this issue when it is merged label Aug 16, 2021
@mattpwhite
Copy link
Contributor Author

Thanks for addressing this, much appreciated.

@ghost ghost locked as resolved and limited conversation to collaborators Sep 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants