Add support for SocketsHttpHandler to use provided SPN

[mconnew]

WCF has the requirement to use non-default SPN's when using Kerberos authentication over HTTP. The AuthenticationHandler class specifies the SPN in this line of code. WCF needs a way to optionally specify which value is used on this line.
Ideally this would be exposed via an api mechanism available on HttpClientHandler/HttpClient/HttpRequestMessage so that WCF wouldn't need to dictate that a client always uses SocketsHttpHandler once available. We need to be able to specify either a mapping between hostname and SPN or to be able to specify on a per-request basis. This is especially important as there are likely to be some compatibility edge cases with SocketsHttpHandler once released and we would need to allow developers to specify which implementation that WCF will use.
An issue to expose the ability to specify the SPN in a general way to support mutual authentication was previously opened in 2015 in issue #15708. This issue is a more specific ask for this support to be available in SocketsHttpHandler and hopefully for it to be exposed in a more generic manner.

[davidsh]

cc: @geoffkizer @karelz @stephentoub

[geoffkizer]

I think you are asking for the equivalent of AuthenticationManager.CustomTargetNameDictionary on SocketsHttpHandler. Is that right?

This doesn't seem that hard to do.

[geoffkizer]

Does WCF allow arbitrary SPNs to be configured? Or is there some limited set of things it does?

[mconnew]

We allow any valid SPN or UPN to be used so we need the ability to specify a freeform value.

[geoffkizer]

Ok, thanks.

First step here would be an API proposal.

[karelz]

@mconnew I expect that we will add new features and APIs primarily into SocketsHttpHandler. I do not expect that we will keep adding the same feature into WinHttp/LibCurl handlers once SocketsHttpHandler is proven & mainstream (hopefully even during 2.1).

[mconnew]

@karelz, I understand that the existing implementations aren't expected to get it. There are multiple ways to do this, for example you could bring over the AuthenticationManager.CustomTargetNameDictionary api and have SocketsHttpHandler use that to find the mapping, that way HttpClientHandler wrapping SocketsHttpHandler could still be used. Another option is to add it to HttpClientHandler and add a SupportXXXX property. This could be supported on the full framework which wraps HttpWebRequest so there would be more than one implementation which supports this api. Adding api surface in a general location doesn't mean it needs to be supported everywhere but would provide greater flexibility.

[karelz]

Good, I was just checking your expectations :) ... I think we are aligned here.

[mconnew]

Just to put some context on this, we believe this is a blocker for some scenarios running WCF services in a container. When running a service in a container, even if the host is domain joined, the container image isn't fully domain joined (things gets very murky as there's no corresponding machine account in the domain). This normally means you can't do Windows authentication to an HTTP service/website running on a container. You can configure a domain gMSA account which the local SYSTEM account gets mapped to. My expectation is that this should allow running a service with a upn identity. WCF supports this on the full framework, but without support for specifying the HTTP server Kerberos identity, a .Net Core client can't use Windows auth.

[karelz]

OK, that makes it more needed.

The code for reference where we set SPN today:
https://github.com/dotnet/corefx/blob/158ed4d059a875e6828c176ce7ecc4a221b331e3/src/System.Net.Http/src/System/Net/Http/SocketsHttpHandler/AuthenticationHelper.NtAuth.cs#L61

SPN is tied to authentication. Adding it per request (likely into HttpRequestMessage) feels weird.
Maybe we should follow SocketsHttpHandler.Credentials model as SPN is similar to that ...

@mconnew did you have some specific API shape in mind?
cc @davidsh @geoffkizer

[davidsh]

In .NET Framework, we have an API that allows a dev to map custom SPN's to URI:

See example: https://stackoverflow.com/questions/39740676/forcing-specific-spn-for-url-in-net

Existing API: https://docs.microsoft.com/en-us/dotnet/api/system.net.authenticationmanager.customtargetnamedictionary?view=netframework-4.7.2

Maybe we should just implement this class on .NET Core and have the HTTP handlers use it. I don't think we need to define a new API for this.

[mconnew]

@davidsh, what I like about your idea is it would allow anyone's existing code using this capability to just work as the faux HttpWebRequest just wraps SocketsHttpHandler/HttpClientHandler. It would make migration easier.

[karelz]

AuthenticationManager is currently not hooked up anywhere - it is noop class.
https://github.com/dotnet/corefx/blob/aea91202c9ed213184562529e66959e602ec7826/src/System.Net.Requests/src/System/Net/AuthenticationManager.cs

The API:
https://github.com/dotnet/corefx/blob/aea91202c9ed213184562529e66959e602ec7826/src/System.Net.Requests/src/System/Net/AuthenticationManager.cs#L16

[geoffkizer]

We've generally moved away from using global-level settings like AuthenticationManager in HttpClient.

Seems like all we need here is CustomTargetNameDictionary on SocketsHttpHandler. Am I missing something?

[davidsh]

We've generally moved away from using global-level settings like AuthenticationManager in HttpClient.
Seems like all we need here is CustomTargetNameDictionary on SocketsHttpHandler. Am I missing something?

First, if we are going to add a new API to SocketsHttpHandler, we should do it to HttpClientHandler as well.

Second, this approach won't help folks still using HttpWebRequest APIs especially if porting from .NET Framework to .NET Core.

If we don't care about HttpWebRequest working with this scenario and we don't care about making people change their code when porting to .NET Core, then we can add this API to SocketsHttpHandler/HttpClientHandler.

[karelz]

Can we enable HttpWebRequest by initializing its SocketsHttpHandler with AuthenticationManager.CustomTargetNameDictionary value?

[geoffkizer]

Second, this approach won't help folks still using HttpWebRequest APIs especially if porting from .NET Framework to .NET Core.

Sure, but we've already got this issue all over the place, don't we? We don't support anything on AuthenticationManager today (or any of the ServicePoint stuff either).

I don't think we should use AuthenticationManager for just this one thing. We should either use it consistently and comprehensively, or not use it at all. Right now we don't use it at all.

[geoffkizer]

Can we enable HttpWebRequest by initializing its SocketsHttpHandler with AuthenticationManager.CustomTargetNameDictionary value?

Seems reasonable to me, since it limits the use of AuthenticationManager specifically to HttpWebRequest, not HttpClient generally.

I do wonder about other settings in AuthenticationManager, e.g. CredentialPolicy. Do we do anything with this today for HttpWebRequest on core?

[karelz]

There are no usages of AuthenticationManager in our code base: https://github.com/dotnet/corefx/search?utf8=%E2%9C%93&q=AuthenticationManager+&type=

[karelz]

Networking API discussion:

• Put it on SocketsHttpHandler (ideally not yet on HttpClientHandler)
• We need full understanding of Uri matching on Desktop. Should we still use string->string? Or something better?
• We need to decide on name - use SPN acronym or same name?
• How to implement it on Linux/Mac?

We should think how to hook it up to HttpWebRequest and SmtpClient eventually. Note that Handler doesn't allow changing properties after creation, while the global AuthenticationManager does.

[mconnew]

@karelz

• Put it on SocketsHttpHandler (ideally not yet on HttpClientHandler)

We consume HttpClientHandler, not SocketsHttpHandler. We recently added the ability to pass a delegate to WCF which takes an HttpClientHandler and returns a MessageHandler to allow developers to customize the HTTP handling so we can't change our internal usage to SocketsHttpHandler. Even if this wasn't the case, it would cause us to need to re-fork this code for UWP. You already have the SupportsFoo pattern for various capabilities so that seems like an ideal pattern for this too.

• We need to decide on name - use SPN acronym or same name?

It's actually not just an SPN that can be used. You can also use a UPN so the name SPN would be inappropriate.

• How to implement it on Linux/Mac?

This shouldn't be a problem. It's already a parameter passed to the code to handle the Nego protocol which is implemented on Linux and Mac. The last time I checked, there is only one implementation of the Negotiate authentication protocol in SocketsHttpHandler. I believe this has already been dealt with in the Nego PAL which uses gssapi on *nix.

[karelz]

We consume HttpClientHandler, not SocketsHttpHandler.

IMO we should wait for the time we are ready to flip HttpClientHandler fully to SocketsHttpHandler (I hope in 2.2). That will be the right time to push all SocketsHttpClient APIs to HttpClientHandler as well.
Either way, let's first decide on SocketsHttpHandler API and implementation, then we can start discussing when and how it will be available on HttpClientHandler. Using the Supports* pattern for a transition period seems overkill.

[karelz]

Triage: We need API design. Ideally in 5.0 to unblock WCF client.

[mconnew]

Here are some 3 api suggestions. My own criteria is for this to work for net standard 2.0.

My first thought was that this is related to credentials, so could be handled by the ICredentials interface. That interface has one method:

    NetworkCredential? GetCredential(Uri uri, string authType);

If you treated the auth type value "SPN" or "ServicePrincipalName" as having special meaning, you could return a NetworkCredential for the SPN. If it's an SPN, it's populated in the Domain property. If the Domain property is empty, then it's a UPN in the UserName property. This approach does have the problem that CredentialCache isn't safe to modify once handed to HttpClientHandler so I would need to create my own implementation of ICredentials, but that's not a big deal.

Another option is to have something that you can add to HttpClientHandler.Properties. You could use a key name ServicePrincipalNameMap and a value of IDictionary<string,string>. On top of that you could add a property on HttpClientHandler called ServicePrinicpalNameMap or SpnMap which is just a helper property which sets or returns the dictionary from the Properties. This way it's still useable from .net standard 2.0.

Another way is (and this is how I believe SocketsHttpHandler should have been implemented in the first place) is to make the authentication in SocketsHttpHandler completely composable. Create a DelegatingHandler implementation for each of the supported authentication schemes so that priority order is easily configured by stacking them in the right order. E.g. if you prefer windows auth over digest, the wrapping order would be Digest -> Windows -> SocketHttpHandler. Create a WindowsAuthenticationHandler with a strongly typed property to specify the SPN map. These composable authentication handlers would be available in a nuget package which supports .NET Standard 2.0.

[prakashguru]

@karelz @mconnew Have any idea on when will this get implemented?

[terrajobst]

@mconnew

Here are some 3 api suggestions. My own criteria is for this to work for net standard 2.0.

Could you describe why you need this for .NET Standard 2.0? We generally don't plan to ship features to older versions unless there is a strong business need for that. All feature work generally goes to .NET vNext only.

[mconnew]

@terrajobst, WCF Core client still only depends on .netstandard2.0. When we ship new features, we frequently get asked by customers waiting for the feature if they need to upgrade to the latest version of .NET to take advantage of it. Because of this, we've continued to only depend on netstandard2.0.

We don't necessarily need the custom SPN feature to work on earlier versions, but implementing this featrue using existing api's like my first two suggestions would allow us to continue targeting netstandard2.0 (we could probably move to the earliest still in support .NET Core version too, but we have no api's we depend on that require that). In that case we would do a runtime version check and use the existing hacked approach of modifying the Host header as a best effort for older runtime versions and use the new mechanism on newer runtimes. The key thing for us is to not require an api which demands a later runtime version.

Basically if our releases require the latest .NET runtime, then why are we OOB on nuget instead of shipping in box with the runtime?

[wfurt]

While it seems like we do not have clear directions here, starting with 8.0 users can help themselves. With something like the fragment bellow they can control all aspects including Impersonation.

using System.Diagnostics;
using System.Net;
using System.Net.Http;
using System.Net.Security;



var options = new NegotiateAuthenticationClientOptions();
options.Credential = new NetworkCredential("foo", "bar");
options.RequiredProtectionLevel = ProtectionLevel.Sign;
options.TargetName = "foo\\bar";
var nego = new NegotiateAuthentication(options);

var handler = new SocketsHttpHandler();
var client = new HttpClient(handler);


string? blob = nego.GetOutgoingBlob((string?)null, out NegotiateAuthenticationStatusCode status);

var uri = new Uri("http://testsite:8081/protected/");

var request = new HttpRequestMessage(HttpMethod.Get, uri);
request.Headers.Add("Authorization", "Negotiate " + blob);
blob = null;
var response = await client.SendAsync(request);

foreach (var header in response.Headers.WwwAuthenticate)
{
    if (header.Scheme == "Negotiate")
      
    {
        blob = nego.GetOutgoingBlob(header.Parameter, out status);
    }
}

Debug.Assert(blob != null);
request = new HttpRequestMessage(HttpMethod.Get, uri);
request.Headers.Add("Authorization", "Negotiate " + blob);

response = await client.SendAsync(request);
Console.WriteLine(response);

var request2 = new HttpRequestMessage(request.Method, request.RequestUri);
response = await client.SendAsync(request2);
Console.WriteLine(response);

But there are some caveats. There is no convenient way how to duplicate the original request and current implementation prevents second use of HttpRequestMessage. For simple cases one can simply copy Headers and Content but it may be more difficult in case of custom content.

For legacy HttpWebRequest we could respect the AuthenticationHandler to ease the migration.