SocketsHttpHandler cannot authenticate NTLM with username format user@domain.com

[rolfbjarne]

From @vinhdp195 on Fri, 27 Oct 2023 04:50:42 GMT

For Xamarin.iOS https://github.com/dotnet/designs/blob/main/accepted/2020/mono-convergence/platform-specific-httpclient.md#xamariniostvoswatchos
There are 3 handlers available (SocketsHttpHandler, CFNetworkHandler, and NSUrlSessionHandler)
After upgrading to NET7, it’s similar.

However, NTLM authentication is not working well.
SocketsHttpHandler cannot authenticate with a username in the form of user@domain.com.
There is a difference between NSUrlSessionHandler and SocketsHttpHandler.
I need to know why this difference exists.

Environment
Currently running on IOS devices with version 15/16 (Iphone/Ipad)
Visual Stuido 2022 version 17.5.4
Xcode_14.2
xamarin.ios-16.0.0.72
MonoFramework-MDK-6.12.0.182.macos10.xamarin.universal

Project sample:
https://github.com/vinhdp195/HttpRequestSample

Code sample:
https://github.com/vinhdp195/HttpRequestSample/blob/main/MauiApp1/MainPage.xaml.cs

In the above example source:
CheckHttpAsync() method uses NSUrlSessionHandler as a parameter for HttpClient
CheckSocketAsync() method uses SocketsHttpHandler as a parameter for HttpClient

User Infomation in AD:
user logon name: admin@sample.com
user logon (pre-windows 2000) : domain\samName
samAccountName is "samName"

Results when authenticating NTLM with SocketsHttpHandler and NSUrlSessionHandler:

No.	Username Input	NSUrlSessionHandler	SocketsHttpHandler
1	samName	〇	〇
2	domain\samName	〇	〇
3	admin@sample.com	〇	ｘ
4	samName@sample.com	ｘ	〇
5	samName@domain.com	ｘ	〇
6	samName@xyz.com	ｘ	〇

It appears that only username is samName can be authenticated (OK) when using SocketsHttpHandler.

• At No.3: using SocketsHttpHandler, it cannot authenticate with the username “admin@sample.com”, but NSUrlSessionHandler can.
• At No.4,5,6: using SocketsHttpHandler, it can authenticate with samName, however, any value after the @ character can also successfully authenticate (xyz is an incorrect value but still authenticates successfully).

What is the reason for the above difference? Is it a bug?
I want to be able to successfully authenticate at No.3 using SocketsHttpHandler, what should I do?
(Besides, the reason I have to use SocketsHttpHandler for my project is because NSUrlSessionHandler on iOS does not support passing in WebProxy)

Copied from original issue xamarin/xamarin-macios#19366

[rolfbjarne]

From @vinhdp195 on Wed, 01 Nov 2023 08:04:32 GMT

I want to be able to successfully authenticate at No.3 using SocketsHttpHandler, what should I do?

[rolfbjarne]

From @rolfbjarne on Thu, 02 Nov 2023 16:31:30 GMT

Moving to dotnet/runtime since that's where the code for SocketsHttpHandler lives, and it seems that's what this issue is about.

[ghost]

Tagging subscribers to this area: @dotnet/ncl
See info in area-owners.md if you want to be subscribed.

Issue Details

---

From @vinhdp195 on Fri, 27 Oct 2023 04:50:42 GMT

For Xamarin.iOS https://github.com/dotnet/designs/blob/main/accepted/2020/mono-convergence/platform-specific-httpclient.md#xamariniostvoswatchos
There are 3 handlers available (SocketsHttpHandler, CFNetworkHandler, and NSUrlSessionHandler)
After upgrading to NET7, it’s similar.

However, NTLM authentication is not working well.
SocketsHttpHandler cannot authenticate with a username in the form of user@domain.com.
There is a difference between NSUrlSessionHandler and SocketsHttpHandler.
I need to know why this difference exists.

Environment
Currently running on IOS devices with version 15/16 (Iphone/Ipad)
Visual Stuido 2022 version 17.5.4
Xcode_14.2
xamarin.ios-16.0.0.72
MonoFramework-MDK-6.12.0.182.macos10.xamarin.universal

Project sample:
https://github.com/vinhdp195/HttpRequestSample

Code sample:
https://github.com/vinhdp195/HttpRequestSample/blob/main/MauiApp1/MainPage.xaml.cs

In the above example source:
CheckHttpAsync() method uses NSUrlSessionHandler as a parameter for HttpClient
CheckSocketAsync() method uses SocketsHttpHandler as a parameter for HttpClient

User Infomation in AD:
user logon name: admin@sample.com
user logon (pre-windows 2000) : domain\samName
samAccountName is "samName"

Results when authenticating NTLM with SocketsHttpHandler and NSUrlSessionHandler:

No.	Username Input	NSUrlSessionHandler	SocketsHttpHandler
1	samName	〇	〇
2	domain\samName	〇	〇
3	admin@sample.com	〇	ｘ
4	samName@sample.com	ｘ	〇
5	samName@domain.com	ｘ	〇
6	samName@xyz.com	ｘ	〇

It appears that only username is samName can be authenticated (OK) when using SocketsHttpHandler.

• At No.3: using SocketsHttpHandler, it cannot authenticate with the username “admin@sample.com”, but NSUrlSessionHandler can.
• At No.4,5,6: using SocketsHttpHandler, it can authenticate with samName, however, any value after the @ character can also successfully authenticate (xyz is an incorrect value but still authenticates successfully).

What is the reason for the above difference? Is it a bug?
I want to be able to successfully authenticate at No.3 using SocketsHttpHandler, what should I do?
(Besides, the reason I have to use SocketsHttpHandler for my project is because NSUrlSessionHandler on iOS does not support passing in WebProxy)

Copied from original issue xamarin/xamarin-macios#19366

Author:	rolfbjarne
Assignees:	-
Labels:	area-System.Net.Http, untriaged
Milestone:	-

[rolfbjarne]

CC @steveisok

[steveisok]

/cc @akoeplinger @simonrozsival

[wfurt]

cc: @filipnavara AFAIK the admin@sample.com should work. I don't understand the difference between 3 & 4.

[filipnavara]

Apple's GSSAPI implementation of NTLM always rewrites admin@sample.com into sample.com\admin, ie. always extracts the domain. This behavior is incorrect but happens to work in many cases. NSUrlSession has its own NTLM implementation that doesn't do this credential rewriting. This is one of the reasons why we exposed UseManagedNtlm in .NET 8 (and made it default for .NET 9 on macOS).

[filipnavara]

I'll try to post an example on how to use UseManagedNtlm later if no one else beats me to it. The gist is that the GSSAPI doesn't work correctly for samName != admin. The names are usually the same but longer ones get truncated, etc. Similarly, this breaks some domain fallback paths when the application server is not on the same domain as the authentication server.

[filipnavara]

@vinhdp195 It would be useful if you can test with the UseManagedNtlm option on .NET 8. To enable it, you can add this to the .csproj file:

<ItemGroup>
    <RuntimeHostConfigurationOption Include="System.Net.Security.UseManagedNtlm" Value="true" />
</ItemGroup>

This should replace the NTLM implementation for SocketsHttpHandler with a managed one written in .NET. This the same implementation that shipped on Android in .NET 7.

[vinhdp195]

@filipnavara

It would be useful if you can test with the UseManagedNtlm option on .NET 8. To enable it, you can add this to the .csproj file:

I tried as you suggested.
The environment is as follows:
.NET 8.0 (8.0.100-preview.3)
Visual studio Enterprise 2022 Version 17.7.6
Device : Ipad ios 16.1

Source code:
https://github.com/vinhdp195/HttpRequestSample/tree/Net8
https://github.com/vinhdp195/HttpRequestSample/blob/Net8/MauiApp3/MauiApp3.csproj
Line 49:

Results:
In case No.3, SocketsHttpHandler still cannot authenticate. It returns Error Code 401.

In your environment, with the source code as above, is it possible to authenticate successfully (Code 200 OK) or not?

[CarnaViire]

Triage: we should investigate in 9.0

[vinhdp195]

@filipnavara @CarnaViire
Thank you for your answer.
Regarding the comment above, can I confirm that the latest Net 8 version rc.2 still cannot fix the bug?
Do I need to wait for the Net9 release to be able to check?

[filipnavara]

In your environment, with the source code as above, is it possible to authenticate successfully (Code 200 OK) or not?

I didn't get to test it yet, and I don't have the server environment readily available at the moment. I hope to test it soon, perhaps later this week.

[vinhdp195]

@filipnavara @CarnaViire
I have successfully tested with net8-rc2.
I was able to authenticate with UPN of patern No.3.

Environment as below:
Source code: https://github.com/vinhdp195/HttpRequestSample/tree/Net8-RC2
.NET version: 8.0.0-rc.2.23479.6
Device : Ipad Gen 6 (IOS 17.1)
Visual Studio version: 17.8.0 Preview 6
Xcode version : 15

[CarnaViire]

@vinhdp195 do I understand correctly that .NET 8 works properly in all cases, and this issue can be closed?

[akoeplinger]

Note that you can also set the <_UseManagedNtlm>true</_UseManagedNtlm> property in the csproj, it will set the same RuntimeHostConfigurationOption: https://github.com/dotnet/sdk/blob/992d660097927ddd4964b52219ea89d4175f66a1/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Sdk.targets#L558-L561

@filipnavara do you remember why this was added as a "private" msbuild property with underscore?

[filipnavara]

@akoeplinger That property was only added in .NET 9 and it was not backported to .NET 8 SDK. Underscore is generally used for unsupported/undocumented properties. In .NET 9 the default value for iOS/macOS was changed anyway, so the property would not help in this case.

[vinhdp195]

@CarnaViire

do I understand correctly that .NET 8 works properly in all cases, and this issue can be closed?

That's right. At Net8, after applying UseManagedNtlm, I was able to resolve the error this time.

[CarnaViire]

Thanks. Closing as answered.

[filipnavara]

@CarnaViire I'd like to ask again about backporting dotnet/sdk#34903 to .NET 8 SDK. It was done late in the .NET 8 cycle and the backport question somehow drowned in the noise. It adds the <_UseManagedNtlm>true</_UseManagedNtlm> property on SDK level instead of relying on <RuntimeHostConfigurationOption Include="System.Net.Security.UseManagedNtlm" Value="true" /> which is hard to discover and error-prone.

[CarnaViire]

I'm not sure it will meet the bar for a backport at this point... given that the app-context switch exists and works... cc @wfurt @dotnet/ncl @vitek-karas @sbomer