add support of TLS 1.3 on macOS POC #104835

wfurt · 2024-07-12T21:25:08Z

This contributes to #1979 that is long due. The API we currently use is deprecated since 10.5 Catalina and can possibly go at any time. Also with recent security push there may be more TLS 1.3 only servers and there really no good workaround for macOS and iOS so those platforms will simply fail to connect.

There are conceptually two challenges.

The new API does not provide capability to work on top of buffers like SslStream expects. To over come it we use Framer approach pretending we are implementing custom protocol. Something @filipnavara demonstrated here: https://gist.github.com/filipnavara/d5fb55bdb5edcceb1981f73078b855c4

The second is that there is no synchronize API. We basically call void function and we provide lambda that would be called once in the future. While that may be generally handy and similar to our 'async` it creates significant complication and platform difference.

To do that both handshake and decrypt have new Task that deliver events. I tried to structure it in a way that it has no impact on other platform. There can perhaps be improvements here. Certainly the "decrypt in place is a lieand we can possibly avoid some copies and memory use by refactoring. Original naive implementation was hitting assets for synchronous API - something I fixed by tracking sync vs async now and using synchronousWhenAny` when needed.

This PR is trying to close the gap by addressing both. It is still in rough shape but I would like to solicit early feedback before spending too much time polishing. I can do basic operations and pass most of the test locally:

=== TEST EXECUTION SUMMARY ===
     System.Net.Security.Tests  Total: 577, Errors: 0, Failed: 9, Skipped: 16, Time: 405.069s

Since the implementation is not complete I keep old one around as well and I use new one only when seems appropriate.
I also envision some possible compat switch so user can opt-in or opt-out as needed. The focus is on client for developers talking to server they cannot control. For simple HTTP(s) it should just work.

From failing tests, the new API is more picky about non ASCIS in SNI name. Something we saw on Android, but it should not matter for HTTP as we puny-code IDN.

Some tests (particularly around cipher suites) are confused as we cannot handshake TLS 1.3 in tests but the client side is capable of doing it.

And there are still some loose ends around closing, disposing and cleanup.

it would be great to get feedback, particularly from
@jkotas for inter
@stephentoub for async
@filipnavara for objective-C and macOS API use
@rzikm for integration with SslStream

and @dotnet/ncl for all the above. Note that the Network Framework can also do Quic so this may be option for use in the distant future.

dotnet-policy-service · 2024-07-12T21:25:34Z

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

src/native/libs/System.Security.Cryptography.Native.Apple/pal_ssl.m

rzikm

Looks interesting so far, but it feels like a substantial change which we are unlikely to get into 9.0

rzikm · 2024-07-15T12:19:28Z

src/libraries/Common/src/Interop/OSX/System.Security.Cryptography.Native.Apple/Interop.Ssl.cs


+        [LibraryImport(Interop.Libraries.AppleCryptoNative, EntryPoint = "AppleCryptoNative_NwCreateContext")]
+        internal static partial System.Net.SafeSslHandle NwCreateContext(int isServer);
+
+        [LibraryImport(Interop.Libraries.AppleCryptoNative, EntryPoint = "AppleCryptoNative_NwInit")]
+        internal static unsafe partial int NwInit(delegate* unmanaged<IntPtr, PAL_NwStatusUpdates, IntPtr, IntPtr, int> statusCallback, delegate* unmanaged<IntPtr, byte*, void**, int> readCallback, delegate* unmanaged<IntPtr, byte*, void**, int> writeCallback);


I assume Nw* and Ssl* functions can't be used interchangeably, would it make sense to move it to its own class like Interop.NetworkFramework?

This would be helpful, especially for the possible future implementations/experiments. (QUIC API is there and can be used with couple of more interop additions to this. (there are some bugs))

src/libraries/System.Net.Security/src/System/Net/Security/SslStream.IO.cs

rzikm · 2024-07-15T12:48:30Z

src/libraries/System.Net.Security/src/System/Net/Security/Pal.OSX/SafeDeleteSslContext.cs

+                int toWrite = (int)length;
+                var inputBuffer = new ReadOnlySpan<byte>(data, toWrite);
+
+                if (context.UseNwFramework)


There are now lots of such branches all over the codebase, It might be a lot of work but reshuffling the code s.t code-paths which uses NetworkFramework is in it's own class (SafeDeleteNwSslContext maybe?) would help readability IMO.

src/libraries/System.Net.Security/src/System/Net/SecurityStatusPal.cs

wfurt · 2024-07-15T17:13:30Z

Looks interesting so far, but it feels like a substantial change which we are unlikely to get into 9.0

My goal would be to make it op-in feature like we did for managed NTLM. e.g. make minimal impact by default but provide something for users who would be stuck otherwise.

src/libraries/Common/src/Interop/OSX/System.Security.Cryptography.Native.Apple/Interop.Ssl.cs

src/libraries/Common/src/System/Net/ArrayBuffer.cs

src/libraries/Common/tests/StreamConformanceTests/System/IO/StreamConformanceTests.cs

src/libraries/System.Net.Security/src/System/Net/Security/Pal.OSX/SafeDeleteSslContext.cs

stephentoub · 2024-07-19T21:55:07Z

src/libraries/System.Net.Security/src/System/Net/Security/SslStream.IO.cs

+            // if we have background task eat any exceptions
+            _frameTask?.ContinueWith(t => {
+                     _ = t.Exception;
+                     _buffer.ReturnBuffer();


What prevents the buffer from being returned twice?

src/libraries/System.Net.Security/src/System/Net/Security/SslStream.IO.cs

stephentoub · 2024-07-19T22:12:54Z

src/libraries/System.Net.Security/src/System/Net/Security/Pal.OSX/SafeDeleteSslContext.cs

+                // We grab reference to prevent disposal while handshake is pending.
+                this.DangerousAddRef(ref add);
+                ObjectDisposedException.ThrowIf(_disposed, this);
+                Interop.AppleCrypto.NwStartHandshake(SslContext, GCHandle.ToIntPtr(gcHandle));


Seems like you might need some GC.KeepAlives in various places. This gcHandle is a weak handle, which means it's not keeping this alive, which means if nothing after the call to this instance method is guaranteed to be keeping the instance alive, it's possible it could be collected at a bad time around here.

dotnet-policy-service · 2024-08-18T23:24:53Z