System.Net.Security.Tests.SslStreamDisposeTest.Dispose_ParallelWithHandshake_ThrowsODE crashes with Debug.Assert in SetSslCertificate

[ManickaP]

Description

System.Net.Security.Tests.SslStreamDisposeTest.Dispose_ParallelWithHandshake_ThrowsODE (OuterLoop) crashes the test process with a Debug.Assert failure (SIGABRT, exit code 134) on Azure Linux 3.0 x64.

Stack Trace

Process terminated.
Assertion failed.
certPtr != null && !certPtr.IsInvalid
   at Interop.OpenSsl.SetSslCertificate(SafeSslContextHandle contextPtr, SafeX509Handle certPtr, SafeEvpPKeyHandle keyPtr)
     in src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OpenSsl.cs:line 1063
   at Interop.OpenSsl.AllocateSslContext(SslAuthenticationOptions sslAuthenticationOptions, SslProtocols protocols, Boolean enableResume)
     in src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OpenSsl.cs:line 285
   at Interop.OpenSsl.<>c.<GetOrCreateSslContextHandle>b__14_0(ValueTuple`3 args)
     in src/libraries/Common/src/Interop/Unix/System.Security.Cryptography.Native/Interop.OpenSsl.cs:line 202
   at System.Net.SafeHandleCache`2.GetOrCreate[TContext](TKey key, Func`2 factory, TContext factoryContext)
   at Interop.OpenSsl.GetOrCreateSslContextHandle(SslAuthenticationOptions sslAuthenticationOptions, Boolean allowCached)
   at Interop.OpenSsl.AllocateSslHandle(SslAuthenticationOptions sslAuthenticationOptions)
   at System.Net.Security.SslStreamPal.HandshakeInternal(SafeDeleteSslContext& context, ...)
   at System.Net.Security.SslStreamPal.AcceptSecurityContext(...)
   at System.Net.Security.SslStream.GenerateToken(...)
   at System.Net.Security.SslStream.NextMessage(...)
   at System.Net.Security.SslStream.ProcessTlsFrame(Int32 frameSize)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](...)
   ...
   at System.Net.Security.Tests.SslStreamDisposeTest.<>c__DisplayClass3_1.<Dispose_ParallelWithHandshake_ThrowsODE>b__3()
     in src/libraries/System.Net.Security/tests/FunctionalTests/SslStreamDisposeTest.cs:line 136

Analysis

The test runs 10,000 parallel iterations that start a TLS handshake and immediately dispose the SslStream. During the race between the server-side AuthenticateAsServerAsync handshake and Dispose(), the CertificateContext.CertificateHandle becomes invalid by the time AllocateSslContext passes it to SetSslCertificate. The Debug.Assert on the certificate handle validity fires and aborts the process instead of throwing an ObjectDisposedException.

The Debug.Assert at Interop.OpenSsl.cs:1063 should probably be converted to a proper check that throws an appropriate exception (e.g., ObjectDisposedException) to handle the dispose-during-handshake race gracefully.

Environment

• OS: Azure Linux 3.0 (amd64), Docker image mcr.microsoft.com/dotnet-buildtools/prereqs:azurelinux-3.0-helix-amd64
• Test category: OuterLoop
• Helix job: 56ac9818-ceaa-4127-8348-605724497d33

Log

Full Helix console log

Observed in PR #124591 (unrelated change — HTTP3 nginx interop test re-enable).

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/ncl, @bartonjs, @vcsjones
See info in area-owners.md if you want to be subscribed.

[vcsjones]

The Debug.Assert at Interop.OpenSsl.cs:1063 should probably be converted to a proper check that throws an appropriate exception (e.g., ObjectDisposedException) to handle the dispose-during-handshake race gracefully.

You can probably just get rid of the assert. The marshaller at the p/invoke boundary will throw ODE if the SafeHandles are closed.