Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ClientAndServer_OneOrBothUseDefaultOk failing in CI #34337

Closed
jaredpar opened this issue Mar 31, 2020 · 5 comments · Fixed by #36875
Closed

ClientAndServer_OneOrBothUseDefaultOk failing in CI #34337

jaredpar opened this issue Mar 31, 2020 · 5 comments · Fixed by #36875
Assignees
@aik-jahoda
Labels
area-System.Net.Security blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' test-run-core Test failures in .NET Core test runs
Milestone
6.0.0

Comments

@jaredpar
Copy link
Member

Only one failure so far but the stack seemed pretty definitive here so filing the issue.

Console Log

System.Net.Security.Tests.ApmSslStreamSystemDefaultTest.ClientAndServer_OneOrBothUseDefault_Ok(clientProtocols: Default, serverProtocols: Ssl2 | Default | Tls11 | Tls12) [FAIL]
      System.ObjectDisposedException : Safe handle has been closed.
      Object name: 'SafeHandle'.
      Stack Trace:
        /_/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/SafeHandle.cs(149,0): at System.Runtime.InteropServices.SafeHandle.DangerousAddRef(Boolean& success)
        /_/src/libraries/Common/src/Interop/Windows/SspiCli/SecuritySafeHandles.cs(354,0): at System.Net.Security.SafeCredentialReference..ctor(SafeFreeCredentials target)
        /_/src/libraries/Common/src/Interop/Windows/SspiCli/SecuritySafeHandles.cs(341,0): at System.Net.Security.SafeCredentialReference.CreateReference(SafeFreeCredentials target)
        /_/src/libraries/System.Net.Security/src/System/Net/Security/SslSessionsCache.cs(201,0): at System.Net.Security.SslSessionsCache.CacheCredential(SafeFreeCredentials creds, Byte[] thumbPrint, SslProtocols sslProtocols, Boolean isServer, EncryptionPolicy encryptionPolicy)
        /_/src/libraries/System.Net.Security/src/System/Net/Security/SecureChannel.cs(830,0): at System.Net.Security.SecureChannel.GenerateToken(ReadOnlySpan`1 inputBuffer, Byte[]& output)
        /_/src/libraries/System.Net.Security/src/System/Net/Security/SecureChannel.cs(726,0): at System.Net.Security.SecureChannel.NextMessage(ReadOnlySpan`1 incomingBuffer)
           at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm)
        /_/src/libraries/Common/src/System/Threading/Tasks/TaskToApm.cs(42,0): at System.Threading.Tasks.TaskToApm.End(IAsyncResult asyncResult)
        /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.cs(233,0): at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult)
        /_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/FutureFactory.cs(513,0): at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
        --- End of stack trace from previous location ---
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(83,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(111,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks)
        /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs(71,0): at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout)
        /_/src/libraries/System.Net.Security/tests/FunctionalTests/SslStreamSystemDefaultsTest.cs(60,0): at System.Net.Security.Tests.SslStreamSystemDefaultTest.ClientAndServer_OneOrBothUseDefault_Ok(Nullable`1 clientProtocols, Nullable`1 serverProtocols)
        --- End of stack trace from previous location ---

System.Net.Security.Tests.ApmSslStreamSystemDefaultTest.ClientAndServer_OneOrBothUseDefault_Ok

Console Log Summary

Builds

Build Pull Request Test Failure Count
#580861 Rolling 1

Configurations

  • netcoreapp5.0-Windows_NT-Release-x86-CoreCLR_checked-Windows.10.Amd64.Open

Helix Logs

Build Pull Request Console Core Test Results Run Client
#580861 Rolling console.log testResults.xml run_client.py
@Dotnet-GitSync-Bot Dotnet-GitSync-Bot added area-System.Net.Security untriaged New issue has not been triaged by the area owner labels Mar 31, 2020
@alnikola alnikola removed the untriaged New issue has not been triaged by the area owner label Apr 30, 2020
@alnikola
Copy link
Contributor

Triage:
It might be a reference counting problem, but we need to first reproduce with latest master.

@alnikola alnikola added this to the 5.0 milestone Apr 30, 2020
@karelz karelz added the test-run-core Test failures in .NET Core test runs label May 7, 2020
@karelz karelz mentioned this issue May 7, 2020
Closed
@aik-jahoda aik-jahoda self-assigned this May 12, 2020
@aik-jahoda aik-jahoda mentioned this issue May 22, 2020
Merged
@karelz karelz modified the milestones: 5.0, Future Jun 5, 2020
@safern
Copy link
Member

safern commented Jun 19, 2020

Failed again... at least two times in the last 100 builds:

System.Net.Security.Tests.ApmSslStreamSystemDefaultTest.ClientAndServer_OneOrBothUseDefault_Ok

Console Log Summary

Builds

Build Pull Request Test Failure Count
#695699 #38004 1
#696589 Rolling 1

Configurations

  • net5.0-Windows_NT-Release-arm-CoreCLR_release-Windows.10.Arm64v8.Open
  • net5.0-Windows_NT-Release-x86-CoreCLR_checked-Windows.10.Amd64.Open

Helix Logs

Build Pull Request Console Core Test Results Run Client
#695699 #38004 console.log testResults.xml run_client.py
#696589 Rolling console.log testResults.xml run_client.py

@safern safern added the blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' label Jun 19, 2020
@jashook jashook mentioned this issue Jun 22, 2020
Closed
@wfurt wfurt mentioned this issue Jul 22, 2020
Closed
@v-haren
Copy link

v-haren commented Jul 28, 2020

failed again in job: runtime-coreclr libraries-jitstress 20200727.1

failed test: System.Net.Security.Tests.SyncSslStreamSystemDefaultTest.ClientAndServer_OneOrBothUseDefault_Ok(clientProtocols: Default, serverProtocols: Ssl2 | Default | Tls11 | Tls12)

Error message

System.ObjectDisposedException : Safe handle has been closed.
Object name: 'SafeHandle'.


Stack trace
   at System.Runtime.InteropServices.SafeHandle.InternalRelease(Boolean disposeOrFinalizeOperation) in /_/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/SafeHandle.cs:line 201
   at System.Runtime.InteropServices.SafeHandle.DangerousRelease() in /_/src/libraries/System.Private.CoreLib/src/System/Runtime/InteropServices/SafeHandle.cs:line 164
   at System.Net.Security.SafeDeleteContext.MustRunAcceptSecurityContext_SECURITY(SafeFreeCredentials& inCredentials, Boolean isContextAbsent, SecBufferDesc* inputBuffer, ContextFlags inFlags, Endianness endianness, SafeDeleteContext outContext, SecBufferDesc& outputBuffer, ContextFlags& outFlags, SafeFreeContextBuffer handleTemplate) in /_/src/libraries/Common/src/Interop/Windows/SspiCli/SecuritySafeHandles.cs:line 848
   at System.Net.Security.SafeDeleteContext.AcceptSecurityContext(SafeFreeCredentials& inCredentials, SafeDeleteSslContext& refContext, ContextFlags inFlags, Endianness endianness, InputSecurityBuffers inSecBuffers, SecurityBuffer& outSecBuffer, ContextFlags& outFlags) in /_/src/libraries/Common/src/Interop/Windows/SspiCli/SecuritySafeHandles.cs:line 741
   at System.Net.SSPISecureChannelType.AcceptSecurityContext(SafeFreeCredentials credential, SafeDeleteSslContext& context, InputSecurityBuffers inputBuffers, ContextFlags inFlags, Endianness endianness, SecurityBuffer& outputBuffer, ContextFlags& outFlags) in /_/src/libraries/Common/src/Interop/Windows/SspiCli/SSPISecureChannelType.cs:line 55
   at System.Net.SSPIWrapper.AcceptSecurityContext(ISSPIInterface secModule, SafeFreeCredentials credential, SafeDeleteSslContext& context, ContextFlags inFlags, Endianness datarep, InputSecurityBuffers inputBuffers, SecurityBuffer& outputBuffer, ContextFlags& outFlags) in /_/src/libraries/Common/src/Interop/Windows/SspiCli/SSPIWrapper.cs:line 160
   at System.Net.Security.SslStreamPal.AcceptSecurityContext(SafeFreeCredentials& credentialsHandle, SafeDeleteSslContext& context, ReadOnlySpan`1 inputBuffer, Byte[]& outputBuffer, SslAuthenticationOptions sslAuthenticationOptions) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStreamPal.Windows.cs:line 69
   at System.Net.Security.SecureChannel.GenerateToken(ReadOnlySpan`1 inputBuffer, Byte[]& output) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SecureChannel.cs:line 769
   at System.Net.Security.SecureChannel.NextMessage(ReadOnlySpan`1 incomingBuffer) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SecureChannel.cs:line 706
   at System.Net.Security.SslStream.ProcessBlob(Int32 frameSize) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 478
   at System.Net.Security.SslStream.ReceiveBlobAsync[TIOAdapter](TIOAdapter adapter) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 441
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](TIOAdapter adapter, Boolean receiveFirst, Byte[] reAuthenticationData, Boolean isApm) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 275
   at System.Net.Security.SslStream.ProcessAuthentication(Boolean isAsync, Boolean isApm, CancellationToken cancellationToken) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.Implementation.cs:line 201
   at System.Net.Security.SslStream.AuthenticateAsServer(SslServerAuthenticationOptions sslServerAuthenticationOptions) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.cs:line 359
   at System.Net.Security.SslStream.AuthenticateAsServer(X509Certificate serverCertificate, Boolean clientCertificateRequired, SslProtocols enabledSslProtocols, Boolean checkCertificateRevocation) in /_/src/libraries/System.Net.Security/src/System/Net/Security/SslStream.cs:line 346
   at System.Net.Security.Tests.SyncSslStreamSystemDefaultTest.<>c__DisplayClass1_0.<AuthenticateServerAsync>b__0() in /_/src/libraries/System.Net.Security/tests/FunctionalTests/SslStreamSystemDefaultsTest.cs:line 161
   at System.Threading.Tasks.Task.InnerInvoke() in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:line 2370
   at System.Threading.Tasks.Task.<>c.<.cctor>b__277_0(Object obj) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:line 2356
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs:line 241
--- End of stack trace from previous location ---
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs:line 266
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread) in /_/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:line 2320
--- End of stack trace from previous location ---
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 82
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 110
   at System.Threading.Tasks.TaskTimeoutExtensions.WhenAllOrAnyFailed(Task[] tasks, Int32 millisecondsTimeout) in /_/src/libraries/Common/tests/System/Threading/Tasks/TaskTimeoutExtensions.cs:line 70
   at System.Net.Security.Tests.SslStreamSystemDefaultTest.ClientAndServer_OneOrBothUseDefault_Ok(Nullable`1 clientProtocols, Nullable`1 serverProtocols) in /_/src/libraries/System.Net.Security/tests/FunctionalTests/SslStreamSystemDefaultsTest.cs:line 57
--- End of stack trace from previous location ---

@directhex
Copy link
Contributor

Definitely still a problem, e.g. https://helix.dot.net/api/2019-06-17/jobs/7ad9e59f-28f1-4e71-8f8c-69ddbc03698d/workitems/System.Net.Security.Tests/files/console.0b2bbc5d.log was within the last 2 weeks

@directhex
Copy link
Contributor

Actually, this is a duplicate of #2353 so closing here. Continue discussion there.

@ghost ghost locked as resolved and limited conversation to collaborators Dec 9, 2020
@karelz karelz modified the milestones: Future, 6.0.0 Jan 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@aik-jahoda aik-jahoda

Labels
area-System.Net.Security blocking-clean-ci Blocking PR or rolling runs of 'runtime' or 'runtime-extra-platforms' test-run-core Test failures in .NET Core test runs
Projects
None yet
Milestone
6.0.0
Development

Successfully merging a pull request may close this issue.

Fix invalid access of Target in SafeCredentialReference aik-jahoda/runtime
8 participants
@jaredpar @directhex @aik-jahoda @karelz @safern @v-haren @Dotnet-GitSync-Bot @alnikola