System.InvalidOperationException: 'Internal connection fatal error.'

[fraserbooth]

Further to the discussion on issue #1116, here is a fresh issue as my situation appears to differ slightly in its root cause.

On my development machine, I've recently started experiencing a System.InvalidOperationException: 'Internal connection fatal error.' with .NET Core 3.1 and Microsoft.Data.SqlClient 3.0.0 on the latest stable as well as the pre-release 1,2&3 with the following code:

using System;
using Microsoft.Data.SqlClient;

namespace ConsoleApp1
{
	class Program
	{
		static void Main(string[] args)
		{
			using (var connection = new SqlConnection("Server=localhost;Database=Test1;Trusted_Connection=True;"))
			{
				connection.Open();
				Console.WriteLine("Connected successfully.");
				Console.ReadKey(true);
			}
		}
	}
}

The stack trace is:

at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)\r\n   at Microsoft.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)\r\n   at Microsoft.Data.SqlClient.SqlInternalConnectionTds.CompleteLogin(Boolean enlistOK)\r\n   at Microsoft.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, Boolean withFailover)\r\n   at Microsoft.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString connectionOptions, SqlCredential credential, TimeoutTimer timeout)\r\n   at Microsoft.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(TimeoutTimer timeout, SqlConnectionString connectionOptions, SqlCredential credential, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance)\r\n   at Microsoft.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, SqlCredential credential, Object providerInfo, String newPassword, SecureString newSecurePassword, Boolean redirectedUserInstance, SqlConnectionString userConnectionOptions, SessionData reconnectSessionData, Boolean applyTransientFaultHandling, String accessToken, DbConnectionPool pool)\r\n   at Microsoft.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)\r\n   at Microsoft.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnectionPool pool, DbConnection owningObject, DbConnectionOptions options, DbConnectionPoolKey poolKey, DbConnectionOptions userOptions)\r\n   at Microsoft.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)\r\n   at Microsoft.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject, DbConnectionOptions userOptions, DbConnectionInternal oldConnection)\r\n   at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, UInt32 waitForMultipleObjectsTimeout, Boolean allowCreate, Boolean onlyOneCheckConnection, DbConnectionOptions userOptions, DbConnectionInternal& connection)\r\n   at Microsoft.Data.ProviderBase.DbConnectionPool.TryGetConnection(DbConnection owningObject, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)\r\n   at Microsoft.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)\r\n   at Microsoft.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)\r\n   at Microsoft.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)\r\n   at Microsoft.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry, SqlConnectionOverrides overrides)\r\n   at Microsoft.Data.SqlClient.SqlConnection.Open(SqlConnectionOverrides overrides)\r\n   at Microsoft.Data.SqlClient.SqlConnection.Open()\r\n   at ConsoleApp1.Program.Main(String[] args) in C:\\Users\\FB-XPS\\source\\repos\\ConsoleApp1\\Program.cs:line 12

This issue started for me when I was forced to apply the latest Windows Update (KB5003637). I'm working on Win 10 Pro with VS2019 (16.9.4).

...any thoughts on ways to investigate and resolve this?

I'm using:
Microsoft SQL Server Developer (64-bit) 15.0.4138.2

Windows 10 Pro (v2004 build 19041.1052) updated as far as KB5003637

[cheenamalhotra]

Hi @fraserbooth

Thanks for opening a new ticket for this one, could you check if SSMS also behaves same way?
You can also compare behavior with sqlcmd as it doesn't seem to be caused by any driver update.

[fraserbooth]

Hi @cheenamalhotra

Thank you for following up over here :)

I explored both SSMS and sqlcmd, both of which are still behaving as expected.

It is only via the .NET Core 3.1 solutions that I'm able to reproduce this.

[cheenamalhotra]

Hi @fraserbooth

Could you also test with System.Data.SqlClient (.NET Core) to get a better picture of scope of issue.
And also if you can try with Microsoft.Data.SqlClient v2.1 and v1.1 latest patches in case it's our driver related.

[fraserbooth]

Well that was interesting, and hopefully will shed more light :)

Using System.Data.SqlClient doesn't give the error, and Microsoft.Data.SqlClient works up to v2.0, and starts failing from v2.1 onwards...

Does that help identify where the problem might be? I guess at some point then the approach connections or drivers changed?

Thank you for your continued exploration :)

All the best,

Fraser

[cheenamalhotra]

I believe your issue is similar to #1151 then.

To resolve, please ensure TLS 1.3 is Disabled for Server as SQL Server doesn't support TLS 1.3 right now.

[fraserbooth]

Hi @cheenamalhotra , sorry for the delayed response, I've been away this last week. I think I made the correct registry changes to disable TLS 1.3, but unfortunately this didn't fix the problem. I am however getting a new error of "Failed to generate SSPI context." which is at "Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)\r\n at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasC..."

Does that give any more clues?

Thank you again for all your help :)

[cheenamalhotra]

Hi @fraserbooth

I am however getting a new error of "Failed to generate SSPI context."

I'm suspecting this might be related to Kerberos authentication as we fixed some issues related to SPN validation. Could you verify your SPN configuration is correct?

More info: https://docs.microsoft.com/en-us/sql/database-engine/configure-windows/register-a-service-principal-name-for-kerberos-connections?view=sql-server-ver15

Alternatively, you could also try to disable/enable Named Pipes by altering connection string (Server=np:<servername>) and running some tests. You will need to check if there's a particular configuration in your case that leads to reproducing this issue. I couldn't reproduce this issue, having all updates installed with latest driver.

[fraserbooth]

Thank you for all your help with this. TLS 1.3 seemed to be the main culprit :)