-
Notifications
You must be signed in to change notification settings - Fork 297
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Random slow Kerberos negotiation #2204
Comments
It seems this latency occurs when creating new connection. (instead of reusing one from the pool) |
Also, it may be unrelated, but I also (sometimes) experience slowness when creating transactions (before sending actual queries).
|
I was experiencing similar issue. The slowness was caused by huge memory IO traffic. Unfortunately there is no official nuget yet published where the fix is included Edit: after reading through the issue again, it is probably a different issue but the symptoms sound very similar. You could try profiling memory allocations to check if that is the issue here. Also in the stacktrace you posted there is method "sync over async" that does not sound optimal. |
We will look into this. |
I could fix/workaround this different issue by disabling MARS. Luckily, it appears this feature is not required for my application. #422 is related to this one. |
Thanks for these ideas. I will try. Also, I was thinking I could try to migrate to .NET 7 as I have noticed Kerberos authentication implementation was different in latest M.D.SqlClient library starting from .NET 7 (Built-in class)... But this could be huge work... without guaranteed success.
Yes, this API contains much . NET 4.x migrated (synchronous) source code... That's why async calls are not always here... However, it should work better. |
@aloene is it possible for you to gather network traces to find out where the delay is happening? |
I have no network traces but I have profiled and then traced SQL connections creations yesterday and I found the issue mainly came from cumulated DNS queries durations (about 30-50ms each) |
I am closing the issue as it was very likely caused by DNS latencies as explained above. |
My ASP.NET 6.0 API uses Microsoft.Data.SqlClient 5.1.1 library to access an SQL Server 2019 DB (default instance) on a remote server (AlwaysOn).
This API is hosted on OpenShift (ubi8/dotnet-60-runtime image, rootless).
The API authenticates using Kerberos (with a keytab + kerberos config file) against SQL Server and... it works well.
Here is the connection string:
Data Source=listenerhost.my.domain.priv;Initial Catalog=MY.DB.NAME;integrated security=True;Connection Timeout=60;MultipleActiveResultSets=True;TrustServerCertificate=True;Command Timeout=900
BUT, sometimes (quite regularly), I face a latency (about 400-500ms) on SQL Server queries. I have profiled the API and the culprit is the authentication step here (400-500ms) :
Microsoft.Data.SqlClient!System.Net.Security.NegotiateStreamPal.GssInitSecurityContext()
so it seems it relates to this call : https://github.com/dotnet/SqlClient/blob/6456d71414347238d4bd519dbfdf1d936ca4cea6/src/Microsoft.Data.SqlClient/netcore/src/Common/src/System/Net/Security/NegotiateStreamPal.Unix.cs#L50C63-L50C63I initially thought it may be linked to the Kerberos ticket renewal but it happens (at least when I checked) without any renewal involved (checked with klist).
I have also checked SPNs and they seems fine, SQL service account registered all SPNs : each node FQDN, Listener FQDN, with and without default port 1433.
I am not sure about the root cause... Could it be any network/DNS issues maybe ?
Or could it be linked to a bug or Kerberos bad configuration ? (See my config below)
Would you have an idea about it and the way I could troubleshoot and find the real cause ?
/etc/krb5.conf
/etc/krb5.conf.d/xxx content
Thank you.
The text was updated successfully, but these errors were encountered: