is ReadSyncOverAsync causing thread starvation?

[alvanxp]

Why EF call to ReadSyncOverAsync call?

I work on a oData project using EF Core 6.0.6 using async calls but it is creating double number of threads than the .net Framework v4.7.2 version of the same code, the big difference that I found that the hospot is the "Microsoft.Data.SqlClient.TdsParserStateObjectNative.ReadSyncOverAsync" method.
is that call causing a thread starvation?

Microsoft.EntityFrameworkCore/6.0.6
Microsoft.EntityFrameworkCore.SqlServer/6.0.6
Microsoft.AspNetCore.OData/8.0.8

Controller Code

This method is only returning the IQueryable, does not execute the db call

    [HttpPost("odata/AdvancedSearch")]
    [EnableQuery(MaxNodeCount = ODataMaxNodeCount)]
    public async Task<IQueryable<AdvancedSearchDto>> AdvancedSearch([FromBody]SearchRequestPayload searchRequestPayload)
    {
        var advancedSearch = await _searchService.AdvancedSearch(searchRequestPayload);
        return advancedSearch.AsQueryable();
    }

Stack Trace

This is the hostpot: ReadSyncOverAsync on the traces:

[Native Frames]
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.TdsParserStateObjectNative.ReadSyncOverAsync(int32,unsigned int32&)
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync()
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.TdsParserStateObject.TryReadNetworkPacket()
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.TdsParserStateObject.TryPrepareBuffer()
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.TdsParserStateObject.TryReadByte(unsigned int8&)
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.TdsParser.TryRun(value class Microsoft.Data.SqlClient.RunBehavior,class Microsoft.Data.SqlClient.SqlCommand,class Microsoft.Data.SqlClient.SqlDataReader,class Microsoft.Data.SqlClient.BulkCopySimpleResultSet,class Microsoft.Data.SqlClient.TdsParserStateObject,bool&)
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.SqlDataReader.TryConsumeMetaData()
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.SqlDataReader.get_MetaData()
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(class Microsoft.Data.SqlClient.SqlDataReader,value class Microsoft.Data.SqlClient.RunBehavior,class System.String,bool,bool,bool)
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.SqlCommand.RunExecuteReaderTds(value class System.Data.CommandBehavior,value class Microsoft.Data.SqlClient.RunBehavior,bool,bool,int32,class System.Threading.Tasks.Task&,bool,bool,class Microsoft.Data.SqlClient.SqlDataReader,bool)
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(value class System.Data.CommandBehavior,value class Microsoft.Data.SqlClient.RunBehavior,bool,class System.Threading.Tasks.TaskCompletionSource`1<class System.Object>,int32,class System.Threading.Tasks.Task&,bool&,bool,bool,class System.String)
  Microsoft.Data.SqlClient!Microsoft.Data.SqlClient.SqlCommand.ExecuteReader(value class System.Data.CommandBehavior)
  Microsoft.EntityFrameworkCore.Relational!Microsoft.EntityFrameworkCore.Storage.RelationalCommand.ExecuteReader(value class Microsoft.EntityFrameworkCore.Storage.RelationalCommandParameterObject)
  Microsoft.EntityFrameworkCore.Relational!Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1+Enumerator[System.Int64].InitializeReader(class Enumerator<!0>)
  Microsoft.EntityFrameworkCore!Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy+<>c__DisplayClass31_0`2[System.__Canon,System.Boolean].<Execute>b__0(class Microsoft.EntityFrameworkCore.DbContext,!0)
  Microsoft.EntityFrameworkCore!Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementation(class System.Func`3<class Microsoft.EntityFrameworkCore.DbContext,!!0,class Microsoft.EntityFrameworkCore.Storage.ExecutionResult`1<!!1>>,class System.Func`3<class Microsoft.EntityFrameworkCore.DbContext,!!0,class Microsoft.EntityFrameworkCore.Storage.ExecutionResult`1<!!1>>,!!0)
  Microsoft.EntityFrameworkCore!Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.Execute(!!0,class System.Func`3<class Microsoft.EntityFrameworkCore.DbContext,!!0,!!1>,class System.Func`3<class Microsoft.EntityFrameworkCore.DbContext,!!0,class Microsoft.EntityFrameworkCore.Storage.ExecutionResult`1<!!1>>)
  Microsoft.EntityFrameworkCore.Relational!Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1+Enumerator[System.Int64].MoveNext()
  System.Linq.il!System.Linq.Enumerable.TryGetSingle(class System.Collections.Generic.IEnumerable`1<!!0>,bool&)
  Anonymously Hosted DynamicMethods Assembly!dynamicClass.lambda_method134(pMT: 00007FFC3582E3F0,pMT: 00007FFC361D43E8)
  ?!?
  System.Private.CoreLib.il!System.Reflection.RuntimeMethodInfo.Invoke(class System.Object,value class System.Reflection.BindingFlags,class System.Reflection.Binder,class System.Object[],class System.Globalization.CultureInfo)
  Microsoft.AspNetCore.OData!Microsoft.AspNetCore.OData.Query.ExpressionHelpers+<>c__DisplayClass0_0.<Count>b__0()
  Microsoft.AspNetCore.OData!Microsoft.AspNetCore.OData.Abstracts.ODataFeature.get_TotalCount()
  Microsoft.AspNetCore.OData!Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer.CreateResourceSet(class System.Collections.IEnumerable,class Microsoft.OData.Edm.IEdmCollectionTypeReference,class Microsoft.AspNetCore.OData.Formatter.Serialization.ODataSerializerContext)
  Microsoft.AspNetCore.OData!Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer+<WriteResourceSetAsync>d__4.MoveNext()
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start(!!0&)
  Microsoft.AspNetCore.OData!Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer.WriteResourceSetAsync(class System.Collections.IEnumerable,class Microsoft.OData.Edm.IEdmTypeReference,class Microsoft.OData.ODataWriter,class Microsoft.AspNetCore.OData.Formatter.Serialization.ODataSerializerContext)
  Microsoft.AspNetCore.OData!Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer+<WriteObjectInlineAsync>d__3.MoveNext()
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start(!!0&)
  Microsoft.AspNetCore.OData!Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer.WriteObjectInlineAsync(class System.Object,class Microsoft.OData.Edm.IEdmTypeReference,class Microsoft.OData.ODataWriter,class Microsoft.AspNetCore.OData.Formatter.Serialization.ODataSerializerContext)
  Microsoft.AspNetCore.OData!Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer+<WriteObjectAsync>d__2.MoveNext()
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start(!!0&)
  Microsoft.AspNetCore.OData!Microsoft.AspNetCore.OData.Formatter.Serialization.ODataResourceSetSerializer.WriteObjectAsync(class System.Object,class System.Type,class Microsoft.OData.ODataMessageWriter,class Microsoft.AspNetCore.OData.Formatter.Serialization.ODataSerializerContext)
  Microsoft.AspNetCore.OData!Microsoft.AspNetCore.OData.Formatter.ODataOutputFormatterHelper+<WriteToStreamAsync>d__1.MoveNext()
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start(!!0&)
  Microsoft.AspNetCore.OData!Microsoft.AspNetCore.OData.Formatter.ODataOutputFormatter.WriteResponseBodyAsync(class Microsoft.AspNetCore.Mvc.Formatters.OutputFormatterWriteContext,class System.Text.Encoding)
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Formatters.TextOutputFormatter.WriteAsync(class Microsoft.AspNetCore.Mvc.Formatters.OutputFormatterWriteContext)
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Infrastructure.ObjectResultExecutor.ExecuteAsync(class Microsoft.AspNetCore.Mvc.ActionContext,class Microsoft.AspNetCore.Mvc.ObjectResult)
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext(value class State&,value class Scope&,class System.Object&,bool&)
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResultFilterAsync()
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResultFilterAwaitedAsync()
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Filters.ActionFilterAttribute+<OnResultExecutionAsync>d__9.MoveNext()
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start(!!0&)
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Filters.ActionFilterAttribute.OnResultExecutionAsync(class Microsoft.AspNetCore.Mvc.Filters.ResultExecutingContext,class Microsoft.AspNetCore.Mvc.Filters.ResultExecutionDelegate)
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext(value class State&,value class Scope&,class System.Object&,bool&)
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(value class State&,value class Scope&,class System.Object&,bool&)
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeAsync()
  Microsoft.AspNetCore.Mvc.Core.il!Microsoft.AspNetCore.Mvc.Routing.ControllerRequestDelegateFactory+<>c__DisplayClass10_0.<CreateRequestDelegate>b__0(class Microsoft.AspNetCore.Http.HttpContext)
  Microsoft.AspNetCore.Routing.il!Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Microsoft.AspNetCore.StaticFiles.il!Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Microsoft.AspNetCore.StaticFiles.il!Microsoft.AspNetCore.StaticFiles.DefaultFilesMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Microsoft.AspNetCore.Authorization.Policy.il!Microsoft.AspNetCore.Authorization.AuthorizationMiddleware+<Invoke>d__6.MoveNext()
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start(!!0&)
  Microsoft.AspNetCore.Authorization.Policy.il!Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Microsoft.AspNetCore.Authentication.il!Microsoft.AspNetCore.Authentication.AuthenticationMiddleware+<Invoke>d__6.MoveNext()
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start(!!0&)
  Microsoft.AspNetCore.Authentication.il!Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Microsoft.AspNetCore.Cors.il!Microsoft.AspNetCore.Cors.Infrastructure.CorsMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext,class Microsoft.AspNetCore.Cors.Infrastructure.ICorsPolicyProvider)
  Microsoft.AspNetCore.StaticFiles.il!Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Swashbuckle.AspNetCore.SwaggerUI!Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware+<Invoke>d__5.MoveNext()
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start(!!0&)
  Swashbuckle.AspNetCore.SwaggerUI!Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Swashbuckle.AspNetCore.Swagger!Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware+<Invoke>d__4.MoveNext()
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start(!!0&)
  Swashbuckle.AspNetCore.Swagger!Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext,class Swashbuckle.AspNetCore.Swagger.ISwaggerProvider)
  Microsoft.AspNetCore.StaticFiles.il!Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Microsoft.AspNetCore.HttpsPolicy.il!Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Serilog.AspNetCore!Serilog.AspNetCore.RequestLoggingMiddleware+<Invoke>d__9.MoveNext()
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start(!!0&)
  Serilog.AspNetCore!Serilog.AspNetCore.RequestLoggingMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Microsoft.AspNetCore.Diagnostics.il!Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware+<Invoke>d__9.MoveNext()
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start(!!0&)
  Microsoft.AspNetCore.Diagnostics.il!Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Microsoft.AspNetCore.HostFiltering.il!Microsoft.AspNetCore.HostFiltering.HostFilteringMiddleware.Invoke(class Microsoft.AspNetCore.Http.HttpContext)
  Microsoft.AspNetCore.Server.Kestrel.Core.il!Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequests>d__223`1[System.__Canon].MoveNext()
  System.Private.CoreLib.il!System.Threading.ExecutionContext.RunInternal(class System.Threading.ExecutionContext,class System.Threading.ContextCallback,class System.Object)
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[System.Threading.Tasks.VoidTaskResult,Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol+<ProcessRequests>d__223`1[System.__Canon]].MoveNext(class System.Threading.Thread)
  System.Private.CoreLib.il!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(class System.Runtime.CompilerServices.IAsyncStateMachineBox,bool)
  System.Private.CoreLib.il!System.Threading.Tasks.Task.RunContinuations(class System.Object)
  System.IO.Pipelines.il!System.IO.Pipelines.StreamPipeReader+<<ReadAsync>g__Core|36_0>d.MoveNext()
  System.Private.CoreLib.il!System.Threading.ExecutionContext.RunInternal(class System.Threading.ExecutionContext,class System.Threading.ContextCallback,class System.Object)
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[System.IO.Pipelines.ReadResult,System.IO.Pipelines.StreamPipeReader+<<ReadAsync>g__Core|36_0>d].MoveNext(class System.Threading.Thread)
  System.Private.CoreLib.il!System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(class System.Runtime.CompilerServices.IAsyncStateMachineBox,bool)
  System.Private.CoreLib.il!System.Threading.Tasks.Task.RunContinuations(class System.Object)
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[System.Int32].SetExistingTaskResult(class System.Threading.Tasks.Task`1<!0>,!0)
  System.Net.Security.il!System.Net.Security.SslStream+<ReadAsyncInternal>d__188`1[System.Net.Security.AsyncReadWriteAdapter].MoveNext()
  System.Private.CoreLib.il!System.Threading.ExecutionContext.RunInternal(class System.Threading.ExecutionContext,class System.Threading.ContextCallback,class System.Object)
  System.Private.CoreLib.il!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1+AsyncStateMachineBox`1[System.Int32,System.Net.Security.SslStream+<ReadAsyncInternal>d__188`1[System.Net.Security.AsyncReadWriteAdapter]].MoveNext(class System.Threading.Thread)  Microsoft.AspNetCore.Server.Kestrel.Core.il!Microsoft.AspNetCore.Server.Kestrel.Core.Internal.DuplexPipeStream+<ReadAsyncInternal>d__27.MoveNext()
  System.Private.CoreLib.il!System.Threading.ExecutionContext.RunInternal(class System.Threading.ExecutionContext,class System.Threading.ContextCallback,class System.Object)
  System.Private.CoreLib.il!System.Threading.ThreadPoolWorkQueue.Dispatch()
  System.Private.CoreLib.il!System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()
  System.Private.CoreLib.il!System.Threading.Thread.StartCallback()

Provider and version information

EF Core version:
Database provider: (e.g. Microsoft.EntityFrameworkCore.SqlServer)
Target framework: (e.g. .NET 6.0)
Operating system: Windows 11

[roji]

Moving to SqlClient as this doesn't seem related to EF Core in any way.

[JRahnama]

Hi @alvanxp. As far as we know there is no thread starvation on Windows and native SNI combination, unless you are using managed SNI on windows which will cause issue #422. Can you post a sample repro plus the used connection staring's properties that we can start the investigation at our end please?

[alvanxp]

Hi @JRahnama ,this issue is related to #422 , I changed it to false and now is working fine, thank you so much