-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
FromSql: Support multiple resultsets #8127
Comments
I think this question is more about retrieving multiple result sets from a sproc, and less so about MVC view models. |
@Eilon it looks there was an issue raised in the past #6026 but it was closed for some reason. Is there any current plan to include this feature in the 2.0 milestone. |
@RickyG-Akl , the friendly folks in this repo will give you an answer 😄 |
@rpundlik @RickyG-Akl There isn't currently any support for this when using EF and FromSql. The workaround is to drop down to ADO.NET and use the DbConnection directly to run the query and handle the multiple resultsets. Putting this issue on the backlog so that we can consider adding support in the future. |
@ajcvickers In EF6 we have ObjectContext.Translate api, which we could use in this kind of scenario. There it was possible to materialize SqlDataReader into the EF entity. However this is missing in EF Core 2.0. The issue has been raised in #4675. We use this feature extensively and without this we won't be able to migrate from EF6 to EF Core. |
@ppN2, I am curious about how you are using |
Generally when the entities are simple for example order, orderdetails etc kind, we don't need to use Translate. The include() method works just fine. There are many cases where we need to return a complex object graph from the stored procedure. Generally this stored procedure will have complex business rules and will return multiple resultsets (in our case about 15-20 resultsets). In these cases we run the ExecuteReader command and load SqlDataReader and using translate apis to dematerialize the entities one by one. |
@ajcvickers, @divega By looking at past responses and the source code, we came up with the following procedure which is equivalent to the EF6 ObjectContext.Translate (We could extend it to include the merge options, similar to EF6 Translate provides). We have tested this code and seems to be working. We do not have deep knowledge of he source code, so our confidence level is not very high. Is it possible for somebody with deeper knowledge to review this code? This will really help us to move forward with the migration to EF Core 2.0. public static List<T> Translate<T>(this DbSet<T> set, DbDataReader reader) where T : class
{
var entityList = new List<T>();
if (reader == null || reader.HasRows == false) return entityList;
var entityType = set.GetService<IModel>().FindEntityType(typeof(T));
var valueBufferParameter = Expression.Parameter(typeof(ValueBuffer));
var entityMaterializerSource = set.GetService<IEntityMaterializerSource>();
var valueBufferFactory = set.GetService<IRelationalValueBufferFactoryFactory>().Create(new[] { typeof(T) }, null);
var stateManager = set.GetService<IStateManager>() as StateManager;
Func<ValueBuffer, T> materializer = Expression.Lambda<Func<ValueBuffer, T>>(
entityMaterializerSource.CreateMaterializeExpression(entityType, valueBufferParameter), valueBufferParameter)
.Compile();
stateManager.BeginTrackingQuery();
while (reader.Read())
{
ValueBuffer valueBuffer = valueBufferFactory.Create(reader);
var entity = materializer.Invoke(valueBuffer);
var entry = stateManager.StartTrackingFromQuery(entityType, entity, valueBuffer, null);
entityList.Add((T)entry.Entity);
}
return entityList;
} The way to use this code will be using (var ctx = new ApplicationDbContext())
{
using (var cnn = ctx.Database.GetDbConnection())
{
var cmm = cnn.CreateCommand();
cmm.CommandType = System.Data.CommandType.Text;
cmm.CommandText = "SELECT AccountId, AccountBalance, AccountName FROM Accounts; SELECT CustomerId, AccountId, CustomerName FROM Customers";
cmm.Connection = cnn;
cnn.Open();
using (var reader = cmm.ExecuteReader())
{
var accounts = ctx.Accounts.Translate(reader);
reader.NextResult();
var customers = ctx.Customers.Translate(reader);
}
}
} |
@sjh37 FYI! |
Multiple result sets from a sproc are supported via the Entity Framework Reverse Poco Generator. This generates code for EF6, which you could copy and put into your EF.Core project. |
@sjh37 Notice that ObjectContext is not present in EF Core |
@ppN2 We looked through your code in triage and we think it should be okay for simple cases. Some things to keep in mind:
Overall, if you are in full control of the types, the queries, and the state of the context, then this should work, but as a general solution it will fall down in some places. |
@sjh37 . On your comment that we can use the multiple resultsets from stored proceudure in an EF core project, did you mean that as part of that hack :
|
@srini1978 I thought it would of been a simple task to generate the code using the generator, then copy n paste the stored proc and result set class to your ef core db context. However I didn't realise at the time that ObjectContext is not present in EF Core. |
Is there any update on this? can I use EF Core and FromSql to get multiple result set using stored procedure in model? |
@anildbest83 This issue is in the Backlog milestone. We haven't done any work on it yet and that it is currently not planned for for the 2.2 or 3.0 release. We will re-assess the backlog following the 3.0 release and consider this item at that time. However, keep in mind that there are many other high priority features with which it will be competing for resources. |
This is a big deal for us because we have very complex load SPs that do a lot of calculating/coalescing to figure out what records need to be included. This work can only be done once and we can return from 10-30 tables in one load SP. I attempted the code written by @ppN2 above but it did not work in the latest EF version (not overly surprised considering the Internals warning). I attempted to take a different approach to load and I was able to get it to work by simply taking data (ADO.NET) from the database and manually creating records and then attaching them to the context. This worked fine and would potentially work for us since we code gen a lot so we would know what tables are where with what columns so code gening this load logic wouldn't be terrible even though it is not ideal as EF should have this capability built in. I have no idea what the performance cost is for this as it wires up all relationship on attach. I do batch AttachRange hoping that would be better but I don't know. Once the context was loaded all entity access would then be done using the Local cache only. Obviously this is not a generic solution that will work for any entity type automatically but at least on the surface seems like a possibility. Wanted to share to hopfully up priority on this issue as well as get any feedback on this approach. |
Somehow I managed to tackle my situation using the code written by @ppN2, thanks |
@anildbest83 What did you do to get it to work? Can you post a snippet? |
Something like this Used SqlHerper.cs class (attached) and cosnume it like this
|
I was wondering if anyone had managed to wrangle a decent workaround for this in EF Core 2.1. I've been attempting to dig into the internals of EF Core to figure out how to materialize and track an entity using a As @spudcud pointed out, it looks like the workaround provided by @ppN2 no longer works for EF Core 2.1. I also attempted to hook up a workaround from #4675 by @jnm2, but see an identical problem there, too. The crux of it is that I receive an exception I guess two real questions holding me up are:
We have a rather large solution that I am working to convert, and this is one of the last hurdles preventing me from completely migrating to EF Core. Any guidance is greatly appreciated. Thanks! |
Crickets chirping... Same here, @mscappini. It never occurred to me that a mature product like EF Core 2.x would not have any means by which to return a stored procedure returned recordset back to the caller. Moving on to ADO.NET, I guess, per @ajcvickers's guidance. BTW, why is this issue titled "Support multiple resultsets"? Best I can tell, EF doesn't even really support single resultsets. (I am disregarding solutions such as this that look painful and awkward.) I very much look forward to the version of EF that not only supports stored procedures, but can import them into projects automatically, as edmx did in the past. Until then, I guess I either live with ancient versions of EF or use some other tool, because I use stored procedures extensively. Am I that unusual? |
@WellspringCS This issue is titled "Support multiple resultsets" because that's what it's about. Returning single result sets is supported for query types and entity types.. |
In EFCore 5 the I hope that this missing feature will be implemented before the index parameter finally disappears from the API. |
I'm having some trouble getting this solution to work. It looks like the materializerAction isn't getting assigned a delegate. But I don't think I understand this code enough to properly troubleshoot. instead, I was able to get something working like this: `using (connection)
|
It works with this tiny change on EF core 5 according to @mwgolden update and as sql parameter value for this extension method type your stored procedure name "dbo.testproc" works for me |
This is not working for me. When I call my SP, it is conditional in what SELECT statements returns. Additionally, there are Instead of Insert Triggers returning from other tables being inserted to. I cannot capture the Errors (return when SP raises error) or Ids (SP success.) All I am getting back is the 2 Trigger returns then nothing. Any ideas? What details can I share? Pseudo code (SP)
Pseudo code (Triggers)
Code calling the MultiResultSetsFromSql
|
@mwgolden @BladeWise @ZarDBA @spasarto TypeMaterializationInfo does not take a RelationalTypeMappingSource as parameter anymore, but a RelationalTypeMapping and I can't seem to get one of those. The update from @ZarDBA takes me a bit further, but not to a working solution. Maybe I should try rewriting like @mwgolden did? update: |
Based on what @mwgolden suggested, here is a working solution for dotnet core 6: using System;
using System.Collections;
using System.Collections.Generic;
using System.Data;
using System.Data.Common;
using System.Linq;
using System.Threading.Tasks;
using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Infrastructure;
using Microsoft.EntityFrameworkCore.Storage;
public static class DbContextExtensions
{
public static async Task<IList<IList>> QueryStoredProcedureWithMultipleResults(
this DbContext dbContext,
List<Type> resultSetMappingTypes,
string storedProcedureName,
params object[] parameters
)
{
var resultSets = new List<IList>();
var connection = dbContext.Database.GetDbConnection();
var parameterGenerator = dbContext.GetService<IParameterNameGeneratorFactory>()
.Create();
var commandBuilder = dbContext.GetService<IRelationalCommandBuilderFactory>()
.Create();
foreach (var parameter in parameters)
{
var generatedName = parameterGenerator.GenerateNext();
if (parameter is DbParameter dbParameter)
commandBuilder.AddRawParameter(generatedName, dbParameter);
else
commandBuilder.AddParameter(generatedName, generatedName);
}
await using var command = connection.CreateCommand();
command.CommandType = CommandType.StoredProcedure;
command.CommandText = storedProcedureName;
command.Connection = connection;
for (var i = 0; i < commandBuilder.Parameters.Count; i++)
{
var relationalParameter = commandBuilder.Parameters[i];
relationalParameter.AddDbParameter(command, parameters[i]);
}
if (connection.State == ConnectionState.Closed)
await connection.OpenAsync();
await using var reader = await command.ExecuteReaderAsync();
int resultIndex = 0;
do
{
Type type = resultSetMappingTypes[resultIndex];
var resultSetValues = (IList) Activator.CreateInstance(typeof(List<>).MakeGenericType(type));
var columns = Enumerable.Range(0, reader.FieldCount).Select(reader.GetName).ToList();
while (reader.Read())
{
var obj = Activator.CreateInstance(type);
if (obj == null)
{
throw new Exception($"Cannot create object from type '{type}'");
}
foreach (var column in columns)
{
var value = reader[column] == DBNull.Value ? null : reader[column];
obj!.GetType().GetProperty(column)?.SetValue(obj, value);
}
resultSetValues!.Add(obj);
}
resultSets.Add(resultSetValues);
resultIndex++;
} while (reader.NextResult());
return resultSets;
}
public static async Task<(IReadOnlyCollection<T1> FirstResultSet, IReadOnlyCollection<T2> SecondResultSet)>
QueryStoredProcedureWithMultipleResults<T1, T2>(
this DbContext dbContext,
string storedProcedureName,
params object[] parameters
)
{
List<Type> resultSetMappingTypes = new List<Type>() {typeof(T1), typeof(T2)};
var resultSets =
await QueryStoredProcedureWithMultipleResults(dbContext, resultSetMappingTypes, storedProcedureName,
parameters);
return ((IReadOnlyCollection<T1>) resultSets[0], (IReadOnlyCollection<T2>) resultSets[1]);
}
public static async
Task<(IReadOnlyCollection<T1> FirstResultSet, IReadOnlyCollection<T2> SecondResultSet,
IReadOnlyCollection<T3>
ThirdResultSet)> QueryStoredProcedureWithMultipleResults<T1, T2, T3>(
this DbContext dbContext,
string storedProcedureName,
params object[] parameters
)
{
List<Type> resultSetMappingTypes = new List<Type>() {typeof(T1), typeof(T2), typeof(T3)};
var resultSets =
await QueryStoredProcedureWithMultipleResults(dbContext, resultSetMappingTypes, storedProcedureName,
parameters);
return ((IReadOnlyCollection<T1>) resultSets[0], (IReadOnlyCollection<T2>) resultSets[1],
(IReadOnlyCollection<T3>) resultSets[2]);
}
} Usage: var (result1, result2, result3) = await dbContext.QueryStoredProcedureWithMultipleResults<Model1, Model2, Model3>("StoredProcedureName"); |
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
This comment was marked as resolved.
how to replace IObjectContextAdapter and replace them with IInfrastructure i am migrating the code from entity framework 6 to entity framework core 6 but i am getting error for IObjectContextAdapter for the below code please suggest - db.Database.Connection.Open();
|
In Entity Framework Core 6, the IObjectContextAdapter interface is no longer available, and instead, the IInfrastructure interface is used. To replace IObjectContextAdapter with IInfrastructure, you can modify your code as follows:
In the above code, GetService() is used to get an instance of the IStateManager interface, which is used to manage the state of entities in Entity Framework Core. Then RegisterStreamedResults is called to register the streamed results with the state manager and the ToList method is called to materialize the results. Note that RegisterStreamedResults takes the entity type Blog as its generic type argument and the name of the result set as its second argument. Also, you need to use the GetDbConnection method to get the underlying DbConnection object and call its Open method to open the connection before executing the command. |
i am getting error at RegisterStreamedResults method |
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
This comment was marked as spam.
Thanks @mwgolden and @ArnaudValensi for this solution. Just adding a small comment, if ever you have nullable enums in your models, you will get an exception, and the following code will help to cast and set the value properly in that case :
|
Another variation that allows getting paginated data and total count using a single query:
With the above method I can query database like this:
The extensions method uses data annotation on the model, so database columns are matched to the suitable properties. The missing part is the usage of converters declared per property and globally. |
The solutions above seem like a lot of extra work to me, and limits things to 1, 2 or 3 result sets. Instead, I first created a simplified extension method (based upon what I saw above, except without the asynchronous logic):
and then called it in a foreach loop for each result set (4 results in my case):
Usage:
It works for me in Entity Framework Core 7.0.11 and is very similar to the ((IObjectContextAdapter)dbContext).ObjectContext.Translate approach from Entity Framework 6.2 (and the .Net Framework). |
@dlwennblom This seems like a more elegant solution. I will maybe change to this one in the future |
@dlwennblom thank you for your version. Indeed, it looks cleaner. |
Is a very slow Solution. Also it would trigger all Property Setters, including prop changed and other stuff. An official Solution would be nice. The advantage of |
From @rpundlik on March 22, 2017 12:9
While retrieving the results using stored procedure how can I retrieve and store multiple result set in view model in .net core
For e.g. from stored procedure I am returning records for below queries
and below is view model
This is how I am executing the stored procedure
How can stored the multiple result set using stored procedure in view model ?
Right now I need to call the three separate stored procedures to get data and merging them into one view model
Thanks for the help !
Copied from original issue: aspnet/Mvc#6011
The text was updated successfully, but these errors were encountered: