Add the ability to run migrations on DbContexts in referenced assemblies

[glen-84]

As far as I know, this is not currently possible. The use case is to create a modular system, where modules (in separate assemblies) each contribute one or more DbContexts to the main application. When you run migration add on the main project (possibly with the --all flag), it will include DbContexts in referenced assemblies.

I don't know if this would scan assemblies or use the DI container (in ASP, at least). Perhaps both. The latter would provide access to connection strings configured in that way.

Related questions:

1. Is this an appropriate way to supply the main application with additional entities, while still having a DbContext local to the module to use in services/repositories specific to that module? It will mean that there will be quite a few DbContexts (zero or more per module), but I don't think that this could affect performance in any way, since they should be lazy-loaded via DI.
2. I haven't yet looked into this, but there will be times when I would need something like an IdentityUser in the module's DbContext. For example, if it's a forum module, it might have a Topic entity with a relationship to an ApplicationUser/IdentityUser. Is it possible to add a DbSet to a DbContext using an interface maybe, where the concrete type is later discovered? Like public DbSet<IIdentityUser> User { get; set; } (that interface doesn't actually exist, it's just an example). The same question would apply to adding the relationship to the entity itself.

[rowanmiller]

We have an item tracking the higher level work for this. Leaving this one open to track this particular scenario though.

[glen-84]

@rowanmiller,

Is the concept described in question 2 above theoretically possible? i.e. Creating a relationship to an entity by interface only (no concrete type immediately available).

PS. If the work item that you refer to is here on GitHub issues, please share the issue number.

[rowanmiller]

Hey,

Not really, here is the work item tracking better interface support - #757. It's not something we are tacking right now, but we do want to add it in the future.

~Rowan

[bricelam]

You should always invoke the commands on the assembly that you want the migrations to live in. If you want this to be a different assembly than the one containing your DbContext, scaffold the first migration in the same assembly as your DbContext then move it into the desired project. You should then be able to add additional migrations to the new project.

Once #2219 (and possibly the higher-level item #639) is fixed, your services, DbContext, and Migrations can all live in different projects. You would invoke commands similar to the following.

cd MyMigrationsProject
dnx . ef migration add MyMigration --startupProjectName MyWebProject

MyMigrationsProject would reference MyDbContextProject.

[glen-84]

@rowanmiller, Thanks, I will reply there.

@bricelam,

One may not have control of the project containing the DbContext. Imagine a "forum" module that you install with NuGet. You can't run migrations there, because you don't have the source, and regardless, it won't produce the desired result, which is a combined migration in the application that included the module.

The second option that you mentioned will not work either.

What is required is the following:

1. I create a web application.
2. I install a "module" (forum/blog/etc.) as a NuGet/DNX dependency.
3. I add the DbContext from the module with services.AddEntityFramework().AddDbContext<TheModule.ForumDbContext)>() or by passing the services container to the module during Startup and letting it add its own context (this is the way that I had planned to do it).
4. I run dnx . ef migration add MyMigration --all from the web application, and it creates a migration that includes all DbContexts (either by looking at contexts added via the services container, or by doing "assembly scanning")

This (along with #757) would be very powerful, and would help in enabling the creation of modular applications. Please keep it in mind when working on related issues.

[bricelam]

Yes, interesting scenario to consider.

I wonder if the package could ship a bootstrap Migration as source that get's copied into the target project.

Another hack would be to create a derived DbContext in the project you want to put migrations in.

Otherwise, yeah, we should probably let you specify an assembly-qualified context type. The command would then look something like dnx . ef migration add InitialCreate --context "TheModule.ForumDbContext, TheModule" (I haven't tested it, but that might already work)

[bricelam]

Of course, if the consumer doesn't derive from the module's DbContext, does that mean they can't extend the model? If so, there's no reason for the user to create migrations for it. The migrations should just ship as part of module and they could be applied as part of module initialization or using a module-specific API/tool.

[davidroth]

@glen-84 I am glad you brought this up as this is a scenario which is very interesting for me as well.

The scenario I am looking for is a modular application, where users can install/uninstall additional modules at any point in time.

The module would define it`s model in its own DbContext (a "comment" or "blog" module for example), but the model would depend on the "core" module of the application host. (For example references to user entities or some core entities which are useful for all derived modules).

So one user might install the core product with modules "A" and "B" at the beginning. Another user might just install the core product, and install module "B" a few months later.

The "install at any point in the part" is interesting, as this requires an additional "line up" between the migrations of the core application and each module.

For example if the host application is installed and has had several migrations (ex. is now installed in version "3").
Now an additional module gets its installed. This module must now provide an migration/installation path to align with the core application in version "3". It might not be possible to install the module by applying its own migrations from "1" to "3", as the migration step "1" or "2" might not be compatible anymore with the current database in version "3" of the host application.
An example for that would be, if the additional module in version "2" required a foreign key to a table in the host application in version "2", which does not exist anymore or has been renamed in version "3".

If the module would have been installed from the beginning, its migration step "3" could handle this deprecated/renamed table and could update/align its own foreign keys to align with version "3" in the core app. But since it gets installed in version "3" it needs to create its tables directly for version "3" of the host application.

Maybe ef could allow you to specify multiple migration paths which could then be invoked based on a parameter? Just thinking out loud, I would be very interested about your thoughts on this topic ;)

[glen-84]

I wonder if the package could ship a bootstrap Migration as source that get's copied into the target project.

What would happen when the package is updated then? Also, how would you run all of these migrations at one time. Another thing is that modules may reference "unresolved" entities (like an IUser [see #757]), so it won't be possible to create a migration in the module itself, the host application will need to resolve the interfaces first.

Another hack would be to create a derived DbContext in the project you want to put migrations in.

I assume this is just to move it into the same assembly. It's a hack yes, because if you're using 17 modules, you'd have to do this 17 times (or use some other type of magic).

Otherwise, yeah, we should probably let you specify an assembly-qualified context type. The command would then look something like ...

This would be a step forward, however it would really be nice if you didn't have to manually run N migrations, one for each context. Also, this would not work with the --all flag, I don't think.

Of course, if the consumer doesn't derive from the module's DbContext, does that mean they can't extend the model?

I can't think of specific reasons why you would want to extend the model (unless this would be another way of introducing the User relation from the host application – i.e. extend Forum.Topic add ApplicationUser relation [but this is too "manual" IMHO]), but I think that the option should still be available.

If so, there's no reason for the user to create migrations for it.

See my points above (first paragraph). The module never runs by itself, it is always part of a "host" application, and the host application controls migrations across all modules. If I add a forum module, and update a blog module, I get a single migration with commands to add a Topic table and alter a BlogPost table.

As far as I know, I'm able to add a DbContext from another assembly using services.AddEntityFramework().AddDbContext<My.ForumDbContext>(), but the context is not listed when you run dnx . ef context list – is it possible to make the migration tools see all DbContexts added in this manner (i.e. via the EntityFrameworkServicesBuilder)?

I misunderstood #1813, thinking that it would allow you to create migrations across multiple DbContexts, not just apply them. Do we need a separate issue for this, or would it fall under this one?

I don't really understand how migration classes are associated with migrations. I assume it requires a --context if you have more than one context? If so, that's not really going to work well if you have multiple contexts and wish to apply migrations for each. Perhaps the DbContext should somehow be referenced within the migration class?

[glen-84]

The scenario I am looking for is a modular application, where users can install/uninstall additional modules at any point in time.

Same.

The module would define it`s model in its own DbContext (a "comment" or "blog" module for example), but the model would depend on the "core" module of the application host.

I'd like to avoid this requirement with the use of interfaces (like IUser, see #757). If the interface is not available in something like the Identity system, it may be necessary to have a shared package with a few common types (as you mention).

The "install at any point in the part" is interesting, as this requires an additional "line up" between the migrations of the core application and each module.

The idea I have is to just have a single migration (or set of migrations) in the host application. The module will not contain any migrations. If you add a module and that module adds a new context, then running migration add AddingNewModuleAndSomethingElseToHost --all would create a migration that adds the new tables to the host database (and may also make other changes, if there are entity/mapping changes in the host application itself).

Trying to manage migrations for many modules individually would be complicated and time consuming, and may not always be possible (if interfaces are used).

[bricelam]

I don't really understand how migration classes are associated with migrations. I assume it requires a --context if you have more than one context? If so, that's not really going to work well if you have multiple contexts and wish to apply migrations for each.

Yes, each context has its own set of migrations. A different context requires a different set of migrations. There is no concept of sharing a migration class between multiple contexts.

To get around this, the "core" DbContext should be extensible by the modules (e.g. they have a hook into OnModelCreating). The bloging module might extend it like this:

modelBuilder.Entity<BlogPost>();

It could then get an instance of the DbContext and query using the Set method.

var recentPosts = db.Set<BlogPost>().OrderByDescending(p => p.PublishDate).Take(3);

If the DbContext was added to the application services, the user could simply add and apply a migration after installing/updating a module.

dnx . ef migration add InstalledBloggingModule
dnx . ef migration apply

[bricelam]

As far as I know, I'm able to add a DbContext from another assembly using services.AddEntityFramework().AddDbContext<My.ForumDbContext>(), but the context is not listed when you run dnx . ef context list – is it possible to make the migration tools see all DbContexts added in this manner (i.e. via the EntityFrameworkServicesBuilder)?

This is a bug. I've filed #2293.

I misunderstood #1813, thinking that it would allow you to create migrations across multiple DbContexts, not just apply them. Do we need a separate issue for this, or would it fall under this one?

Yes, this should be filed as a separate issue. This would, however, be a fundamental design change to Migrations.

[glen-84]

Yes, each context has its own set of migrations. A different context requires a different set of migrations. There is no concept of sharing a migration class between multiple contexts.

If having a single migration class for multiple contexts is too tricky (or impossible), then it should still be possible to use ef migration add Name --all, and have it generate a separate class per context. ef migration apply --all would then apply all migrations. This shouldn't be too difficult, right?

It does of course mean a large number of classes, one for each installed module, for what is "conceptually" a single migration. It would be awesome if the system could group the contexts by connection string, and create a single migration class per connection, but I don't know about how migrations are linked to contexts, etc. so this might not be possible.

To get around this, the "core" DbContext should be extensible by the modules (e.g. they have a hook into OnModelCreating).

I don't quite understand how this would work. If it means that the module needs a reference to the application, or that the application needs to do something funky with its main context, then it will not be ideal.

I wanted to iterate through each module and pass it the IServiceCollection (in ConfigureServices), but now I'm wondering if this would work, since it seems to require method chaining. If the application has services.AddEntityFramework().AddSqlServer().AddDbContext<ApplicationDbContext>(), then I'm not sure how to add additional contexts without first calling AddEntityFramework() again. How do I get the EntityFrameworkServicesBuilder from services after the initial chaining?

This is a bug. I've filed #2293.

Thanks. :-)

[ghiloufi]

DNX . ef Migration add Unable to load file or assembly Microsoft.Framework.Configuration.Json #814
i need help

[bricelam]

I have no idea what work needs to be done for this issue anymore. 😃 I suspect most of it will be covered by #2294. Moving to a discussion and closing. Please create new issues for anything that you think is not already tracked elsewhere.