Update model from database

[rowanmiller]

We'll add a new comand (e.g. Update-DbContext and dotnet ef dbcontext update) to re-scaffold your DbContext and entity types to incorporate any changes you've been made to the database schema. This will allow you to update your model to be compatible with the database but preserve any customizations you've made (e.g. navigation property names) to the originally scaffolded code.

Scope

The folowing types of customizations will be preserved.

• Entity type names
• Entity type property names (including navigation properties)
• Entity type inheritance
• DbSet property names

The following are not a priority during the initial implementation.

• Non-EF annotations on entity type properties (could be enabled via [MetadataType])
• Configuration that deviates from the database (database always wins)
  • Additional constraints added to the model

Files will be blindly overwritten during this process. To allow additional customizations to the model, we'll continue generating partial classes and reccommend adding custom members to a new file with another partial definition of the class.

Implementation

We'll leverage the existing Reverse Engineering assets for this work. Specifically:

• IDatabaseModelFactory--Reads the database schema.
• IScaffoldingModelFactory--Transforms the schema into a compatible IModel.
• IScaffoldingCodeGenerator--Transforms the IModel into code.

We'll read the current database schema, transform it into a model and compare it to the current model. Anything that has changed will be merged with the current model and the resulting model will be scaffolded into code. Existing files will be overwritten and removed types will be deleted.

During the inital reverse engineer, we scaffold an OnConfiguring() stub. We'll need to move this into it's own file so we can generate it once, but not during update.

TODO

• Refactor generated code to be update friendly (consider RevEng: Enable generating IEntityTypeConfiguration classes (avoids large OnModelCreating) #8434 & SqlServer RevEng: Scaffold default schema #9318)
• Create a scaffolding model differ
• Update IScaffoldingCodeGenerator to handle new concepts (e.g. inheritance)

[bricelam]

In theory, this isn't too different than Migrations: Diff the current model with the reverse engineered model from database and generate operations to perform on the Roslyn syntax tree.

[rowanmiller]

Yeah, I think we'd want to keep a snapshot when the model was reverse engineered, so that we can work out what changed without loosing changes the developer has manually made to the model. Probably ok if we overwrite changes on something that also changed in the database, but not things that remained the same. But, as you said, all those pieces are there from migrations.

[RandyBuchholz]

One of the things I encounter often when working on the early stages of a project and scaffolding from PM> is not being able to update when I can't build. I may have changed my business data and database to match, but now my project is full of errors. In some cases, running "update-model" is the solution to removing the errors, but I can't because of the errors. I'm not sure why scaffold-dbcontext requires code to compile since it is only creating new objects and has AFAIK, no dependencies on the target project business objects. When I work DB-first I have a rule to never touch the EF code. I don't want to go in and fix the all the errors just so I can overwrite what I just did. I usually end up having a separate project I scaffold into and cut and paste. But then I have to go back and adjust all of the namespaces.

It would be nice to be able to scaffold when there are errors, and to have drop-recreate capability. Sometimes the best "update" is starting over. For example scaffold-update-model -o Models -witherrors -recreate would delete everything in the Models folder, invoke scaffolding even if there are errors in my code, and then do a clean construction into /Models/.

It would also be nice to have a -namespace switch.

[bricelam]

...why scaffold-dbcontext requires code to compile...

There are a couple reasons we do this:

• To pick up any design-time services that have been overridden in the project (e.g. customizing the code generation)
• If the provider's design-time package was just installed, we need to build to ensure it's copied into the output directory.

It is also just done in general for all commands since every other command besides Scaffold-DbContext needs to operate on your project's assembly.

In theory, thouth, it is optional for Scaffold-DbContext, and we could consider using the -Force switch to continue anyway (with a warning).

[bricelam]

It would also be nice to have a -namespace switch.

I think the -OutputDir parameter can be used to configure the namespace and directory of the model. If so, it looks like we should update the documentation.

[RandyBuchholz]

Thanks for the explanation.

Skipping compile on -force with a warning would work great.

-o -OutputDir does add the namespace of the folder you gen to. I may be the rarer case, but I don't keep a strict 1:1: folder to namespace mapping. I often use container folders to subdivide or group things, and skip them in the namespace. Generally I'll do this by architecture areas. I group my database related folders under DB/ to keep them together in the explorer, but skip it in my namespace. So my models are in the DB/Models folder, but in the Models namespace. It would be cool to be able to get that with -o DB/Models -namespace Models.

I put a feature request in to VS a while ago for a "Use Parent Namespace" bool property on folders. Anytime you right click and add a file, it skips those folders when it creates the namespace.

[marchy]

Just to throw a very critical scenario into the mix, some of us that started out our codebases on EF Core 1.0, then had to downgrade to EF6 due to the slew of limitations along the development path for it (ie: pulling full database tables in memory so the framework can perform operations that were not yet implemented --> how was this ever an idea that gained any traction? This is so unfeasible for any web server I don't even know where to begin.... but I digress) - we've now had migrations diverge as we have evolved our models in EF6.

At some point, we will want to go back to EFCore and it means we have:

• EFCore1 migration
• EFCore1 migration
• EFCore1 migration
  ....
• EF6 migration
• EF6 migration
• EF6 migration
  ....
• EFCore2 migration (future)

There will likely need to be a big catch-up migration on the EFCore side to capture the deltas between the last EFCore migration and all the migrations that happened in EF6 which are not captured in the EFCore model. Being able to reverse-engineer model updates from the database could do loads to make this pain anything less than excruciating --> though I'm not entirely certain how that would work: would it add/change properties to the POCO classes? What about the DBContext mappings? I'm afraid for any changes in indexes/specifications that might not get caught up (or might differ from what EF6 performed).

Any other approaches you guys can think of for dealing with this situation?

[bricelam]

(Updated the description with a draft "spec" based on our previous design meetings.)

[GunboatDiplomat]

Any updates on this feature? Is it marked for any near future release?

[ajcvickers]

@GunboatDiplomat This issue is in the Backlog milestone. This means that it is not going to happen for the 3.0 release. We will re-assess the backlog following the 3.0 release and consider this item at that time. However, keep in mind that there are many other high priority features with which it will be competing for resources.

[shyamal890]

Would this be released in EF Core 5.0?

[ajcvickers]

@shyamal890 This issue is in the Backlog milestone. This means that it is not planned for the next release (EF Core 5.0). We will re-assess the backlog following the this release and consider this item at that time. However, keep in mind that there are many other high priority features with which it will be competing for resources.

[cgountanis]

It would be cool if it also add/update, would check for entities that do not exist any more on the DB side. I find when things get renamed or removed, scaffold leaves orphaned files in the models folder. Maybe just a message that says these files were found but did not have a db-first match or something?

[miloshavlicek]

More than 5 years on backlog. So sad.

[miloshavlicek]

Is anyone considering implementing it in the near future or should I implement it on my own?

[ajcvickers]

@miloshavlicek It's not currently scheduled. A robust implementation of this is non-trivial, but if you do come up with a good solution then you might consider contributing. We would certainly be interested in seeing whatever you come up with.

[ErikEJ]

@miloshavlicek Have you had a look at EF Core Power Tools?

[simeyla]

...why scaffold-dbcontext requires code to compile...

In theory, though, it is optional for Scaffold-DbContext, and we could consider using the -Force switch to continue anyway (with a warning).

Probably my most popular answer ever on Stackoverflow is for a question with nearly 100k views “Build failed” on Database First Scaffold-DbContext so I can speak with some authority (and constant reminder notifications!) that people really hate this behavior!

If you ever decide to change this behavior please answer this question with the new switch :-) Not that I don't mind all the points.

[simeyla]

Here's the thing about both T4 templates and EF Core Power Tools - I simply don't have time to learn them (and learn to trust them) when literally the only change I want to make is to comment out the default constructor and the OnConfiguring method. Not that I don't appreciate the community work, but it's honestly crazy that it's so hard to do that.

Since these are the ONLY THINGS I want to do I feel far more comfortable just removing them by hand and I have been doing exactly this, hundreds of times - including at least two times when I checked in the default constructor by mistake and broke everything.

But no more!

Presenting some Regexes and a Powershell script to do it for me:

fix-dbcontext.ps1

$filename=$args[0]

# load context file
$content = (Get-Content -Raw $filename) 

[regex] $commentOutConstructorRegex = '(?ms)(?<=: DbContext\s*.*?)(public.*?)(?=\s*public)'
$content = $commentOutConstructorRegex.Replace($content, '// Default constructor removed', 1)

[regex] $removeOnConfiguringRegex = '(?ms)(protected override void OnConfiguring).*?(?=\s*protected)'
$content = $removeOnConfiguringRegex.Replace($content, '// Generated OnConfiguring removed', 1)

[regex] $dateCommentRegex = '(?ms)(?=\s*public partial class)'
$content = $dateCommentRegex.Replace($content, "`r`n`t// Generated " + (Get-Date).ToString() + "`r`n", 1)

$content | Out-File -Encoding UTF8 $filename

This will:

1. Remove the default constructor
2. Remove the OnConfiguring method including hardcoded connection string
3. Add the date in a comment

Just run it with .\fix-dbcontext.ps1 .\MyDBContext.cs and remove parts you don't want.

From a second script I run the ef scaffold command and then my script. Here's the result:

I'd recommend changing $filename at the end to test.txt until you're sure it works.

[janseris]

Hello?