-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow FKs to exist in the model but avoid creating them in the database #15854
Comments
RemoveForeignKey
does not usable.
@chrsas Can you give some details as to why you want to remove all foreign keys from the model? |
@ajcvickers The project I am working on was an upgrade of an existing system, which have more than 500 tables and lots of data in the database. The original model is dbfirst, and no foreign keys have been established. In addition to non-standard operations, many data in the system have been unable to establish foreign key constraints. The upgrade cycle is so short that we don't have enough time to clean up the data. Besides, some data are not easy to clean up. For example, product A has been deleted, but in order to prevent audit problems, all relevant sales records cannot be cleared. |
@chrsas "The original model is dbfirst, and no foreign keys have been established." Does this mean there are no foreign key constraints defined in the database, or that they are in the database but not in the EF model? |
There are no foreign key constraints defined in the database |
@chrsas Thanks for the additional information. So, if I can clarify, the database has no foreign key constraints, but when this was scaffolded into an EF Core model there still ended up being FKs in the model? Or did you create the EF Core model in some other way? |
@ajcvickers Thanks for your patience, Sorry for my poor English, My problem is that:
|
@chrsas For navigation properties to work EF needs there to be FKs in the model. This is because the relationship is represented by the FK, with the navigations acting as a kind of view over the FK. However, EF doesn't care if these FKs really exist in the database or not. So you can keep them in the model to allow navigation properties to work, but then avoid creating them in the database. There currently isn't any way to configure EF to not create FKs in the migration if they are in the model, but you can manually remove them from the migration, like you said above. So I think what you are doing is the best approach available right now. |
@ajcvickers Thanks for your help. |
See also #2725, but leaving this as a separate issue since FKs may require a different mechanism. |
We are in the same position of having a legacy database with EF Core retrofitted. The requirement for foreign keys removes the ability to use navigation properties with this data, which significantly reduces the benefit of using EF Core. It would be helpful to have a fluent API method to specify that no foreign keys are required for a given relationship, so that migrations never attempt to create them in the first place. |
@zejji I'm curious; if you have a legacy database without FK constraints, but you're using migrations to evolve the schema, then why not add FK constraints as you update the schema? |
@ajcvickers - it's a large, monolithic database with many years of data and we don't control the client deployments (i.e. there are multiple installs not under our direct control). Accordingly, right now, we don't want to change the performance characteristics or be forced to go through a data sanitization process - we simply don't have the time budget at the moment as there are hundreds of tables in the database - and we just want to be able to use navigation properties in our code rather than ugly explicit joins. Furthermore, we'd be willing to handle data integrity errors at runtime pending a more complete solution. In an ideal world we would begin from a different starting position but it's very much a brownfield project. |
@zejji So what kinds of changes are you making to the database? |
@ajcvickers - Features are still being added to the application, which means that any new migrations generally create new tables (which can happily include foreign + unique key constraints because they don't contain any legacy data). The issue is that we want to use EF Core effectively with the old database tables. |
@zejji Thanks, understood. |
Any updates on this? I am in a similar position. Existing database uses not-null |
@dahovey Sounds like you will need #13146, since key value zero violates the normal constraint. Both these issues are in the Backlog milestone. This means that it is not planned for the next release (EF Core 6.0). We will re-assess the backlog following the this release and consider this item at that time. However, keep in mind that there are many other high priority features with which it will be competing for resources. |
I have a related concern for SQLite - that case there's supposed to be a way to just turn off FK constraints globally: I was following the SO sample from @bricelam: But when I try to run it with that set I get: Any Help appreciated! Running: For now I'm hacking around this with like:
|
So, can we navigate without any real database foreign key constraints now? I'm from #20744 |
+1 to this |
+1 for this feature |
+1 |
Everyone, please refrain from posting more comments saying "I need this" - these don't help us prioritize this issue. Make sure you vote (:+1:) on the top post, we do look at that. |
We manually removed all the foreign keys from the migrations and it worked fine at first glance. But we are using an SQLite DB and it turned out that there are many cases when EF migrations must rebuild a table. And when the table gets rebuilt, all foreign keys get added again and there is no way to customize the migration because the rebuild happens magically behind the scenes. In such cases, a lot of migration code must be written manually. See dotnet/EntityFramework.Docs#4429 |
@berhir See if the following helps your use case: |
Yes, on SQLite, you can disable all the database logic for foreign keys by adding The foreign keys will still be created in the SQL, but SQLite will simply ignore them. |
Thanks, I am aware of this flag, and I have already used it successfully. |
Our implementation of table rebuilds assumes that your EF model is in sync with the database. Instead of altering the database schema after a migration is generated, you should alter the EF model to reflect what you want the database to look like. But alas, removing the foreign keys from the EF model would require this issue to be implemented. |
@bricelam I have the same problem with triggers, they get removed too. Unfortunately, my suggestion I mentioned above to make the table rebuilds transparent by adding the code to the generated migration was moved to the docs repo. Not sure how better docs can solve this. |
@roji If you need some ammo, we are a native, mobile-first tech startup, who have left the Windows ecosystem over a decade ago, run Swift and Kotlin as our primary languages, and have every reason to have our back-end stack be anything OTHER than MS/.NET. EXCEPT for the fact that we value domain-driven design and the best technology to meet that promise in any ecosystem is EF Core. Therefore fine, we take on requiring a different tech stack, language, IDE, cloud provider (.NET, C#, Rider and Azure respectively) all so that we can access this unique ORM - itself made possible by C#’s equally unique Expressions API (which has become very stagnant unfortunately and stuck in pre-nullable era). I am the CEO of this company, and have had every chance to choose otherwise, despite the recruitment challenges of running a .NET backend stack in the startup world. THAT is how paramount ORM technology is. Is the very lifeblood of the backend stack - MUCH more than the endpoint wrappers (ASP.NET/Functions) or infrastructure components (Azure) that support/expose it. Please DO pass this message up the chain and seriously ask for a doubling/tripling of the team in the year ahead - because we would not be on Azure and we would not be on .NET if it wasn’t for Entity Framework. Fact. You guys need more people like us succeeding, spreading the word and getting others to incorporate the best-of-each-world paradigm for each piece of the tech stack —> particularly why we run iOS/Swift native, Kotlin/Android native, and C#/.NET/Azure for back-end, because for crunching data and modeling your business domain it has the potential to be the best technology in any tech stack period. In a world where MS lost on the entire mobile front and cloud is the best strategic focus for the company as a whole - it is in Microsoft’s core interest to win as a leader in the cloud space, with EF as the heart of that differentiation in the market. But you guys badly need more throughput and to fill in all the holes in the promises to actually fulfill the standout potential at hand. PS: This is NOT a vote for the importance of this particular issue, we don’t have need for this specific scenario. It is a comment on the very valid and broad gaps on edge case / quality of life / MVP mindset which has been consistently prominent in the development of EF Core. I don’t think you guys are necessarily off to hyper-prioritize across horizontally rather than knocking one single feature completely out of the park (though arguments could likely be made there), but there IS a big disconnect between the resources the team actually has VS what is intrinsically needed to tackle the underlying challenges at hand. The result is that feature after feature for the past 3+ EF releases we cannot adopt any of it because no POC of major new EF functionality ever makes it to production-grade readiness. Not one. Please make the case - and go get - a doubling/tripling of the EF team. It’s that simple. 💪 🚀 |
@marchy thank you for writing all that - it is greatly appreciated - really (and may indeed be helpful in internal discussions). We get surprisingly little messages like this one, explaining the actual place of EF in users' actual, concrete projects and considerations - I wish we had more like these, both positive and negative. We're indeed aware that there are quite a few rough edges, and various features aren't "complete" (or don't always interoperate well with other features).. That's indeed something we'd like to make better. But I'd be interested in knowing more specifics about recent EF features which you consider unusable/not production-grade; if you're interested in having that conversation, please write to the email on my github profile (so as to prevent over-spamming this issue). Thanks again for taking the time to write the above! ❤️ |
Holy wow, yeah agreed on all counts @marchy. @roji - If you want business cases... Hi, I'm Alex ™️. As of this comment, I'm an architect at one of the worlds largest travel companies. In the past, I've held similar roles at smaller, successful companies where I've done everything from migrate to .NET Core, to introducing .NET anew. I've also moved companies away from Azure as a consequence of the sloppy out-of-our-hands issues and gaps we faced. My background prior to .NET was in the very strong Symfony/Laravel ecosystems, so the bar is very high. Wearing my psuedo-anthropological hat, I think the model Microsoft has adopted today is far too press-release vs. product-quality oriented. The MVP approach eluded to by marchy actually plagues all of Microsoft. The consistency in gaps between teams and departments makes it obvious that there's something top-down going on here and the reputational cost isn't being factored for. I've become fairly accustomed to .NET features and technologies only getting enough depth to pass a stage demo. The consequence? Most new EF features aren't having a big impact on making my life as a developer easier. How many stale topics are festering, waiting for enough votes on to get certain JSON or polymorphism features in place? Yet release after release, I see them "pushed to 6, pushed to 7, pushed to 8, pushed to 9". And I get it, 'softies probably grumble when I show up, poking holes in your product strategies and trying to offer insights. I even bet some think I'm ungrateful. But the treatment I receive sometimes and the gaps are just so consistently similar in nature and origin. We're your customers and I've absolutely made decisions based on my experiences that impact your bottom line. |
If anyone runs into this issue when trying to disable the foreign key constraints whilst using SQLite(I did when unit testing) the following code worked for me:
If you'd prefer use a mssql db, the following c#/sql code drops all the foreign keys in the db:
|
I found my way here after about an hour of banging my head against:
This cropped up in what feels like a stupid-simple, convention-based model. To be clear, I do NOT want to use foreign keys in my database. I actually don't understand why they are even truly required to enable the navigations feature. I feel if the EFCore team had the same convinctions some of us have against FKs we could very quickly find a happy path to make this work. We are going to start hedging back towards common queries in an extension method w/ Dapper. I can tell that a "no FK" branch of EFCore is going take longer than we are willing to wait. |
Please implement this with an option to create the FK index but without the constraint. I don't want the DB to enforce the relationship, but I still want the performance benefits of the index. My application contains supporting data brought in from external integrations. That supporting data is fully modeled in our application so it can be queried using EF. However, data in the external systems has reached retention age and the old data must be purged from our system. The constraints make that process very inefficient. Currently, with the FKs constrained:
If the FK could generate the related index without the constraint, this process is simpler:
I would like to see a configuration syntax of something like this:
That would not create the FK constraint in the database, but will tell EF to use the index for navigation work and would create an index similar to this:
|
Gotta join the party too. I have a similar use case where I need to setup a navigation property w/o foreign key constrains. The particular use case is, that we sync some data via message bus across services when ever they change in the authoriatve service, for performance reason (its hard to make an efficient join with an data that only exists in a different microservice which has its own database). The issue is, some entities synchronized that way have references to other objects, that can be changed on their own (and hence need a separate event triggered, because EF core do not flag the parent entity as changed), so we have subscribes for that type of entity too public class ParentEntity
{
public int Id { get; set;}
public string Name { get; set;}
public ReferencedEntity Reference { get; set;}
}
public class ReferencedEntity
{
public int Id { get; set;}
public string Name { get; set;}
public int? ParentId { get; set;}
public ParentEntity Parent { get; set;}
} Now due to nature of message buses, sending messages and re-queuing when an error happens, the messages can arrive out of order, i.e. the So we'd like to not create the foreign key in the first place (but still have the relationship configured, so include can be used), then Alternatively, I'd be happy if there'd be an easy mechanism to notify when any referenced entity of a specific parent entity (aggregate root) changes, i.e. when Then we could of course also change the messaging to only be based on the aggregate/root entity that contains the full object graph, but I have no idea on how to attempt that w/o tons of reflection and additional tracking @ajcvickers @roji Any directions? |
@TsengSR it sounds like if your messages really arrive out of order and you just apply the changes, your database really will be in an inconsistent state (at least temporarily); that means that if someone queries the database at that moment, they'll see an incorrect view of things, and EF may actually completely fail (e.g. since there's no principal although there must be one). I generally really encourage people to think carefully before asking for "no foreign keys"; in many cases, that indicates a problematic design that should be addressed elsewhere. Note that database also sometimes support "deferrable" transactions, where foreign key constraints are only enforced when the transaction is actually commit; that can allow a temporary inconsistent state, but only within the transaction itself (and the inconsistency is therefore invisible outside of it). I don't know enough about your application to give any advice here - but changing "the messaging to only be based on the aggregate/root entity that contains the full object graph" sounds like it's too much (or missing the main thing). The point is that each change you apply (via your message) should always lead to a consistent state in your data - this is even regardless of whether foreign keys are enforced in your database or not. Disabling foreign keys just pushes the problem down further down to whoever is reading from your database, as they now have to deal with inconsistent states. |
Isn't an issue for me in this case, since the property will be unreachable anyways until the parent property gets processed/synched. But as far as I see it, there won't even be any exceptions, since the relation is just used to build joins when doing includes and then either the parent or child entity is missing it gives no result or null result, depending on whether or not its an inner, outer or left/right join. The synched data is assumed as eventual consistent (no different then any other eventually consistent system really), so it is made with the assumption that there may be some time where its not 100% consistent, as long as it gets eventually consistent its fine, since its not authoritative data. The authoritative data (source of truth) is the original service
Yea, but this doesn't work with messaging, where a message is processed one after other. For regular transactions this approach is fine and all of the a services authoritative tables have and use foreign keys and enforce them on database level. Its just in that special case where I need to synchronize some entities (or just part of them with a minimal set of properties) across another service to be able to perform queries. Assume a service which manages storage (how much of each material is located in which part of the storage hall, which shelve and which location within the shelf) and another service with material (materialnumber, description, weight, dimensions etc.) and now every time a material is added or changed, the accordingly data in the storage service needs to be updated too. Why need the data there? Well, people want to be able to query for a specific material, material type or just when listing whats currently in the storage want to know which material is there and also have the changed in the material service reflected. Since its a lot of data, the only efficient way to include this data into the query to display for the user is by joining it on the database rather than have to fetch each material from the material service.
Well if it would be an object database, it would be easy really, since the material and all of its references would be a single object. The problem is, its not a document database and how EF Core tracks modified entities. Using the example above, when someone updates ReferenceEntity, there will be no indication on ParentEntity entity = await dbContext.Parents.Include(p => p.Reference).SingleOrDefault(p => p.Id == 1);
entity.Reference.Name = "New Name";
await dBContext.SaveChangesAsync(); In this, only That's why for now, I send for every changed entity a message. However, when doing an add (adding new parent and new reference) the reference, two messages are sent and the reference one is more often than not the first one to arrive. And I can't think of any way for a generic solution that would properly and automatically determine the parent entity (or aggregate root more specifically) to do notifications via this |
Something else to consider is what is the alternative solution for this issue. For my system:
1 must be done for every new relationship and query. That adds development and maintenance overhead, and is easily overlooked. There are number of EF features and functionality that come with warnings about their use. I would put this feature into the same category. A big yellow warning box on the feature documentation mentioning the concerns about data integrity is sufficient. It should only be used when needed, and the implementer should understand the risks. |
Note: This issue is about only not creating the constraint in the database. EF will still treat the relationship as constrained. If you want an unconstrained relationship--that is, a relationship where an FK value is non-null but there is no corresponding PK in the database--then please vote for #13146. |
It has been noted in this thread that the upvotes count is considered when implemented 'quality of life' improvements to .EF, and that these are important to the team. Given that this issue has over 200 upvotes it has to be one of the most in demand requests for the framework on the metric of Popularity / [Required Effort]. At first blush it appears that providing a property/attribute to exclude a key from the migration wouldn't be a major task compared to many low value additions that do make the cut for new versions. Can we please just get it done? The community has been giving clear feedback over 5 years as it has been asked to do, but it doesn't appear that anyone is really reviewing the guidance users are providing as to important issues for us through the only platform we really have - even though we try to do this [generally] with respect and in accordance with the directions we are given. |
It's impressive to see this issue from over five years ago still not being done. I'm working with a database export, in a situation where I can't exactly massage it into the correct shape with FKs and all (since it's not going to be bundled with my application), but I do want to follow the navigable data. And my case is also much like many above: old data that doesn't quite have the correct values to make a FK work. |
I ran thru the same issue and added this code to my DBContext seems like it works and does not create and Foreign Keys ` public class CustomMigrationOperationGenerator : MigrationsSqlGenerator
} ` |
Note: This issue is about only not creating the constraint in the database. EF will still treat the relationship as constrained. If you want an unconstrained relationship--that is, a relationship where an FK value is non-null but there is no corresponding PK in the database--then please vote for #13146.
I try to use
RemoveForeignKey
to remove all foreign keys from my DbContext, but it is not usable.Test DbContext
The foreign key is still in the generated
Snapshot
.Further technical details
EF Core version: .Net Core 2.2
Database Provider: Microsoft.EntityFrameworkCore.SqlServer 2.2.4
Operating system:
IDE: Visual Studio 2019 16.1.1
The text was updated successfully, but these errors were encountered: