Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RevEng: InvalidOperationException scaffolding primary/foreign key columns with different precisions #19793

Open
rkaushik15 opened this issue Feb 4, 2020 · 14 comments
Labels
area-model-building area-scaffolding customer-reported needs-design punted-for-5.0 punted-for-6.0 punted-for-7.0 Originally planned for the EF Core 7.0 (EF7) release, but moved out due to resource constraints. type-bug
Milestone

Comments

@rkaushik15
Copy link

rkaushik15 commented Feb 4, 2020

I am working with an application using .NET Core 2.2 and Oracle.EntityFrameworkCore 2.19.60 provider for Oracle DB and running into this issue.

When trying to scaffold an Oracle database (using Oracle's EntityFrameworkCore Provider) which contains a Foreign Key such as NUMBER(12,0) (Scale is Zero) referencing a Principal Key such as NUMBER (No Precision or Scale specified), an error is thrown.

SQL Server does NOT allow creating a database containing a Foreign Key of type NUMERIC(*, 0) that references a Principal Key of type NUMERIC. Hence the behavior of the MS EFCORE seems to be in sync with SQL Server's behavior.
But in Oracle DB, it is possible to have a database containing a Foreign Key of type NUMBER(*, 0) referencing a Principal Key of type NUMBER.

Example of the error being thrown:

The types of the properties specified for the foreign key {'ScaffIssueTestFk'} on entity type 'ScaffIssueTest' do not match the types of the properties in the principal key {'ScaffIssueTestPk'} on entity type 'ScaffIssueTest'.

Oracle DB allows such an FK, PK relationship as these types are considered compatible, and hence scaffolding a database with such a relationship should not throw any errors.

On analyzing the issue a bit I think the root cause of the issue lies here:

efcore/src/EFCore/Metadata/Internal/EntityType.cs

I can see the AddForeignKey() method calls the static ForeignKey.AreComapatible() method which checks for type compatibility. Following the flow, the exception is thrown when the ArePropertyTypesCompatible() method is called which has the following code:

private static bool ArePropertyTypesCompatible(
            IReadOnlyList<IProperty> principalProperties, IReadOnlyList<IProperty> dependentProperties)
            => principalProperties.Select(p => p.ClrType.UnwrapNullableType()).SequenceEqual(
                dependentProperties.Select(p => p.ClrType.UnwrapNullableType()));

Here the ClrType of the principalProperties and dependentProperties is different in the case of the Oracle DB Provider, but nevertheless compatible.
But since in the case of SQL Server these must be STRICTLY the same, the code is not causing any issue.

NOTE: NUMBER type maps to the ClrType System.Decimal and NUMBER(*,0) type maps to the ClrType System.Int64 or System.Int32, etc. depending on the Precision.

Steps to reproduce

  1. Connect to an Oracle DB instance and run the SQL statements present in the attached file create_user.txt. Switch to the new user created and run the SQL statement in the attached file create_schema.txt.

  2. Open a VS 2017 command prompt.

  3. Run the following commands:

 dotnet new globaljson --sdk-version 2.2.402
 dotnet new webapi -n api
 cd api
 dotnet add package Oracle.EntityFrameworkCore -v 2.19.60
  1. Modify the following command with your DB schema info and run:
dotnet ef dbcontext scaffold "Data
Source=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=127.0.0.1)(PORT=1521))(CONNEC
T_DATA=(SERVER=DEDICATED)(SERVICE_NAME=ORCLCDB.localdomain)));User Id=test_user;Password=pass;" Oracle.EntityFrameworkCore -o Models

Exception Message and Stack Trace

>dotnet ef dbcontext scaffold "Data Source=<data_source>;User Id=test_user;Password=pass;" Oracle.EntityFrameworkCore -o Models
System.InvalidOperationException: The types of the properties specified for the foreign key {'ScaffIssueTestFk'} on entity type 'ScaffIssueTest' do not match the types of the properties in the principal key {'ScaffIssueTestPk'} on entity type 'ScaffIssueTest'.
   at Microsoft.EntityFrameworkCore.Metadata.Internal.ForeignKey.AreCompatible(IReadOnlyList`1 principalProperties, IReadOnlyList`1 dependentProperties, EntityType principalEntityType, EntityType dependentEntityType, Boolean shouldThrow)
   at Microsoft.EntityFrameworkCore.Metadata.Internal.ForeignKey.AreCompatible(EntityType principalEntityType, EntityType dependentEntityType, MemberInfo navigationToPrincipal, MemberInfo navigationToDependent, IReadOnlyList`1 dependentProperties, IReadOnlyList`1 principalProperties, Nullable`1 unique, Nullable`1 required, Boolean shouldThrow)
   at Microsoft.EntityFrameworkCore.Metadata.Internal.EntityType.AddForeignKey(IReadOnlyList`1 properties, Key principalKey, EntityType principalEntityType, Nullable`1 configurationSource)
   at Microsoft.EntityFrameworkCore.Metadata.Internal.EntityType.Microsoft.EntityFrameworkCore.Metadata.IMutableEntityType.AddForeignKey(IReadOnlyList`1 properties, IMutableKey principalKey, IMutableEntityType principalEntityType)
   at Microsoft.EntityFrameworkCore.MutableEntityTypeExtensions.GetOrAddForeignKey(IMutableEntityType entityType, IReadOnlyList`1 properties, IMutableKey principalKey, IMutableEntityType principalEntityType)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.RelationalScaffoldingModelFactory.VisitForeignKey(ModelBuilder modelBuilder, DatabaseForeignKey foreignKey)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.RelationalScaffoldingModelFactory.VisitForeignKeys(ModelBuilder modelBuilder, IList`1 foreignKeys)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.RelationalScaffoldingModelFactory.VisitDatabaseModel(ModelBuilder modelBuilder, DatabaseModel databaseModel)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.RelationalScaffoldingModelFactory.Create(DatabaseModel databaseModel, Boolean useDatabaseNames)
   at Microsoft.EntityFrameworkCore.Scaffolding.Internal.ReverseEngineerScaffolder.ScaffoldModel(String connectionString, IEnumerable`1 tables, IEnumerable`1 schemas, String namespace, String language, String contextDir, String contextName, ModelReverseEngineerOptions modelOptions, ModelCodeGenerationOptions codeOptions)
   at Microsoft.EntityFrameworkCore.Design.Internal.DatabaseOperations.ScaffoldContext(String provider, String connectionString, String outputDir, String outputContextDir, String dbContextClassName, IEnumerable`1 schemas, IEnumerable`1 tables, Boolean useDataAnnotations, Boolean overwriteFiles, Boolean useDatabaseNames)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.ScaffoldContextImpl(String provider, String connectionString, String outputDir, String outputDbContextDir, String dbContextClassName, IEnumerable`1 schemaFilters, IEnumerable`1 tableFilters, Boolean useDataAnnotations, Boolean overwriteFiles, Boolean useDatabaseNames)
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.ScaffoldContext.<>c__DisplayClass0_1.<.ctor>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.<>c__DisplayClass3_0`1.<Execute>b__0()
   at Microsoft.EntityFrameworkCore.Design.OperationExecutor.OperationBase.Execute(Action action)
The types of the properties specified for the foreign key {'ScaffIssueTestFk'} on entity type 'ScaffIssueTest' do not match the types of the properties in the principal key {'ScaffIssueTestPk'} on entity type 'ScaffIssueTest'.

Further technical details

EF Core version: Microsoft.EntityFrameworkCore 2.2.4
Database provider: Oracle.EntityFrameworkCore 2.19.60
Target framework: .NET Core 2.2 (netcoreapp2.2)

@ajcvickers
Copy link
Member

@rkaushik15

NOTE: NUMBER type maps to the ClrType System.Decimal and NUMBER(*,0) type maps to the ClrType System.Int64 or System.Int32, etc. depending on the Precision.

EF can't deal with C# entity types that have different primary and foreign key types. For example, this cannot be mapped by EF:

public class Item
{
    public int Id { get; set; }
    public ICollection<Tag> Tags { get; set; }
}

public class Tag
{
    public int Id { get; set; }
    
    public decimal ItemId { get; set; }
    public Item Item  { get; set; }
}

So it's not the validation that is wrong, but rather EF can't handle the model that is being created.

As a workaround, make the CLR types match but map one to a different column type. For example:

public class Item
{
    public int Id { get; set; }
    public ICollection<Tag> Tags { get; set; }
}

public class Tag
{
    public int Id { get; set; }
    
    [Column(TypeName = "NUMBER")]
    public int ItemId { get; set; }
    public Item Item  { get; set; }
}

@rkaushik15
Copy link
Author

@ajcvickers Thanks for your prompt reply.

As a workaround, make the CLR types match but map one to a different column type.

I understand that the workaround would be to change the mapping in the model, owing to the limitations of EF in dealing with different primary and foreign key types.

But for my particular use case, the issue occurs when I am trying to scaffold an existing database. The exception is thrown during the scaffolding process and hence, the model is never created in the first place.

Would there be a way to apply this workaround or any other way to achieve the same effect, before the model is created, i.e., when scaffolding (reverse engineering)?

@ajcvickers
Copy link
Member

@rkaushik15 We'll discuss the impact on scaffolding and get back to you.

@ajcvickers
Copy link
Member

@rkaushik15 We discussed this and we plan to either:

  • At least not make this abort scaffolding
  • Add the converter in the scaffolding if low cost

For now, the only workaround the scaffolding aspect is probably to exclude those tables that are like this and then write the code for those manually. You could also try duplicating the schema and the modifying it just so scaffolding will complete.

@ajcvickers ajcvickers added this to the 5.0.0 milestone Feb 8, 2020
@bricelam bricelam changed the title System.InvalidOperationException when scaffolding primary/foreign key columns with different precisions RevEng: InvalidOperationException scaffolding primary/foreign key columns with different precisions Feb 11, 2020
@ajcvickers ajcvickers modified the milestones: 5.0.0, Backlog Jun 9, 2020
@kevin-shelaga
Copy link

Any updates on this? Running into this after converting a db from mssql to postgresql.

bigint -> integer

@ajcvickers
Copy link
Member

@kevin-shelaga This issue is in the Backlog milestone. This means that it is not planned for the next release (EF Core 5.0). We will re-assess the backlog following the this release and consider this item at that time. However, keep in mind that there are many other high priority features with which it will be competing for resources.

@bricelam bricelam removed this from the 7.0.0 milestone Aug 30, 2022
@ajcvickers
Copy link
Member

@bricelam I believe #29026 should make "Add the converter in the scaffolding" feasible.

@ajcvickers ajcvickers added propose-punt punted-for-7.0 Originally planned for the EF Core 7.0 (EF7) release, but moved out due to resource constraints. labels Sep 9, 2022
@ajcvickers ajcvickers added this to the Backlog milestone Sep 9, 2022
@danarrib
Copy link

danarrib commented Nov 19, 2022

Hello. Any updates on this? I'm have this exact same issue (trying to scaffold a database with two tables that has different data types relationship).

The types of the properties specified for the foreign key {'LocationId'} on entity type 'NotificationSetting' do not match the types of the properties in the principal key {'Id'} on entity type 'Location'.

It's a PostgreSQL database. Location table has Id column of type serial(4), and NotificationSetting table has the LocationId column of type int8. Why? I don't know... The guy before me is kinda crazy.

Thank you.

@bricelam bricelam removed their assignment Jul 8, 2023
@mt3sliv
Copy link

mt3sliv commented Sep 7, 2023

Any update on this?

@mt3sliv
Copy link

mt3sliv commented Sep 7, 2023

Can T4 templates be a possible work around?

@danarrib
Copy link

danarrib commented Sep 7, 2023

What I did to solve MY problem:

  • Generated the SQL Script for all database objects
  • Created a new local database and run the script to create the empty tables
  • Change the columns types as needed
  • Once the model is generated, I can use it on the actual production database without problems

@mt3sliv
Copy link

mt3sliv commented Sep 7, 2023

@danarrib Thanks for the response! Not a bad approach at all but I don't have the ability/privileges to create a local database to do this :/

@cmpnnt

This comment has been minimized.

@roji

This comment has been minimized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-model-building area-scaffolding customer-reported needs-design punted-for-5.0 punted-for-6.0 punted-for-7.0 Originally planned for the EF Core 7.0 (EF7) release, but moved out due to resource constraints. type-bug
Projects
None yet
Development

No branches or pull requests

10 participants