Add EF Core 7 style JSON support (owns & ToJson)
#1752
Comments
mkozlina
changed the title
|
Same issue |
|
Any solution or workaround? I am facing the same issue as well. |
|
same issue, dose anyone has solution? |
|
Pomelo has extended JSON support for both, the Microsoft stack and the Newtonsoft stack. Because we already implemented JSON support a couple of years ago, it works a bit different than the simple JSON support now supported by EF Core. For some (older) sample code, with lots of different ways to use JSON, see #14 (comment). The basics steps are to reference either the |
|
@lauxjpn does it mean that we cannot use The 2 properties are mutually exclusive : What would be the workaround to have OwnEntities with JSON representation in database with the current version of Pomelo ? Thanks ! Edit: With version : |
|
In case this helps someone I had this very exeption: And what solved the issue for me was removing from the entity configuration builder.OwnsOne(x => x.Properties, ownedNavigationBuilder =>
{
ownedNavigationBuilder.ToJson();
});and replacing that with: builder.Property(x => x.Properties).HasColumnType("json"); |
|
@lauxjpn - I have the same question as @Angelinsky7. Will using the builder.Property(p => p.Data)
.HasColumnType("json");as suggested. But I think this limits us to not be fine grained about how the owned JSON aggregate should be stored in the database. For example, what if we want to ignore a subset of the JSON data if we wanted to be meticulous, can we achieve that? So, something like for example: builder .OwnsOne(p => p.Data, dataBuilder => {
dataBuilder.Ignore(p => p.SomeProp);
});where we don't want to store a specific property part of Does something like this already exists and you can point me to? Or will something like be supported in the 8.0 release or later? |
|
@anasgauba It is questionable if we will support the However, since we have general support for both, the Microsoft JSON stack and the Newtonsoft JSON stack, all the general stack rules and features should be available, which include ignoring specific properties (e.g. How to ignore properties with System.Text.Json). Here is a quick example, using the Microsoft JSON stack (by calling Program.csusing System;
using System.Diagnostics;
using System.Linq;
using System.Text.Json.Serialization;
using Microsoft.EntityFrameworkCore;
using Microsoft.Extensions.Logging;
namespace IssueConsoleTemplate;
public enum OrderStatus
{
Pending,
Shipped
}
// JSON object class
public class StreetAddress
{
public string Street { get; set; }
public string City { get; set; }
[JsonIgnore] // <-- ignore property "Comments"
public string Comments { get; set; }
}
// EF Core entity
public class Order
{
public int Id { get; set; }
public OrderStatus Status { get; set; }
public StreetAddress ShippingAddress { get; set; }
}
public class Context : DbContext
{
public DbSet<Order> Orders { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
if (!optionsBuilder.IsConfigured)
{
var connectionString = "server=127.0.0.1;port=3306;user=root;password=;database=Issue1752";
var serverVersion = ServerVersion.AutoDetect(connectionString);
optionsBuilder
.UseMySql(
connectionString,
serverVersion,
options => options.UseMicrosoftJson()) // <-- enable json support
.LogTo(Console.WriteLine, LogLevel.Information)
.EnableSensitiveDataLogging()
.EnableDetailedErrors();
}
}
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Order>(
entity =>
{
entity.Property(e => e.ShippingAddress)
.HasColumnType("json"); // <-- explicitly specify the column type as "json"
});
}
}
internal static class Program
{
private static void Main()
{
using (var creationContext = new Context())
{
creationContext.Database.EnsureDeleted();
creationContext.Database.EnsureCreated();
}
using (var saveContext = new Context())
{
saveContext.Add(
new Order
{
Id = 101,
Status = OrderStatus.Pending,
ShippingAddress = new StreetAddress
{
City = "London",
Street = "221 B Baker St",
Comments = "This property will be ignored.",
}
});
saveContext.SaveChanges();
}
using (var queryContext = new Context())
{
var orders = queryContext.Orders.ToList();
Trace.Assert(orders.Count == 1);
Trace.Assert(orders[0].Id == 101);
Trace.Assert(orders[0].ShippingAddress.Comments is null);
}
}
}Output (SQL)If you want even more fine-grained control, you could override or replace the default value converter that we supply when you specify the public class MyCustomMySqlJsonMicrosoftPocoValueConverter<T> : MySqlJsonMicrosoftPocoValueConverter<T>
{
public MyCustomMySqlJsonMicrosoftPocoValueConverter()
: base(
v => ConvertToProviderCore(v),
v => ConvertFromProviderCore(v))
{
}
private static string ConvertToProviderCore(T v)
=> JsonSerializer.Serialize(v); // do whatever you want here
private static T ConvertFromProviderCore(string v)
=> JsonSerializer.Deserialize<T>(v); // do whatever you want here
}
// ...
entity.Property(e => e.ShippingAddress)
.HasColumnType("json")
.HasConversion<MyCustomMySqlJsonMicrosoftPocoValueConverter<StreetAddress>>()Or declare your custom conversion inline: entity.Property(e => e.ShippingAddress)
.HasColumnType("json")
.HasConversion(
v => JsonSerializer.Serialize(v, (JsonSerializerOptions)null),
v => JsonSerializer.Deserialize<StreetAddress>(v, (JsonSerializerOptions)null));(@Angelinsky7 Feel free to ask follow-up questions, if something is unclear or confusing.) |
|
@lauxjpn it feels sad, because for now it only means that pomelo don't "correctly" support ownsEntities and toJson... |
|
Yes, we currently don't support the However, we have been supporting JSON in a much more comprehensive way than We will implement |
|
@lauxjpn i totally understand your point of view.... sadly i don't think people reason like this : Like you said, you'll need to to this is the future for compatibility reasons, the sad thing is that it seem complicated for you to do it. But thanks again for you work and your support (i have used the HasConversion replacement in my onw code with a compiler flag to switch to sqlServer when building the integration tests, not what i would like but it's working) |
|
I'll put it on the list for the next release ( |
lauxjpn
changed the title
ToJson)
|
My app uses Pomelo in the production, and it uses Sqlite in-memory provider for integration tests, the model configuration is as follows. I got an error running integration tests, looks like Sqlite in-memory database doesn't support Could you please suggest a way for me to use JSON for Files property that works with both Pomelo provider and Sqlite in-memory provider? Could you also share the progress on Pomelo provider v8.0 supporting |
@LiangZugeng that's very much expected, SQLite is a different database and cannot behave like MySQL. It's highly recommended to test against MySQL instead, using something like testcontainers, which allow you to very easily spin up MySQL in a container and test against it. For more information on testing against fakes vs. against your actual database system, see these EF docs. |
|
@roji thanks for the advice of using testcontainers or similar for integration tests, I have been using Sqlite In-memory provider for integration tests all the time, but I'll try it out (it may be the better way). The reason I raised this question is that one of my .NET 6 projects is using Innofactor.EfCoreJsonValueConverter to convert properties to JSON (I have no use of query of JSON properties so that's relatively simpler), and this approach works with both Pomelo MySQL provider and Sqlite In-memory provider without changing any DBContext configuration code. Innofactor author decided to stop maintain that library as .NET 7's ToJson feature and she/he suggested using the new feature. While I'm starting a .NET 8 application, I just want to have a unified way for EF Core DbContext configurations. Anyway, while waiting for Pomelo provider to support |
|
@LiangZugeng if what you're looking for is to just store JSON text data (no querying, partial updating or the more advanced logic), you can absolutely continue doing that via a value converter. I have no idea what the Innofactor package does, but just defining a value converter that serializes/deserializes to JSON should be trivial - you probably don't need a package to do that. |
|
@roji Yep, I added the code as a value converter and everything works fine. Thank you. |
|
I'm confused. ToJson was added or not in the version of EF Core 8? |
|
I have a List in my entity and to make it work with EF Core I need to create an extra table. Whereas with ToJson this could be stored as json and retrieved/queried easily. |
|
Just a comment about this issue. I've been using JSON support with Pomelo for a while but now that a few enterprise applications I'm developing are getting bigger we are introducing integration tests with To speed up this kind of test we are using SqlLite (in memory connection) instead a mariadb docker one (we have already persistence tests). My problem now is that SqlLite supports toJson since ef core 8.0 and Pomelio supports only HasColumnType('json'). For now, we have found a workaround using property value converters, on other applications we have different I write this comment only to point out that in these contexts support ef core standards will be greatly appreciated. |
@meriturva Without wanting to dive deeper into the subject here, I highly suggest to use an instance similar to your actual target database in your integrations tests. Sqlite and its EF Core providers are different in their feature set and query capabilities to Pomelo and MariaDB. My practical suggestion would be to create docker containers as part of the integration test initialization instead. |
|
Testing is a very recurring question, and I very much agree with @lauxjpn that using SQLite as a database fake is a bad idea. I'll just add that testcontainers make it very easy to spin up a database in a container as part of the tests. |
|
Thanks, @roji and @lauxjpn I know your advice and, in fact, we have different tests with testcontainer.net and it works like a charm. What I would like to underline is that enterprise projects have different facets and technical evolutions. There are many integrations with many layers to talk about and test. Sometimes you make choices, even questionable ones, but all this is the result of technical discussions on where you want to place your attention in your tests. Having said this, I reiterate that having standard support means being able to make choices that are otherwise locked to the chosen ef core driver. My two cents. |
|
how about nest class? like } public class PerProcessingRules } public class SegmentationRulse } builder.Entity(b => b.Property(p => p.Rules).HasColumnType("json")); it can not works |
Code
Steps to reproduce
The issue
Throws a DbUpdateException on context.SaveChanges(), with an inner InvalidCastException saying that it cannot cast from 'String' to 'JsonElement'.
Further technical details
Database version: MySQL v8.0.27 and MariaDB v10.6.11
Operating system: Windows 10 build 19044.2486
Pomelo.EntityFrameworkCore.MySql version: 7.0.0
Microsoft.AspNetCore.App: 6.0.13 and 7.0.2
Other info
I tried this on multiple machines and always end up with the same exception. It only happens when aggregating to JSON (nb.ToJson()).
The text was updated successfully, but these errors were encountered: