Skip to content

Commit

Permalink
Document the new support for SQL Server UTF-8 columns
Browse files Browse the repository at this point in the history
  • Loading branch information
roji committed Sep 17, 2022
1 parent dfcd712 commit 0b0dc2c
Showing 1 changed file with 32 additions and 6 deletions.
38 changes: 32 additions & 6 deletions entity-framework/core/providers/sql-server/columns.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,41 @@ This page details column configuration options that are specific to the SQL Serv

## Unicode and UTF-8

SQL Server has two column types for storing textual data: [`nvarchar(x)`](/sql/t-sql/data-types/nchar-and-nvarchar-transact-sql) and [`varchar(x)`](/sql/t-sql/data-types/char-and-varchar-transact-sql); these have traditionally been used to hold Unicode data in the UTF-16 encoding and non-Unicode data, respectively. SQL Server 2019 [introduced](/sql/relational-databases/collations/collation-and-unicode-support#utf8) the ability to store UTF-8 Unicode data in `varchar(x)` columns.
SQL Server 2019 introduced [introduced UTF-8](/sql/relational-databases/collations/collation-and-unicode-support#utf8) support, which allows storing UTF-8 data in `char` and `varchar` columns by configuring them with special UTF-8 collations. EF Core 7.0 introduced full support for mapping to UTF-8 columns, and it's possible to use them in previous EF versions as well, with some extra steps.

Unfortunately, this does not currently work out-of-the-box with EF Core's SQL Server provider. To map a string property to a `varchar(x)` column, the Fluent or Data Annotation API is typically used to disable Unicode ([see these docs](xref:core/modeling/entity-properties#unicode)). While this causes the correct column type to be created, it also makes EF Core send database parameters in a way which is incompatible with UTF-8 data: `DbType.AnsiString` is used (signifying non-Unicode data), but `DbType.String` is needed to properly send Unicode data.
### [EF Core 7.0](#tab/ef-core-7)

To store UTF-8 data in SQL Server, follow these steps:
EF Core 7.0 includes first-class support for UTF-8 columns. To configure them, simply configure the column's type to `char` or `varchar`, specify a UTF-8 collation (ending with `_UTF8`), and specify that the column should be Unicode:

* Configure the collation for the property with one of SQL Server's UTF-8 collations; these have a `UTF8` suffix ([see the docs on collations](xref:core/modeling/entity-properties##column-collations)).
* Do not disable Unicode on the property; this will cause EF Core to create an `nvarchar(x)` column.
* Edit the migrations and manually set the column type to `varchar(x)` instead.
```c#
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Blog>()
.Property(b => b.Name)
.HasColumnType("varchar(max)")
.UseCollation("LATIN1_GENERAL_100_CI_AS_SC_UTF8")
.IsUnicode();
}
```

#### [Older versions](#tab/older-versions)

In EF Core versions prior to 7.0, UTF-8 columns do not work out-of-the-box with EF Core's SQL Server provider. To map a string property to a `varchar(x)` column, the Fluent or Data Annotation API is typically used to disable Unicode ([see these docs](xref:core/modeling/entity-properties#unicode)). While this causes the correct column type to be created in the database, it also makes EF Core send database parameters in a way which is incompatible with UTF-8 data: `DbType.AnsiString` is used (signifying non-Unicode data), but `DbType.String` is needed to properly send Unicode data.

As a result, you'll have to configure the EF property as a regular `nvarchar` column, ensuring that parameters are sent with the correct `DbType`:

```c#
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Blog>()
.Property(b => b.Name)
.UseCollation("LATIN1_GENERAL_100_CI_AS_SC_UTF8");
}
```

Once you've created the migration for the column, edit it and manually change the column's type from `nvarchar` to `varchar`.

***

## Sparse columns

Expand Down

0 comments on commit 0b0dc2c

Please sign in to comment.