Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document the new support for SQL Server UTF-8 columns #4035

Merged
merged 1 commit into from
Sep 19, 2022
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 32 additions & 6 deletions entity-framework/core/providers/sql-server/columns.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,41 @@ This page details column configuration options that are specific to the SQL Serv

## Unicode and UTF-8

SQL Server has two column types for storing textual data: [`nvarchar(x)`](/sql/t-sql/data-types/nchar-and-nvarchar-transact-sql) and [`varchar(x)`](/sql/t-sql/data-types/char-and-varchar-transact-sql); these have traditionally been used to hold Unicode data in the UTF-16 encoding and non-Unicode data, respectively. SQL Server 2019 [introduced](/sql/relational-databases/collations/collation-and-unicode-support#utf8) the ability to store UTF-8 Unicode data in `varchar(x)` columns.
SQL Server 2019 introduced [introduced UTF-8](/sql/relational-databases/collations/collation-and-unicode-support#utf8) support, which allows storing UTF-8 data in `char` and `varchar` columns by configuring them with special UTF-8 collations. EF Core 7.0 introduced full support for mapping to UTF-8 columns, and it's possible to use them in previous EF versions as well, with some extra steps.

Unfortunately, this does not currently work out-of-the-box with EF Core's SQL Server provider. To map a string property to a `varchar(x)` column, the Fluent or Data Annotation API is typically used to disable Unicode ([see these docs](xref:core/modeling/entity-properties#unicode)). While this causes the correct column type to be created, it also makes EF Core send database parameters in a way which is incompatible with UTF-8 data: `DbType.AnsiString` is used (signifying non-Unicode data), but `DbType.String` is needed to properly send Unicode data.
### [EF Core 7.0](#tab/ef-core-7)

To store UTF-8 data in SQL Server, follow these steps:
EF Core 7.0 includes first-class support for UTF-8 columns. To configure them, simply configure the column's type to `char` or `varchar`, specify a UTF-8 collation (ending with `_UTF8`), and specify that the column should be Unicode:

* Configure the collation for the property with one of SQL Server's UTF-8 collations; these have a `UTF8` suffix ([see the docs on collations](xref:core/modeling/entity-properties##column-collations)).
* Do not disable Unicode on the property; this will cause EF Core to create an `nvarchar(x)` column.
* Edit the migrations and manually set the column type to `varchar(x)` instead.
```c#
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Blog>()
.Property(b => b.Name)
.HasColumnType("varchar(max)")
.UseCollation("LATIN1_GENERAL_100_CI_AS_SC_UTF8")
.IsUnicode();
}
```

#### [Older versions](#tab/older-versions)

In EF Core versions prior to 7.0, UTF-8 columns do not work out-of-the-box with EF Core's SQL Server provider. To map a string property to a `varchar(x)` column, the Fluent or Data Annotation API is typically used to disable Unicode ([see these docs](xref:core/modeling/entity-properties#unicode)). While this causes the correct column type to be created in the database, it also makes EF Core send database parameters in a way which is incompatible with UTF-8 data: `DbType.AnsiString` is used (signifying non-Unicode data), but `DbType.String` is needed to properly send Unicode data.

As a result, you'll have to configure the EF property as a regular `nvarchar` column, ensuring that parameters are sent with the correct `DbType`:

```c#
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity<Blog>()
.Property(b => b.Name)
.UseCollation("LATIN1_GENERAL_100_CI_AS_SC_UTF8");
}
```

Once you've created the migration for the column, edit it and manually change the column's type from `nvarchar` to `varchar`.

***

## Sparse columns

Expand Down