Testing with different database providers

More docs and samples

Fixes #430
Fixes #1304
Fixes #1724
Fixes #1929
Fixes #1929
Fixes #2236
Fixes #2262
Fixes #2301

entity-framework/core/miscellaneous/testing/in-memory.md

@@ -1,64 +1,23 @@
 ---
 title: Testing with InMemory - EF Core
-author: rowanmiller
+author: ajcvickers
+description: Using the EF in-memory database to test an EF Core application
 ms.date: 10/27/2016
-ms.assetid: 0d0590f1-1ea3-4d5c-8f44-db17395cd3f3
 uid: core/miscellaneous/testing/in-memory
 ---
 
-# Testing with InMemory
+# Using the EF in-memory database to test an EF Core application
 
-The InMemory provider is useful when you want to test components using something that approximates connecting to the real database, without the overhead of actual database operations.
+> [!WARNING]
+> The in-memory database often behaves differently than common relational databases.
+> Only use the in-memory database after fully understanding the issues and trade-offs involved, as discussed in [Testing code that uses EF Core](xref:core/miscellaneous/testing/index).  
 
-> [!TIP]  
-> You can view this article's [sample](https://github.com/dotnet/EntityFramework.Docs/tree/master/samples/core/Miscellaneous/Testing) on GitHub.
+> [!TIP]
+> SQLite can use in-memory databases.
+> Consider using this for testing to more closely match common relational database behaviors.
+> This is covered in [Using SQLite to test an EF Core application](xref:core/miscellaneous/testing/sqlite).   
 
-## InMemory is not a relational database
-
-EF Core database providers do not have to be relational databases. InMemory is designed to be a general purpose database for testing, and is not designed to mimic a relational database.
-
-Some examples of this include:
-
-* InMemory will allow you to save data that would violate referential integrity constraints in a relational database.
-* If you use DefaultValueSql(string) for a property in your model, this is a relational database API and will have no effect when running against InMemory.
-* [Concurrency via Timestamp/row version](xref:core/modeling/concurrency#timestamprowversion) (`[Timestamp]` or `IsRowVersion`) is not supported. No [DbUpdateConcurrencyException](https://docs.microsoft.com/dotnet/api/microsoft.entityframeworkcore.dbupdateconcurrencyexception) will be thrown if an update is done using an old concurrency token.
-
-> [!TIP]  
-> For many test purposes these differences will not matter. However, if you want to test against something that behaves more like a true relational database, then consider using [SQLite in-memory mode](sqlite.md).
-
-## Example testing scenario
-
-Consider the following service that allows application code to perform some operations related to blogs. Internally it uses a `DbContext` that connects to a SQL Server database. It would be useful to swap this context to connect to an InMemory database so that we can write efficient tests for this service without having to modify the code, or do a lot of work to create a test double of the context.
-
-[!code-csharp[Main](../../../../samples/core/Miscellaneous/Testing/BusinessLogic/BlogService.cs)]
-
-## Get your context ready
-
-### Avoid configuring two database providers
-
-In your tests you are going to externally configure the context to use the InMemory provider. If you are configuring a database provider by overriding `OnConfiguring` in your context, then you need to add some conditional code to ensure that you only configure the database provider if one has not already been configured.
-
-[!code-csharp[Main](../../../../samples/core/Miscellaneous/Testing/BusinessLogic/BloggingContext.cs#OnConfiguring)]
-
-> [!TIP]  
-> If you are using ASP.NET Core, then you should not need this code since your database provider is already configured outside of the context (in Startup.cs).
-
-### Add a constructor for testing
-
-The simplest way to enable testing against a different database is to modify your context to expose a constructor that accepts a `DbContextOptions<TContext>`.
-
-[!code-csharp[Main](../../../../samples/core/Miscellaneous/Testing/BusinessLogic/BloggingContext.cs#Constructors)]
-
-> [!TIP]  
-> `DbContextOptions<TContext>` tells the context all of its settings, such as which database to connect to. This is the same object that is built by running the OnConfiguring method in your context.
-
-## Writing tests
-
-The key to testing with this provider is the ability to tell the context to use the InMemory provider, and control the scope of the in-memory database. Typically you want a clean database for each test method.
-
-Here is an example of a test class that uses the InMemory database. Each test method specifies a unique database name, meaning each method has its own InMemory database.
-
->[!TIP]
-> To use the `.UseInMemoryDatabase()` extension method, reference the NuGet package [Microsoft.EntityFrameworkCore.InMemory](https://www.nuget.org/packages/Microsoft.EntityFrameworkCore.InMemory/).
-
-[!code-csharp[Main](../../../../samples/core/Miscellaneous/Testing/TestProject/InMemory/BlogServiceTests.cs)]
+The information on this page now lives in other locations:
+* See [Testing code that uses EF Core](xref:core/miscellaneous/testing/index) for general information on testing with the in-memory database.
+* See [Sample showing how to test applications that use EF Core](xref:core/miscellaneous/testing/testing-sample) for a sample using the in-memory database.
+* See [The in-memory database provider](xref:core/providers/in-memory/index) for general information about the EF in-memory database.

entity-framework/core/miscellaneous/testing/index.md

@@ -1,8 +1,8 @@
 ---
-title: Test components using EF Core - EF Core
+title: Testing code that uses EF Core - EF Core
 description: Different approaches to testing applications that use EF Core
 author: ajcvickers
-ms.date: 03/23/2020
+ms.date: 04/22/2020
 uid: core/miscellaneous/testing/index
 ---
 # Testing code that uses EF Core
@@ -14,17 +14,20 @@ Testing code that accesses a database requires either:
 
 This document outlines the trade offs involved in each of these choices and shows how EF Core can be used with each approach.  
 
+> [!TIP]
+> See [EF Core testing sample](xref:core/miscellaneous/testing/testing-sample) for code demonstrating the concepts introduced here. 
+
 ## All database providers are not equal
 
 It is very important to understand that EF Core is not designed to abstract every aspect of the underlying database system.
 Instead, EF Core is a common set of patterns and concepts that can be used with any database system.
 EF Core database providers then layer database-specific behavior and functionality over this common framework.
 This allows each database system to do what it does best while still maintaining commonality, where appropriate, with other database systems. 
 
-Fundamentally, this means that switching out the database provider will change EF Core behavior and the application can't be expected to function correctly unless it explicitly accounts for all differences in behavior.
+Fundamentally, this means that switching out the database provider will change EF Core behavior and the application can't be expected to function correctly unless it explicitly accounts for any differences in behavior.
 That being said, in many cases doing this will work because there is a high degree of commonality amongst relational databases.
 This is good and bad.
-Good because moving between databases can be relatively easy.
+Good because moving between database systems can be relatively easy.
 Bad because it can give a false sense of security if the application is not fully tested against the new database system.  
 
 ## Approach 1: Production database system
@@ -42,7 +45,7 @@ That being said, it is still wise to run tests against SQL Azure itself before g
 ### LocalDb 
 
 All the major database systems have some form of "Developer Edition" for local testing.
-SQL Server also also has a feature called [LocalDb](/sql/database-engine/configure-windows/sql-server-express-localdb?view=sql-server-ver15).
+SQL Server also has a feature called [LocalDb](/sql/database-engine/configure-windows/sql-server-express-localdb?view=sql-server-ver15).
 The primary advantage of LocalDb is that it spins up the database instance on demand.
 This avoids having a database service running on your machine even when you're not running tests.
 
@@ -52,7 +55,9 @@ LocalDb is not without it's issues:
 * It can cause lag on first test run as the service is spun up.
 
 Personally, I've never found it a problem having a database service running on my dev machine and I would generally recommend using Developer Edition instead.
-However, it may be appropriate for some people, especially on less powerful dev machines.  
+However, LocalDb may be appropriate for some people, especially on less powerful dev machines.
+
+Running SQL Server (or any other database system) in a Docker container (os similar) is another way to avoid running the database system directly on your development machine.  
 
 ## Approach 2: SQLite
 
@@ -103,4 +108,4 @@ Instead we use the in-memory database when unit testing something that uses DbCo
 In this case using the in-memory database is appropriate because the test is not dependent on database behavior.
 Just don't do this to test actual database queries or updates.   
 
-See [Testing with the in-memory provider](xref:core/miscellaneous/testing/in-memory) for EF Core specific guidance on using the in-memory database for unit testing.
+The [EF Core testing sample](xref:core/miscellaneous/testing/testing-sample) demonstrates tests using the EF in-memory database, as well as SQL Server and SQLite. 

entity-framework/core/miscellaneous/testing/sharing-databases.md

@@ -0,0 +1,103 @@
+---
+title: Sharing databases between tests - EF Core
+description: Sample showing how to share a database between multiple tests
+author: ajcvickers
+ms.date: 04/25/2020
+uid: core/miscellaneous/testing/sharing-databases
+---
+
+# Sharing databases between tests
+
+The [EF Core testing sample](xref:core/miscellaneous/testing/testing-sample) showed how to test applications against different database systems.
+For that sample, each test created a new database.
+This is a good pattern when using SQLite or the EF in-memory database, but it can involve significant overhead when using other database systems.
+
+This sample builds on the previous sample by moving database creation into a test fixture.
+This allows a single SQL Server database to be created and seeded only once for all tests.
+
+> [!TIP]
+> Make sure to work through the [EF Core testing sample](xref:core/miscellaneous/testing/testing-sample) before continuing here.
+
+It's not difficult to write multiple tests against the same database.
+The trick is doing it in a way that the tests don't trip over each other as they run.
+This requires understanding:
+* How to safely share objects between tests
+* When the test framework runs tests in parallel
+* How to keep the database in a clean state for every test  
+
+## The Fixture
+
+We will use a test fixture for sharing objects between tests.
+The [XUnit documentation](https://xunit.net/docs/shared-context.html) states that a fixture should be used "when you want to create a single test context and share it among all the tests in the class, and have it cleaned up after all the tests in the class have finished."
+
+> [!TIP]
+> This sample uses [XUnit](https://xunit.net/), but similar concepts exist in other testing frameworks, including [NUnit](https://nunit.org/).
+
+This means that we need to move database creation and seeding to a fixture class.
+Here's what it looks like:
+
+[!code-csharp[SharedDatabaseFixture](../../../../samples/core/Miscellaneous/Testing/ItemsWebApi/SharedDatabaseTests/SharedDatabaseFixture.cs?name=SharedDatabaseFixture)]
+
+For now, notice how the constructor:
+* Creates a single database connection for the lifetime of the fixture
+* Creates and seeds that database by calling the `Seed` method 
+
+Ignore the locking for now; we will come back to it later.
+
+> [!TIP]
+> The creation and seeding code does not need to be async.
+> Making it async will complicate the code and will not improve performance or throughput of tests.
+
+The database is created by first deleting any existing database and then creating a new database.
+This ensures that the database matches the current EF model even if it has been changed since the last test run.
+
+> [!TIP]
+> It can be faster to "clean" the existing database using something like [respawn](https://jimmybogard.com/tag/respawn/) rather than re-create it each time.
+> However, care must be taken to ensure that the database schema is up-to-date with the EF model when doing this.
+
+The database connection is disposed when the fixture is disposed.
+You may also consider deleting the test database at this point.
+However, this will require additional locking and reference counting if the fixture is being shared by multiple test classes.
+Also, it is often useful to have the test database still available for debugging failed tests.  
+
+## Using the fixture
+
+XUnit has a common pattern for associating a test fixture with a class of tests:
+
+[!code-csharp[UsingTheFixture](../../../../samples/core/Miscellaneous/Testing/ItemsWebApi/SharedDatabaseTests/SharedDatabaseTest.cs?name=UsingTheFixture)]
+
+XUnit will now create a single fixture instance and pass it to each instance pf the test class.
+(Remember that XUnit creates a new test class instance every time it runs a test.)
+This means that the database will be created and seeded once and then each test will use this database.
+
+Note that tests within a single class will not be run in parallel.
+This means it is safe for each test to use the same database connection, even though the `DbConnection` object is not thread-safe.
+
+## Maintaining database state
+
+Tests often need to mutate the test data with inserts, updates, and deletes.
+But these changes will then impact other tests which are expecting a clean, seeded database.
+
+This can be dealt with by running mutating tests inside a transaction.
+For example:
+
+[!code-csharp[CanAddItem](../../../../samples/core/Miscellaneous/Testing/ItemsWebApi/SharedDatabaseTests/SharedDatabaseTest.cs?name=CanAddItem)]
+
+Notice that the transaction is created as the test starts and disposed when it is finished.
+Disposing the transaction causes it to be rolled back, so none of the changes will be seen by other tests.
+
+The helper method for creating a context (see the fixture code above) accepts this transaction and opts the DbContext into using it. 
+
+## Sharing the fixture
+
+You may have noticed locking code around database creation and seeding.
+This is not needed for this sample since only one class of tests use the fixture, so only a single fixture instance is created.
+
+However, you may want to use the same fixture with multiple classes of tests.
+XUnit will create one fixture instance for each of these classes.
+These may be used by different threads running tests in parallel.
+Therefore, it is important to have appropriate locking to ensure only one thread does the database creation and seeding.
+
+> [!TIP]
+> A simple `lock` is fine here.
+> There is no need to attempt anything more complex here, such as lock-free patterns.

entity-framework/core/miscellaneous/testing/sqlite.md

@@ -1,49 +1,42 @@
 ---
 title: Testing with SQLite - EF Core
-author: rowanmiller
-ms.date: 10/27/2016
-ms.assetid: 7a2b75e2-1875-4487-9877-feff0651b5a6
+description: Using SQLite to test an EF Core application
+author: ajcvickers
+ms.date: 04/24/2020
 uid: core/miscellaneous/testing/sqlite
 ---
 
-# Testing with SQLite
+# Using SQLite to test an EF Core application
 
-SQLite has an in-memory mode that allows you to use SQLite to write tests against a relational database, without the overhead of actual database operations.
+> [!WARNING]
+> Using SQLite can be an effective way to test an EF Core application.
+> However, problems can arise where SQLite behaves differently from other database systems. 
+> See [Testing code that uses EF Core](xref:core/miscellaneous/testing/index) for a discussion of the issues and trade-offs.  
 
-> [!TIP]  
-> You can view this article's [sample](https://github.com/dotnet/EntityFramework.Docs/tree/master/samples/core/Miscellaneous/Testing) on GitHub
+This document builds uses on the concepts introduced in [Sample showing how to test applications that use EF Core](xref:core/miscellaneous/testing/testing-sample).
+The code examples shown here come from this sample.
 
-## Example testing scenario
+## Using SQLite in-memory databases
 
-Consider the following service that allows application code to perform some operations related to blogs. Internally it uses a `DbContext` that connects to a SQL Server database. It would be useful to swap this context to connect to an in-memory SQLite database so that we can write efficient tests for this service without having to modify the code, or do a lot of work to create a test double of the context.
+Normally, SQLite creates databases as simple files and accesses the file in-process with your application.
+This is very fast, especially when using a fast [SSD](https://en.wikipedia.org/wiki/Solid-state_drive). 
 
-[!code-csharp[Main](../../../../samples/core/Miscellaneous/Testing/BusinessLogic/BlogService.cs)]
+SQLite can also uses databases created purely in-memory.
+This is easy to use with EF Core as long as you understand the in-memory database lifetime:
+* The database is created when the connection to it is opened
+* The database is deleted when the connection to it is closed
 
-## Get your context ready
+EF Core will use an already open connection when given one, and will never attempt to close it.
+So the key to using EF Core with an in-memory SQLite database is to open the connection before passing it to EF.  
 
-### Avoid configuring two database providers
+The [sample](xref:core/miscellaneous/testing/testing-sample) achieves this with the following code:
 
-In your tests you are going to externally configure the context to use the InMemory provider. If you are configuring a database provider by overriding `OnConfiguring` in your context, then you need to add some conditional code to ensure that you only configure the database provider if one has not already been configured.
+[!code-csharp[SqliteInMemory](../../../../samples/core/Miscellaneous/Testing/ItemsWebApi/Tests/SqliteInMemoryItemsControllerTest.cs?name=SqliteInMemory)]
 
-> [!TIP]  
-> If you are using ASP.NET Core, then you should not need this code since your database provider is configured outside of the context (in Startup.cs).
+Notice:
+* The `CreateInMemoryDatabase` method creates a SQLite in-memory database and opens the connection to it.
+* The created `DbConnection` is extracted from the `ContextOptions` and saved.
+* The connection is disposed when the test is disposed so that resources are not leaked. 
 
-[!code-csharp[Main](../../../../samples/core/Miscellaneous/Testing/BusinessLogic/BloggingContext.cs#OnConfiguring)]
-
-### Add a constructor for testing
-
-The simplest way to enable testing against a different database is to modify your context to expose a constructor that accepts a `DbContextOptions<TContext>`.
-
-[!code-csharp[Main](../../../../samples/core/Miscellaneous/Testing/BusinessLogic/BloggingContext.cs#Constructors)]
-
-> [!TIP]  
-> `DbContextOptions<TContext>` tells the context all of its settings, such as which database to connect to. This is the same object that is built by running the OnConfiguring method in your context.
-
-## Writing tests
-
-The key to testing with this provider is the ability to tell the context to use SQLite, and control the scope of the in-memory database. The scope of the database is controlled by opening and closing the connection. The database is scoped to the duration that the connection is open. Typically you want a clean database for each test method.
-
->[!TIP]
-> To use `SqliteConnection()` and the `.UseSqlite()` extension method, reference the NuGet package [Microsoft.EntityFrameworkCore.Sqlite](https://www.nuget.org/packages/Microsoft.EntityFrameworkCore.Sqlite/).
-
-[!code-csharp[Main](../../../../samples/core/Miscellaneous/Testing/TestProject/SQLite/BlogServiceTests.cs)]
+> [!NOTE]
+> [Issue #16103](https://github.com/dotnet/efcore/issues/16103) is tracking ways to make this connection management easier.