Add runtime migration creation and application

[thromel]

Summary

Implements #37342: Allow creating and applying migrations at runtime without recompiling.

This adds support for creating and applying migrations at runtime using Roslyn compilation, enabling scenarios like .NET Aspire and containerized applications where recompilation isn't possible.

CLI Usage

# Standard update (existing behavior)
dotnet ef database update [migration]

# Create and apply a new migration in one step
dotnet ef database update MigrationName --add [--output-dir <DIR>] [--namespace <NS>] [--json]

The -o/--output-dir, -n/--namespace, and --json options require --add to be specified.

PowerShell Usage

# Standard update (existing behavior)
Update-Database [-Migration <migration>]

# Create and apply a new migration in one step
Update-Database -Migration MigrationName -Add [-OutputDir <DIR>] [-Namespace <NS>]

Architecture

Component	Purpose
IMigrationCompiler / CSharpMigrationCompiler	Internal: Roslyn-based compilation of scaffolded migrations
IMigrationsAssembly.AddMigrations(Assembly)	Registers dynamically compiled migrations
MigrationsOperations.AddAndApplyMigration()	Orchestrates scaffold → compile → register → apply workflow

Design Decisions

• Extends existing services: Uses IMigrationsScaffolder for scaffolding and IMigrator for applying, adding only the new IMigrationCompiler service
• AddMigrations(Assembly): Extended IMigrationsAssembly interface to accept additional assemblies containing runtime-compiled migrations
• Always persists to disk: Like AddMigration, files are always saved to enable source control and future recompilation
• No pending changes behavior: If no model changes are detected, applies any existing pending migrations without creating a new one
• Internal compiler API: IMigrationCompiler and CSharpMigrationCompiler are in the .Internal namespace as they require design work for public API
• Error handling with cleanup: If compilation or migration application fails, saved migration files are cleaned up to prevent orphans
• Thread safety: MigrationsAssembly uses locking to protect against race conditions when adding migrations concurrently

Workflow

User runs: dotnet ef database update InitialCreate --add
    │
    ▼
MigrationsOperations.AddAndApplyMigration()
    │
    ├─► Check for pending model changes
    │       └─► If none: apply existing migrations, return
    │
    ├─► IMigrationsScaffolder.ScaffoldMigration() - Generate code
    │
    ├─► try {
    │       ├─► IMigrationsScaffolder.Save() - Write files to disk
    │       ├─► IMigrationCompiler.CompileMigration() - Roslyn compile
    │       ├─► IMigrationsAssembly.AddMigrations() - Register migration
    │       └─► IMigrator.Migrate() - Apply to database
    │   } catch {
    │       └─► Clean up saved files on failure
    │   }
    │
    └─► Return migration files

Robustness Features

1. Exception handling with cleanup: AddAndApplyMigration wraps the save-compile-register-apply chain in try-catch, deleting saved files on failure to prevent orphans
2. Context disposal on validation failure: PrepareForMigration ensures the DbContext is disposed if validation or service building fails
3. Thread-safe migration registration: MigrationsAssembly uses locking to protect shared state (migrations dictionary, model snapshot, additional assemblies list)

Limitations

• Requires dynamic code generation (incompatible with NativeAOT) - marked with [RequiresDynamicCode]
• C# only (no VB.NET/F# support)

Test plan

• Unit tests for CSharpMigrationCompiler
• Unit tests for MigrationsOperations.AddAndApplyMigration
• Integration tests in RuntimeMigrationTestBase (SQLite and SQL Server implementations)
• Tests for validation (empty name, invalid characters)
• Tests for RemoveMigration with dynamically created migrations
• All existing EFCore.Design.Tests pass
• All existing EFCore.Relational.Tests pass

Fixes #37342

[thromel]

Note on SQL Server Integration Tests

The RuntimeMigrationSqlServerTest tests are marked with [SqlServerCondition(SqlServerCondition.IsNotAzureSql | SqlServerCondition.IsNotCI)] and are skipped in CI. This follows the same pattern used by MigrationsInfrastructureSqlServerTest.

Why these tests are skipped in CI:

• They require creating fresh databases dynamically for each test to properly test the migration flow from scratch
• The Helix CI environment has SQL Server available but with limited permissions configured for shared/pre-configured databases
• Tests that use SqlServerTestStore.CreateInitializedAsync with dynamic database names don't work in the CI SQL Server setup

Test coverage is still maintained:

• Core runtime migration logic is covered by unit tests in EFCore.Design.Tests (which run in CI)
• The SQL Server integration tests run locally for developers with SQL Server configured
• SQLite integration tests in EFCore.Sqlite.FunctionalTests also validate the end-to-end flow

This is consistent with how other complex migration infrastructure tests handle CI limitations.

[thromel]

Thank you for the thorough review @AndriySvyryd! I've addressed all your feedback:

1. SQL Server tests: Now using static database name "RuntimeMigrationTest" and removed EnsureDeleted() calls since CreateInitializedAsync already cleans the database.

2. CLI validation: Added validation that shows an error if -o or -n is used without --add.

3. Renamed to AddAndApplyMigration: Renamed CreateAndApplyMigration to AddAndApplyMigration and reordered parameters as suggested.

4. Extracted common validation: Created ValidateMigrationName() and ValidateMigrationNameNotContextName() helper methods that are shared by both AddMigration and AddAndApplyMigration.

5. Added EnsureMigrationsAssembly call: Now calling EnsureMigrationsAssembly in AddAndApplyMigration.

6. Removed unnecessary IDesignTimeModel registration: Confirmed there's no duplicate registration.

7. Merged IDynamicMigrationsAssembly into IMigrationsAssembly: Added AddMigrations(Assembly) method to IMigrationsAssembly and deleted IDynamicMigrationsAssembly, DynamicMigrationsAssembly, and their tests.

8. Replaced CompiledMigration with AddMigrations(Assembly): IMigrationCompiler.CompileMigration now returns Assembly directly, and RuntimeMigrationService uses _migrationsAssembly.AddMigrations() to register compiled migrations. Removed the CompiledMigration class.

All tests pass locally (SQLite: 26 tests, SQL Server: 7 tests, CSharpMigrationCompiler: 4 tests, MigrationsOperations: 2 tests).

[AndriySvyryd]

Practical Risk Assessment

The race is unlikely because:

• AddMigrations is only called during design-time database update --auto operations
• It's not expected to be called concurrently with Migrations access
• The scenario requires very specific timing

I agree that it's very unlikely, keep NonCapturingLazyInitializer and remove the lock from AddMigrations

[AndriySvyryd]

You should be able to call Fixture.ReseedAsync() instead.

You can also call it from IAsyncLifetime.InitializeAsync to avoid repeating the call in each method

[AndriySvyryd]

Sorry, I forgot that ReseedAsync also creates tables. Change InitializeAsync to

{
    using var context = CreateContext();
    return Fixture.TestStore.CleanAsync(context, createTables: false);
}

Then add the bool createTables = true parameter to TestStore.CleanAsync, SqlServerDatabaseFacadeExtensions.EnsureClean, SqliteDatabaseFacadeTestExtensions.EnsureClean and RelationalDatabaseCleaner.Clean

[AndriySvyryd]

Why did you have to specify false here?

[AndriySvyryd]

Re: UsePooling => false in RuntimeMigrationFixtureBase

Connection pooling is disabled because these tests dynamically alter the database schema (creating/dropping tables, applying/reverting migrations). With pooling enabled, pooled connections might hold stale schema information or cached state that conflicts with the schema changes made during migration operations.
However, if you think pooling should work fine for these tests, I can remove the override and test it.

This property controls DbContext pooling, not connection pooling, so leaving the default value (true) should be fine for these tests

[AndriySvyryd]

I see now, if you use context pooling then the tests will get a context instance that still contains the runtime migrations assembly from the previous test, making it fail in most cases. So in this case, disabling DbContext pooling is the right choice

[thromel]

Re: UsePooling => false in RuntimeMigrationFixtureBase

Connection pooling is disabled because these tests dynamically alter the database schema (creating/dropping tables, applying/reverting migrations). With pooling enabled, pooled connections might hold stale schema information or cached state that conflicts with the schema changes made during migration operations.

However, if you think pooling should work fine for these tests, I can remove the override and test it.

[thromel]

Addressed the remaining unresolved comments:

1. CSharpMigrationCompiler - Simplified GetMetadataReferences to use cached references from loaded assemblies plus context assembly. Removed explicit Assembly.Load() calls since assemblies should already be loaded by that point.

2. MigrationsOperations - Removed duplicate name validation from AddMigration and AddAndApplyMigration since PrepareForMigration already validates.

3. UsePooling - Removed the UsePooling => false override since it controls DbContext pooling, not connection pooling.

Commit: e7b6329

[AndriySvyryd]

Ok, then remove this check since it's already checked before this call