Guideline for Microsoft.EntityFrameworkCore.Relational.Specification.Tests

[guscuate]

Hello Team,
We would like to run your test suite into our data provider (MySQL Connector/Net), is there any documentation or guide about what is the right way to use Microsoft.EntityFrameworkCore.Relational.Specification.Tests package?

Regards.

[ErikEJ]

Maybe look at how other providers use the package?

[roji]

@guscuate great that you're looking at the specification tests! They're a very useful way of making sure your provider works well.

There's no documentation for using them, but it should be quite straightforward:

1. Create your own test project (i.e. MySql.EntityFrameworkCore.FunctionalTests)
2. Pick a test class from Microsoft.EntityFrameworkCore.Relational.Specification.Tests, and start extend it in your project. For example, you could start with NorthwindWhereQueryRelationalTestBase, which you'd extend from NorthwindWhereQueryMySqlTest.
3. You'll have a bit of test infrastructure to implement first. For example, for NorthwindWhereQueryMySqlTest you'll have to implement NorthwindWhereQueryMySqlFixture, which will require a MySqlNorthwindTestStoreFactory, which would need a Northwind.sql script to seed the MySQL version of the Northwind database. I suggest keeping another EF Core provider's test suite open nearby, and simply following what it does (you can use the SQL Server provider's tests, or SQLite's or Npgsql's).
4. Once the infrastructure for the test class is done, you'll start seeing some green tests on it. You can investigate the failing tests, or temporarily skip them for later investigation. In this way you can add more and more test classes.
5. At some point, when you've extended most of the upstream test classes, you can also create MySqlComplianceTest, which extends RelationalComplianceTestBase. This has a test which will fail if your own test project doesn't extend an upstream test class - it's a great way to know whether your test suite is complete, and also whether we add a new test class on our side in a new version. You can also opt out of extending specific test classes if they're not ready (or not relevant).

I'd recommend giving the above a try - we'll be here to help with any concrete issues you come across!

PS #681 tracks adding documentation around this and other aspects of writing an EF Core provider.

[guscuate]

Thank you very much for your responses this is very useful!.

[ajcvickers]

Note from triage: put screenshot of this in docs. 😜

[guscuate]

Hi Roji, I saw you updated the documentation and you mentioned to create an Xunit project, is this strictly required or this can also work using a different test framework like Nunit?

[roji]

@guscuate the specification test suite itself is written using xunit, so you'll have to use xunit as well.

Note that this doesn't preclude you from also having an additional test project using nunit or anything else. However, for an EF provider, the specification test suite should generally be the most important testing resource - so it's worth using it even if it isn't your preferred unit testing framework.

FWIW I prefer nunit to xunit (and use nunit in Npgsql), but I use xunit in the Npgsql EF provider for the specification test suite.