test: add TiDB into the engine tests #3308

Mini256 · 2022-10-20T06:18:06Z

janpio · 2022-11-03T20:15:54Z

introspection-engine/introspection-engine-tests/tests/identify_version/mod.rs

@@ -215,7 +215,7 @@ async fn introspect_postgres_prisma2(api: &TestApi) -> TestResult {

 //Mysql

-#[test_connector(tags(Mysql))]
+#[test_connector(tags(Mysql), exclude(TiDB))]


What is the problem with this test? Is there an issue on TiDB side tracking this?

Because TiDB currently does not support the point type, related issue: pingcap/tidb#6347

For this reason, I added an additional test case for TiDB:

prisma-engines/introspection-engine/introspection-engine-tests/tests/identify_version/mod.rs

Line 352 in 8177d21

t.inject_custom("location json");

@janpio when doing the prisma documentation probably this can be tagged as an exception ? Any other suggestions ?

We do not know yet. Right now a database either supports our feature set, or it does not. We did not have the case where it does, but not a specific type. Depending on the overall story and situation this might be be handled with a comment in the docs and may be a database specific error message - but if there are more things that is probably not enough and we would need to handle this on a higher level (new connector, which explodes our complexity).

introspection-engine/introspection-engine-tests/tests/rpc_calls/get_metadata_command_tests.rs

janpio · 2022-11-03T20:17:31Z

introspection-engine/introspection-engine-tests/tests/tables/mysql.rs

@@ -4,7 +4,7 @@ use introspection_engine_tests::test_api::*;
 use sql_introspection_connector::SqlIntrospectionConnector;
 use url::Url;

-#[test_connector(tags(Mysql))]
+#[test_connector(tags(Mysql), exclude(TiDB))]


Not supported in TiDB? Does an issue exist?

It seems that there is a compatibility problem here, the name of the generated unique key in TiDB is not consistent with MySQL.

Expect: ---- model Test { id Int @id authorId Int @unique(map: "authorId") @default(autoincrement()) } ---- Actual: ---- model Test { id Int @id authorId Int @unique(map: "authorid") @default(autoincrement()) } ----

I have created an issue pingcap/tidb#39080, and push the enginers to take a look on it.

Ah, probably related to TiDB bein case-insensitive for column names by default. Interesting.

This issue has been fixed by @wxbty. @wxbty, thank you for your work.

Amazing. Do you have an issue or PR link? Just to connect the dots.

😢 We may miss this release cycle, this fix PR may have to wait until TiDB 6.5. Version 6.4 has been frozen earlier.

Shame. When will that be?

Is there a preview release or something? Happy to test with prerelease.

We will publish the Nightly version in the Docker image.

The Nightly version can pass the test smoothly. But I'm not sure whether you can accept the Nightly version in the prisma test.

According to the previous release frequency, TiDB will release a mirror version every one to two months.

https://docs.pingcap.com/tidb/dev/release-notes

We can totally put the nightly image in the test here, as long as it is stable enough for us to learn what we want to learn - where Prisma and TiDB do not work together well right now.

Before merging this, we of course would love to switch to a different image then. (If we switch to nightly, I would open another PR that forks of the branch with the nightly to replace it with the current stable/minor so we can also monitor what current users of TiDB would experience)

Can you please open another PR into this branch that replaces the docker container with the nightly version and removes this test file modification @Mini256?

introspection-engine/introspection-engine-tests/tests/tables/mysql.rs

janpio · 2022-11-03T20:19:27Z

migration-engine/migration-engine-tests/tests/existing_data/mod.rs

@@ -779,7 +779,7 @@ fn set_default_current_timestamp_on_existing_column_works(api: TestApi) {
 }

 // exclude: there is a cockroach-specific test. It's unexecutable there.
-#[test_connector(preview_features("referentialIntegrity"), exclude(CockroachDb))]
+#[test_connector(preview_features("referentialIntegrity"), exclude(CockroachDb, TiDB))]


Why does this not work on TiDB?

Test result:

called `Result::unwrap()` on an `Err` value: ConnectorErrorImpl { user_facing_error: None, message: Some("Unsupported modify column: this column has primary key flag"), source: None, context: SpanTrace [] } Unsupported modify column: this column has primary key flag

Reason:

Does not support modifying the Reorg-Data types on the primary key columns but supports modifying the Meta-Only types.

Doc: https://docs.pingcap.com/tidb/dev/sql-statement-modify-column#mysql-compatibility

migration-engine/migration-engine-tests/tests/migrations/advisory_locking.rs

janpio · 2022-11-03T20:20:20Z

migration-engine/migration-engine-tests/tests/migrations/id.rs

+// Ignoring TiDB, unsupported drop primary key when the table's pkIsHandle is true.
 // TODO: ignore because not possible on cockroachdb. We would need a multi-step process there.
-#[test_connector(exclude(Vitess), exclude(CockroachDb))]
+#[test_connector(exclude(Vitess), exclude(CockroachDb, TiDB))]


What does that mean exactly?
Is there an issue for this?

I did some investigation and I learned that this problem is also caused by TIDB does not support deleting the primary key on the cluster index table.

However, for some historical reasons, if the primary key type is integer (INT or BIGINT) and the table is cluster index table. The pkIsHandle flag of the table will be set to true, which means that the table will use the primary key as the row handle.

If we try to drop a primary key of integer type on a clustered table, it gives us the following error:

(8200, "Unsupported drop primary key when the table's pkIsHandle is true")

If we drop a primary key that is not of type integer, it will throw this error:

(8200, 'Unsupported drop primary key when the table is using clustered index')

Our R&D engineers are considering optimizing this error message.

Is there an issue for this work? Might be good to link there so we can notice when this is fixed one day.

migration-engine/migration-engine-tests/tests/migrations/id.rs

janpio · 2022-11-03T20:22:09Z

migration-engine/migration-engine-tests/tests/migrations/id.rs

+// Ignoring TiDB, we can't remove auto_increment without @@tidb_allow_remove_auto_inc enabled.
 // We test this separately on cockroachdb.
-#[test_connector(exclude(Sqlite, CockroachDb))]
+#[test_connector(exclude(Sqlite, CockroachDb, TiDB))]


Is there a link for this config option?
Are there alternative strategies to achieve the same outcome?

Link of this option: https://docs.pingcap.com/tidb/dev/system-variables#tidb_allow_remove_auto_inc-new-in-v2118-and-v304

🤔 Is there any way to enable this system variable before starting testing, so we can add an additional test case for testing.

I think for other databases this can be configured either via environment variables (in our docker-compose), or we need to create a customized Docker image that executes some SQL on startup: https://github.com/prisma/engine-images/blob/main/cockroach/22.1.0/Dockerfile#L2 + https://github.com/prisma/engine-images/blob/main/cockroach/22.1.0/prisma_init.sql But that is suuuuper unflexible and error prone - so better to avoid.

I don't think we have a global hook that can be configured via the test suite unfortunately.

migration-engine/migration-engine-tests/tests/migrations/mysql.rs

janpio · 2022-11-03T20:23:31Z

migration-engine/migration-engine-tests/tests/migrations/reset_tests.rs

@@ -58,7 +58,7 @@ fn reset_then_apply_with_migrations_directory_works(api: TestApi) {
        .assert_has_table("_prisma_migrations");
 }

-#[test_connector]
+#[test_connector(exclude(TiDB))]


Why does this not work for TiDB?

@Mini256 another one can you please respond ?

I tried to solve this problem, but I found that when this testcase executes to the line api.diagnose_migration_history(&dir).send_sync();, in the absence of _prisma_migrations tables in both TiDB and MySQL MySQL will not throw an error, while TiDB will cause the test to terminate due to the following error thrown.

called `Result::unwrap()` on an `Err` value: ConnectorErrorImpl { user_facing_error: None, message: Some("Schema change caused error: [schema:1146]Table 'reset_then_diagnostics_with_migrations_directory_works._prisma_migrations' doesn't exist"), source: None, context: SpanTrace [] } Schema change caused error: [schema:1146]Table 'reset_then_diagnostics_with_migrations_directory_works._prisma_migrations' doesn't exist

@janpio, Do you have any idea about this problem?

No details, but seems TiDB behaves a bit different which leads to an error where MySQL is fine. We'll need to figure out later if this is a problem on Prisma or TiDB side.

migration-engine/migration-engine-tests/tests/native_types/mysql.rs

query-engine/connector-test-kit-rs/query-engine-tests/tests/new/interactive_tx.rs

query-engine/connector-test-kit-rs/query-engine-tests/tests/new/occ.rs

...engine/connector-test-kit-rs/query-engine-tests/tests/new/ref_actions/on_delete/no_action.rs

...gine/connector-test-kit-rs/query-engine-tests/tests/new/ref_actions/on_delete/set_default.rs

query-engine/connector-test-kit-rs/query-engine-tests/tests/new/regressions/prisma_10935.rs

query-engine/connector-test-kit-rs/query-engine-tests/tests/new/regressions/prisma_7434.rs

janpio · 2022-11-03T20:28:07Z

...engine/connector-test-kit-rs/query-engine-tests/tests/queries/aggregation/group_by_having.rs

@@ -42,7 +42,7 @@ mod aggr_group_by_having {
                _sum { int }
              }
            }"#,
-            MongoDb(_) => vec![
+            MongoDb(_) | TiDB => vec![


Why in this case?

Also related to the default order issue: pingcap/tidb#37285

Note that the linked issue was resolved, but unclear if the underlying problem is now gone as well.

…xclude making_an_existing_id_field_autoincrement_works test

janpio · 2023-04-26T10:15:02Z

(Nice work you are doing here right now @Mini256!)

Mini256 · 2023-04-26T13:39:10Z

@janpio Thank you for your attention to the pull request.

We hope to continue promoting this PR to merge, and after this PR is merged, our team will also continue to optimize the user experience of developers using Prisma with TiDB.

In the next stage, we will promote our TiDB Serverless product so that developers can use HTAP capabilities to build their applications with just MySQL skills (Just like how we used TiDB to build OSSInsight).

The code commit history may look a bit lengthy, so let me briefly summarize the recent changes I have made:

Updated the Docker image to version 7.0.0
Restart the tests skipped due to insufficient support for foreign keys previously

But also I found some compatibility issues of naming, such as Create index doesn't rename the old index that has same columns as the new one pingcap/tidb#43267 (It doesn't prevent users from using related functions, but it may cause mismatches with some test case assertions. Currently, I am skipping these tests and will update them after fixing the issue.
Set the schema engine's testing to single-thread mode for TiDB

The migration-engine-tests perform GET_LOCK operations during the apply_migration stage, some tests may encounter errors due to lock conflicts.

But this issue only occurs when tidb-server is running separately (using mockkv to mock the underlying storage engine TiKV), and it can pass the test normally in a complete cluster, which means it will not affect user usage. I guess this may be due to some differences in the implementation of mockkv and TiKV, related issues: Unstable test expression TestGetLock pingcap/tidb#35155 (comment)

Mini256 · 2023-07-05T10:39:49Z

@janpio Updated the Docker image of TiDB to version 7.1.0. (Ready to merged)
_{TiDB 7.1.0 Release Note}

laplab · 2024-04-11T12:22:28Z

Thank you for submitting this PR, @Mini256!

We mentioned in our documentation that TiDB has a community-maintained driver adapter. This driver adapter already allows users to effortlessly query TiDB databases. Thus, we currently do not plan to merge this PR and prisma/prisma#14631.

janpio · 2024-04-23T18:24:52Z

...engine/connector-test-kit-rs/query-engine-tests/tests/queries/batch/in_selection_batching.rs

+    // Exclude TiDB, TiDB does not guarantee the order of results returned when the `ORDER BY`
+    // clause is not specified
+    #[connector_test(exclude(TiDB))]


Is there an issue for this?

Mini256 added 7 commits October 20, 2022 14:13

test: add tidb 6.3 docker

9ed409a

test: add test setup for tidb

2c04cd0

test: add tidb into the migration engine tests

4dba8b9

test: add tidb into the introspection engine tests

a4952f8

feat: add tidb into query engine tests

17709f5

fix ci

95504db

fix ci

49ab8b8

Mini256 marked this pull request as ready for review October 31, 2022 01:36

Mini256 requested a review from a team as a code owner October 31, 2022 01:36

janpio mentioned this pull request Nov 3, 2022

test(integration): run integration test for TiDB prisma/prisma#14631

Closed