Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rename tables trigger error and add Ignore-sql configured not effective, unable to skip the rename ddl #9531

Closed
Chad20N13 opened this issue Aug 9, 2023 · 0 comments · Fixed by #9549
Assignees
Labels
affects-6.5 affects-7.1 affects-7.3 area/ticdc Issues or PRs related to TiCDC. severity/moderate type/bug The issue is confirmed as a bug.

Comments

@Chad20N13
Copy link

What did you do?

1、changefeed configuration:
[filter]
rules = ['cdctest.ta', 'cdctest.tb', 'cdctest.tc', 'cdctest.td']

2、 rename tables on upstream:

rename table cdctest.tc to cdctest.tc_bak,cdctest.td to cdctest.tc,cdctest.tc_bak to cdctest.td;

3、cdc error:

[
{
"id": "test-ticdc-tatb-task",
"namespace": "default",
"summary": {
"state": "error",
"tso": 443433743909650434,
"checkpoint": "2023-08-09 15:18:10.378",
"error": {
"time": "2023-08-09T15:29:16.383357628+08:00",
"addr": "172.16.201.141:8335",
"code": "CDC:ErrSnapshotTableNotFound",
"message": "[CDC:ErrSnapshotTableNotFound]table 111 not found in schema snapshot"
}
}
}
]

4、add ignore-sql configuration :
ignore-sql = ["^rename"]
or
ignore-sql = ["rename table cdctest.tc to cdctest.tc_bak,cdctest.td to cdctest.tc,cdctest.tc_bak to cdctest.td"]

5、update changefeed :

/home/tidb/.tiup/components/ctl/v6.5.3/cdc cli changefeed update --pd=http://xxxxx:4199 --sink-uri="mysql://root:hxxxx@172.xxxx1:4651/" --changefeed-id="test-ticdc-tatb-task" --config ticdc_task_001.toml

6、changefeed can't resume

[
{
"id": "test-ticdc-tatb-task",
"namespace": "default",
"summary": {
"state": "error",
"tso": 443433743909650434,
"checkpoint": "2023-08-09 15:18:10.378",
"error": {
"time": "2023-08-09T16:21:42.632679465+08:00",
"addr": "172.16.201.141:8335",
"code": "CDC:ErrSnapshotTableNotFound",
"message": "[CDC:ErrSnapshotTableNotFound]table 111 not found in schema snapshot"
}
}
}
]

What did you expect to see?

skip the rename ddl after add Ignore-sql configuration

What did you see instead?

[
{
"id": "test-ticdc-tatb-task",
"namespace": "default",
"summary": {
"state": "error",
"tso": 443433743909650434,
"checkpoint": "2023-08-09 15:18:10.378",
"error": {
"time": "2023-08-09T16:21:42.632679465+08:00",
"addr": "172.16.201.141:8335",
"code": "CDC:ErrSnapshotTableNotFound",
"message": "[CDC:ErrSnapshotTableNotFound]table 111 not found in schema snapshot"
}
}
}
]

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

MySQL [(none)]> SELECT tidb_version();
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_version()                                                                                                                                                                                                                                                                               |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Release Version: v6.5.3
Edition: Community
Git Commit Hash: 71e6696f020ccf42408976d4a82cafb95717a2e4
Git Branch: heads/refs/tags/v6.5.3
UTC Build Time: 2023-06-12 13:52:32
GoVersion: go1.19.9
Race Enabled: false
TiKV Min Version: 6.2.0-alpha
Check Table Before Drop: false
Store: tikv |
+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

Upstream TiKV version (execute tikv-server --version):

TiKV 
Release Version:   6.5.3
Edition:           Community
Git Commit Hash:   fd5f88a7fdda1bf70dcb0d239f60137110c54d46
Git Commit Branch: heads/refs/tags/v6.5.3
UTC Build Time:    2023-06-09 10:55:15
Rust Version:      rustc 1.67.0-nightly (96ddd32c4 2022-11-14)
Enable Features:   pprof-fp jemalloc mem-profiling portable sse test-engine-kv-rocksdb test-engine-raft-raft-engine cloud-aws cloud-gcp cloud-azure
Profile:           dist_release

TiCDC version (execute cdc version):

Release Version: v6.5.3
Git Commit Hash: 1e2f277f2e3d9b57b15db9a2a9b2c62832c071ca
Git Branch: heads/refs/tags/v6.5.3
UTC Build Time: 2023-06-05 10:19:02
Go Version: go version go1.19.9 linux/amd64
Failpoint Build: false
@Chad20N13 Chad20N13 added area/ticdc Issues or PRs related to TiCDC. type/bug The issue is confirmed as a bug. labels Aug 9, 2023
@asddongmen asddongmen self-assigned this Aug 11, 2023
ti-chi-bot bot pushed a commit that referenced this issue Aug 14, 2023
ti-chi-bot bot pushed a commit that referenced this issue Aug 16, 2023
3AceShowHand pushed a commit to 3AceShowHand/tiflow that referenced this issue Aug 29, 2023
ti-chi-bot bot pushed a commit that referenced this issue Sep 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.5 affects-7.1 affects-7.3 area/ticdc Issues or PRs related to TiCDC. severity/moderate type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants