Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data lost when pause and resume changefeed while executing drop table DDL. #7682

Closed
asddongmen opened this issue Nov 22, 2022 · 1 comment · Fixed by #7683
Closed

Data lost when pause and resume changefeed while executing drop table DDL. #7682

asddongmen opened this issue Nov 22, 2022 · 1 comment · Fixed by #7683
Assignees
Labels
affects-5.1 affects-5.2 affects-5.3 affects-5.4 This bug affects the 5.4.x(LTS) versions. affects-6.0 affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.2 affects-6.3 affects-6.4 area/ticdc Issues or PRs related to TiCDC. component/replica-model Replication model component. severity/major type/bug The issue is confirmed as a bug.

Comments

@asddongmen
Copy link
Contributor

What did you do?

All steps are in this integration case: https://github.com/pingcap/tiflow/blob/master/tests/integration_tests/batch_update_to_no_batch/run.sh

  1. create changefeed c1.
  2. drop table t1 in upstream
  3. recover table t1 in upstream
  4. insert a row r1 into t1
  5. pause c1
  6. resume c1

What did you expect to see?

row r1 can be replicated to downstream

What did you see instead?

there is no r1 in downstream

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

(paste TiDB cluster version here)

Upstream TiKV version (execute tikv-server --version):

(paste TiKV version here)

TiCDC version (execute cdc version):

master
@asddongmen
Copy link
Contributor Author

asddongmen commented Nov 22, 2022

When a changefeed is paused while there is a DDL executing, DDL's executing would be canceld, and ddlEventCache would not be clean. While changefeed resume in auto retring, the DDL in ddlEventCache will be sent to downstream directly but not be applied to the schema of the changefeed.
So, changefeed will loss the tables, which lead to data lost.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-5.1 affects-5.2 affects-5.3 affects-5.4 This bug affects the 5.4.x(LTS) versions. affects-6.0 affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.2 affects-6.3 affects-6.4 area/ticdc Issues or PRs related to TiCDC. component/replica-model Replication model component. severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
2 participants