TiCDC panicked when handling update events with enable-old-value=false

[zhaoxinyu]

What did you do?

1. set enable-old-value = false in changefeed configuration file.
2. create a table with generated columns
3. execute update DMLs in upstream TiDB

What did you expect to see?

TiCDC can properly handle this case and panic does not happen.

What did you see instead?

TiCDC panicked

Versions of the cluster

Upstream TiDB cluster version (execute SELECT tidb_version(); in a MySQL client):

5.4.1

Upstream TiKV version (execute tikv-server --version):

5.4.1

TiCDC version (execute cdc version):

5.4.1

[zhaoxinyu]

/assign @zhaoxinyu

[nongfushanquan]

/assign @hi-rustin

[nongfushanquan]

/unassign @zhaoxinyu

[Rustin170506]

You can reproduction with these steps:

1. start tidb and ticdc cluser
2. create a changefeed and disable the old value
3. create table

CREATE TABLE person2 (
    name VARCHAR(255) NOT NULL PRIMARY KEY,
    address_info JSON,
    city VARCHAR(64) AS (JSON_UNQUOTE(JSON_EXTRACT(address_info, '$.city'))),
    KEY (city)
);

4. insert data

INSERT INTO person2 (name, address_info) VALUES ('Morgan', JSON_OBJECT('city', 'Canada'));

5. Continuously updated data

#!/bin/bash

for ((i = 1; i <= 1000000000000; i++)); do
	mysql --comments --host 127.0.0.1 --port 4000 -u root --password="" <update.sql
done

use test;
UPDATE person2 set name='Morgan' where name='test';
UPDATE person2 set name='test' where name='Morgan';

6. Continuously DDL

#!/bin/bash

for ((i = 1; i <= 1000000000000; i++)); do
	mysql --comments --host 127.0.0.1 --port 4000 -u root --password="" <add.sql
done

use test;
ALTER TABLE person2 ADD c1 int;
ALTER TABLE person2 DROP COLUMN c1;

7. TiCDC Panic

[Rustin170506]

The reason is:

tiflow/cdc/entry/mounter.go

Line 287 in 97f6592

if !exist && !fillWithDefaultValue {

When we execute the add column DDL, the data not exist and fillWithDefaultValue is false.

[nongfushanquan]

#6403

[nongfushanquan]

/close

[ti-chi-bot]

@nongfushanquan: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[MimeLyc]

/remove-label affects-6.2

[seiya-annie]

/found customer