Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redo log can only tolerate storage error for 6s #8089

Closed
fubinzh opened this issue Jan 17, 2023 · 2 comments · Fixed by #8181
Closed

Redo log can only tolerate storage error for 6s #8089

fubinzh opened this issue Jan 17, 2023 · 2 comments · Fixed by #8181
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. severity/moderate type/bug The issue is confirmed as a bug.

Comments

@fubinzh
Copy link

fubinzh commented Jan 17, 2023

What did you do?

  1. Create CDC changefeed with redo log
  2. Run workload
  3. Inject network partition to redo log storage for 10min

What did you expect to see?

CDC should be able to tolerate 10min for redo log storage access error

What did you see instead?

CDC currently only tolerates storage error for 6s:

img_v2_cb9f0ce1-2ad3-4092-af60-69d6e3ca6c3g

Versions of the cluster

bash-5.1# /cdc version
Release Version: v6.5.0-11-g3808d9732-dirty
Git Commit Hash: 3808d97
Git Branch: release-6.5
UTC Build Time: 2023-01-05 07:30:18
Go Version: go version go1.19.1 linux/amd64
Failpoint Build: false

@fubinzh fubinzh added area/ticdc Issues or PRs related to TiCDC. type/bug The issue is confirmed as a bug. labels Jan 17, 2023
@fubinzh
Copy link
Author

fubinzh commented Jan 17, 2023

/severity Moderate

@jebter
Copy link

jebter commented Jan 29, 2023

/severity moderate

@VelocityLight VelocityLight added the affects-6.5 This bug affects the 6.5.x(LTS) versions. label Jan 30, 2023
@CharlesCheung96 CharlesCheung96 added the affects-6.1 This bug affects the 6.1.x(LTS) versions. label Feb 6, 2023
CharlesCheung96 added a commit to ti-chi-bot/tiflow that referenced this issue Feb 13, 2023
CharlesCheung96 added a commit to ti-chi-bot/tiflow that referenced this issue Feb 13, 2023
CharlesCheung96 added a commit to ti-chi-bot/tiflow that referenced this issue Mar 27, 2023
CharlesCheung96 added a commit to ti-chi-bot/tiflow that referenced this issue Mar 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. severity/moderate type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants