Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Consider to restrict batch dml SQL size #8084

Closed
2 tasks done
amyangfei opened this issue Jan 16, 2023 · 5 comments
Closed
2 tasks done

Consider to restrict batch dml SQL size #8084

amyangfei opened this issue Jan 16, 2023 · 5 comments
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. severity/major type/bug The issue is confirmed as a bug.

Comments

@amyangfei
Copy link
Contributor

amyangfei commented Jan 16, 2023 •
edited
Loading

Is your feature request related to a problem?

ref to batch DML feature: #7653

The count of RowChanges is not checked in GenUpdateSQL, if the SQL is too large, downstream may return error or performance could have regression.

Describe the feature you'd like

  • Benchmark multi update SQL performance, with different batch sizes.
  • Add limit to row changes count or SQL size in one single batch update SQL.

Describe alternatives you've considered

No response

Teachability, Documentation, Adoption, Migration Strategy

No response
@amyangfei amyangfei added area/ticdc Issues or PRs related to TiCDC. type/feature Issues about a new feature labels Jan 16, 2023
@amyangfei amyangfei mentioned this issue Jan 16, 2023
Merged
@amyangfei
Copy link
Contributor Author

related to: #7960

@amyangfei amyangfei mentioned this issue Feb 8, 2023
Merged
ti-chi-bot pushed a commit that referenced this issue Feb 8, 2023
@amyangfei
sinkv2(ticdc): add max-multi-update-row config in mysql sink (#8191)
1ec6426 
ref #8084
@ti-chi-bot ti-chi-bot mentioned this issue Feb 8, 2023
Merged
@nongfushanquan nongfushanquan mentioned this issue Feb 13, 2023
Merged
14 tasks
@nongfushanquan
Copy link
Contributor

/label bug

@ti-chi-bot
Copy link
Member

@nongfushanquan: The label(s) bug cannot be applied. These labels are supported: duplicate, bug-from-internal-test, bug-from-user, affects-4.0, affects-5.0, affects-5.1, affects-5.2, affects-5.3, affects-5.4, affects-6.0, affects-6.1, affects-6.2, affects-6.3, affects-6.4, affects-6.5, affects-6.6, may-affects-4.0, may-affects-5.0, may-affects-5.1, may-affects-5.2, may-affects-5.3, may-affects-5.4, may-affects-6.0, may-affects-6.1, may-affects-6.2, may-affects-6.3, may-affects-6.4, may-affects-6.5, may-affects-6.6, needs-cherry-pick-release-4.0, needs-cherry-pick-release-5.0, needs-cherry-pick-release-5.1, needs-cherry-pick-release-5.2, needs-cherry-pick-release-5.3, needs-cherry-pick-release-5.4, needs-cherry-pick-release-6.0, needs-cherry-pick-release-6.1, needs-cherry-pick-release-6.2, needs-cherry-pick-release-6.3, needs-cherry-pick-release-6.4, needs-cherry-pick-release-6.5, needs-cherry-pick-release-6.6, question, release-blocker, wontfix, require-LGT1, require-LGT3.

In response to this:

/label bug

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@nongfushanquan
Copy link
Contributor

/label affects-6.5

@ti-chi-bot ti-chi-bot added the affects-6.5 This bug affects the 6.5.x(LTS) versions. label Feb 15, 2023
@amyangfei amyangfei added type/bug The issue is confirmed as a bug. severity/major labels Feb 15, 2023
ti-chi-bot added a commit that referenced this issue Feb 15, 2023
@ti-chi-bot
sinkv2(ticdc): add max-multi-update-row config in mysql sink (#8191) (#…
a1a198e 
…8204)

ref #8084
@amyangfei amyangfei mentioned this issue Apr 3, 2023
Merged
@nongfushanquan
Copy link
Contributor

/label affects-6.1

@ti-chi-bot ti-chi-bot added the affects-6.1 This bug affects the 6.1.x(LTS) versions. label Apr 12, 2023
ti-chi-bot pushed a commit that referenced this issue Apr 13, 2023
@amyangfei
cherry pick batch update optimization (#8069) (#8191) (#8452) (#8727)
95b5db3 
ref #8084
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
affects-6.1 This bug affects the 6.1.x(LTS) versions. affects-6.5 This bug affects the 6.5.x(LTS) versions. area/ticdc Issues or PRs related to TiCDC. severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@amyangfei @ti-chi-bot @nongfushanquan