Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BitSail][Connector] Support CDC Source Connector #160

Open
garyli1019 opened this issue Nov 10, 2022 · 0 comments
Open

[BitSail][Connector] Support CDC Source Connector #160

garyli1019 opened this issue Nov 10, 2022 · 0 comments
Assignees
@garyli1019
Labels
connector requirement New Connector requiremnet

Comments

@garyli1019
Copy link
Collaborator

garyli1019 commented Nov 10, 2022 •
edited
Loading

We would like to support a CDC source connector and provide a complete CDC ingestion and integration solution to our user.

We will break this task into 5 stage:

  1. Migrate the legacy Mysql JDBC connector to the V1 interface and refactoring the implementation to include watermark
    Tracking task: Migrate mysql connector to V1 interface and refactoring #199
  2. Support Debezium connector to read mysql binlog source.
  3. Implement the snapshot and binlog swap algorithm according to the Netflix DBlog paper
  4. Extend this solution to other database, including oracle mongo postgres sqlserver.
  5. Binlog client performance improvement and advanced feature support(realtime schema evolution, whole database sync e.t.c).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@garyli1019 garyli1019

Labels
connector requirement New Connector requiremnet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@garyli1019