Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[debezium] Improve convertToBinary for RowDataDebeziumDeserializeSchema #2435

Merged
merged 4 commits into from
Dec 2, 2023
Merged

[debezium] Improve convertToBinary for RowDataDebeziumDeserializeSchema #2435

merged 4 commits into from
Dec 2, 2023

Conversation

gong
Copy link
Contributor

@gong gong commented Aug 28, 2023

dbzObj may be string when mysql cdc set with param 'debezium.binary.handling.mode' = 'base64'.
So add

if (dbzObj instanceof String) {
     String data = (String) dbzObj;
     return data.getBytes(StandardCharsets.UTF_8);
  }

@gong
Copy link
Contributor Author

gong commented Aug 28, 2023

@ruanhang1993 Hello, please help review it.

@gong
Copy link
Contributor Author

gong commented Aug 29, 2023

The pipeline is TiDBE2eITCase exception. It is not relative with this PR.

@gong gong force-pushed the flink-cdc-improve-convertToBinary branch from 7d80b63 to 6552a59 Compare December 1, 2023 08:02
@gong
Copy link
Contributor Author

gong commented Dec 1, 2023

@leonardBang PTAL

@leonardBang leonardBang self-requested a review December 1, 2023 08:25
leonardBang
leonardBang reviewed Dec 1, 2023
View reviewed changes
Copy link
Contributor

@leonardBang leonardBang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gong for the contribution, the change looks good to me, could you add a test for this change?

@gong
Copy link
Contributor Author

gong commented Dec 1, 2023

Thanks @gong for the contribution, the change looks good to me, could you add a test for this change?
@leonardBang ok. I will add UT.

@gong
Copy link
Contributor Author

gong commented Dec 1, 2023

@leonardBang I add an UT.

@gong gong requested a review from leonardBang December 1, 2023 09:52
@gong gong force-pushed the flink-cdc-improve-convertToBinary branch from acef89b to 479d145 Compare December 2, 2023 05:07
leonardBang
leonardBang reviewed Dec 2, 2023
View reviewed changes
Copy link
Contributor

@leonardBang leonardBang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gong for the explanation and ITCase, the code change part looks good to me.

Could you add parameter 'debezium.binary.handling.mode' = 'base64' to cdc connector option part of docs ? A short paragraph to describe the user case and parameter is recommended

@gong
Copy link
Contributor Author

gong commented Dec 2, 2023

Thanks @gong for the explanation and ITCase, the code change part looks good to me.

Could you add parameter 'debezium.binary.handling.mode' = 'base64' to cdc connector option part of docs ? A short paragraph to describe the user case and parameter is recommended

@leonardBang Ok. I will add docs to decribe the user case

@github-actions github-actions bot added the docs Improvements or additions to documentation label Dec 2, 2023
@gong gong requested a review from leonardBang December 2, 2023 06:32
leonardBang
leonardBang approved these changes Dec 2, 2023
View reviewed changes
Copy link
Contributor

@leonardBang leonardBang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @gong for the update, LGTM . Will merge this PR once the CI green

@leonardBang
Copy link
Contributor

The failed test is not related to current PR, merging.

@leonardBang leonardBang merged commit 5f31834 into apache:master Dec 2, 2023
15 of 17 checks passed
@leonardBang leonardBang added this to the V3.0.0 milestone Dec 2, 2023
lvyanquan added a commit to lvyanquan0624/flink-cdc-connectors that referenced this pull request Dec 3, 2023
@lvyanquan
Merge branch 'schema' into e2eTest
1fee062 
* schema:
  [pipeline-common] use column name to judge whether a column is existed in a specific schema.
  [cdc-composer] Add common and runtime JAR when submit (apache#2795)
  [cdc-connector][debezium] Supports conversion  from binary bytes to base64 encoded string (apache#2435)
  [cdc-connector][mysql] MysqlTableSource return metadata in a fixed order (apache#2578)
  [cdc-connector][mysql] Fix snapshot fetch size conf does not take effect (apache#2766)
ChaomingZhangCN pushed a commit to ChaomingZhangCN/flink-cdc that referenced this pull request Jan 13, 2025
@gong
[cdc-connector][debezium] Supports conversion from binary bytes to ba…
24a4210 
…se64 encoded string (apache#2435)

This closes apache#2435.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@leonardBang leonardBang leonardBang approved these changes

Assignees

@gong gong

Labels
docs Improvements or additions to documentation mysql-cdc-connector
Projects
None yet
Milestone
V3.0.0
Development

Successfully merging this pull request may close these issues.
2 participants
@gong @leonardBang