Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][transforms-v2] Support append only stream from cdc source #7763

Merged
merged 2 commits into from
Oct 14, 2024

Conversation

hawk9821
Copy link
Contributor

@hawk9821 hawk9821 commented Sep 28, 2024

Purpose of this pull request

issues: #7756
Add a new transform which can transform cdc row to append only row that contains the cdc rowkind or other needed info.

Example:
CDC row: +D 1, test1, test2
new Row: +I 1,test1,test2,DELETE

Does this PR introduce any user-facing change?

How was this patch tested?

UT: CdcRowTransformFactoryTest CdcRowTransformTest
ETE: TestCdcRowTransformIT

Check list

@cobolbaby
Copy link

Is it possible to use the current SQL transform to turn one record into multiple records?

@Hisoka-X
Copy link
Member

cc @dailai and @hailin0

@hawk9821 hawk9821 force-pushed the transforms_append_only_stream branch from bf9ff26 to 15395c2 Compare October 10, 2024 07:27
@hawk9821
Copy link
Contributor Author

Is it possible to use the current SQL transform to turn one record into multiple records?
the transformation logic is not consistent with the SQL transform

@hawk9821 hawk9821 force-pushed the transforms_append_only_stream branch 2 times, most recently from d6f2cdf to 6d5c70b Compare October 10, 2024 08:57
@dailai
Copy link
Contributor

dailai commented Oct 11, 2024

Please fix ci.

@hawk9821 hawk9821 force-pushed the transforms_append_only_stream branch from 6d5c70b to ec905fe Compare October 14, 2024 03:21
@hawk9821 hawk9821 force-pushed the transforms_append_only_stream branch from ec905fe to 83e6b18 Compare October 14, 2024 04:23
dailai
dailai previously approved these changes Oct 14, 2024
Copy link
Contributor

@dailai dailai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM If ci passed

Hisoka-X
Hisoka-X previously approved these changes Oct 14, 2024
@dailai dailai added the don't merge There needs to be a specific reason in the PR, and it cannot be merged for the time being. label Oct 14, 2024
@Hisoka-X Hisoka-X dismissed stale reviews from dailai and themself via e34cb67 October 14, 2024 09:42
@dailai dailai removed the don't merge There needs to be a specific reason in the PR, and it cannot be merged for the time being. label Oct 14, 2024
Copy link
Member

@Hisoka-X Hisoka-X left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @hawk9821 and @dailai for review.

@Hisoka-X Hisoka-X merged commit e4e1224 into apache:dev Oct 14, 2024
8 of 9 checks passed
CosmosNi added a commit to CosmosNi/seatunnel that referenced this pull request Oct 15, 2024
@hawk9821 hawk9821 deleted the transforms_append_only_stream branch October 19, 2024 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants