Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FLINK-36858][pipeline-connector][kafka] Fix compatibility with Flink 1.20 JsonRowDataSerializationSchema #3784

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

MOBIN-F
Copy link
Contributor

@MOBIN-F MOBIN-F commented Dec 9, 2024

pipeline.yml

source:
  type: mysql
  name: MySQL Source
  hostname: localhost
  port: 3306
  username: test
  password: test
  tables: testDB.testTable
  server-id: 5206
  server-time-zone: Asia/Shanghai
  
sink:
  type: kafka
  name: Kafka Sink
  properties.bootstrap.servers: localhost:9092
  topic: test_topic
pipeline:
  name: MySQL to Kafka Pipeline
  parallelism: 1 
Caused by: java.lang.NoSuchMethodError: 'void org.apache.flink.formats.json.JsonRowDataSerializationSchema.<init>(org.apache.flink.table.types.logical.RowType, org.apache.flink.formats.common.TimestampFormat, org.apache.flink.formats.json.JsonFormatOptions$MapNullKeyMode, java.lang.String, boolean)'
        at org.apache.flink.cdc.connectors.kafka.serialization.JsonSerializationSchema.buildSerializationForPrimaryKey(JsonSerializationSchema.java:134)
        at org.apache.flink.cdc.connectors.kafka.serialization.JsonSerializationSchema.serialize(JsonSerializationSchema.java:101)
        at org.apache.flink.cdc.connectors.kafka.serialization.JsonSerializationSchema.serialize(JsonSerializationSchema.java:47)
        at org.apache.flink.cdc.connectors.kafka.sink.PipelineKafkaRecordSerializationSchema.serialize(PipelineKafkaRecordSerializationSchema.java:99)
        at org.apache.flink.cdc.connectors.kafka.sink.PipelineKafkaRecordSerializationSchema.serialize(PipelineKafkaRecordSerializationSchema.java:44) 

In flink>=1.20, the constructor of JsonRowDataSerializationSchema has 6 parameters, and in flink<1.20, the constructor of JsonRowDataSerializationSchema has 5 parameters.

@MOBIN-F
Copy link
Contributor Author

MOBIN-F commented Dec 9, 2024

@lvyanquan @yuxiqian cc~

Copy link
Contributor

@lvyanquan lvyanquan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this contribution, left some minor comments.

Copy link
Contributor

@lvyanquan lvyanquan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants