You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The jdbc-source-connector does not create kafka keys by default. I found some question on stackoverlow that mentions using a kafka transformer. Instead why not use primary keys as keys in Kafka? What fields are primary keys is already retrieved via GenericDatabaseDialect amd stored in ColumnDefinition. I'd think it would be not too hard to extend the jdbc-source-connecor to use the analoguous property PK_MODE. When set to RECORD_KEY the source connector could create keys as well using primary keys.
Advantage would be that compacting could be enabled for topics created using the jdbc-source-connector.
At first glance I'd think this feature wouldn't be too hard to add. But before I'd start working an a pull-request, Are there any reasons why this feature is not added? Is there additional complexity that I am missing?
The text was updated successfully, but these errors were encountered:
gerbrand
changed the title
Using primary keys to construct kafka keys
Using primary or unique keys to construct kafka keys
Apr 26, 2021
The jdbc-source-connector does not create kafka keys by default. I found some question on stackoverlow that mentions using a kafka transformer. Instead why not use primary keys as keys in Kafka? What fields are primary keys is already retrieved via GenericDatabaseDialect amd stored in ColumnDefinition. I'd think it would be not too hard to extend the jdbc-source-connecor to use the analoguous property PK_MODE. When set to RECORD_KEY the source connector could create keys as well using primary keys.
Advantage would be that compacting could be enabled for topics created using the jdbc-source-connector.
At first glance I'd think this feature wouldn't be too hard to add. But before I'd start working an a pull-request, Are there any reasons why this feature is not added? Is there additional complexity that I am missing?
The text was updated successfully, but these errors were encountered: