Source:Kafka - Avro format message support

[sivankumar86]

What

Describe what the change is solving
User would be able to read/sync avro format message from kafka queue

How

I have added KafkaAvroSerializer API to deserialize the Avro message. Please refer
https://docs.confluent.io/platform/current/schema-registry/serdes-develop/serdes-avro.html for more details.

🚨 User Impact 🚨

This would be helpful to many users who are using avro format and want to integrate with Aribyte

It would resovles #15640 #15641

[sivankumar86]

@marcosmarxm would it be possible to you to assist on this change review ? I would appreciate it.

[sajarin]

Assigned to @prasrvenkat Edit: now open to maintainers

[sivankumar86]

@sajarin @prasrvenkat Thanks for checking the pull request. I would appreciate your feedback and marge to master asap.

[dariof28]

Cool! When will this be available on master?

[claudiocnc]

Waiting for this merge too... 👍

[ptkdev]

Any news on this merge?

[vincentkoc]

@sivankumar86 all the unit testing and changes are all good, just some cosmetic change:

• Adding confluent maven packages to the main deps.toml
• Reword some markdown files and spec
• Update the rest of the docs in relation to AVRO, Schema Registry and Max Topic support ondocs/integrations/sources/kafka.md

[sivankumar86]

@koconder thank you for taking a look. Could you take a look on new changes ?

[vincentkoc]

@sivankumar86 its all good, just waiting on a final confirmation from team if we need to add new maven dep to toml file and a final intergation test

[sivankumar86]

@koconder @sajarin @grishick Thank you for reviewing the changes. Could you merge to master ?

[sajarin]

/test connector=connectors/source-kafka

🕑 connectors/source-kafka https://github.com/airbytehq/airbyte/actions/runs/3046110561
✅ connectors/source-kafka https://github.com/airbytehq/airbyte/actions/runs/3046110561
No Python unittests run

Build Passed

Test summary info:

All Passed

[sajarin]

/publish connector=connectors/source-kafka

🕑 Publishing the following connectors:
connectors/source-kafka
https://github.com/airbytehq/airbyte/actions/runs/3046209554

Connector	Did it publish?	Were definitions generated?
connectors/source-kafka	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

[sajarin]

Thanks @sivankumar86 for the PR and thank you @koconder for the review!