Address issue #1699 (config option to deserialize proto message keys)

[jdechicchis]

Closes #1699

• Breaking change? (if so, please describe the impact and migration path for existing application instances)

What changes did you make? (Give an overview)
Add protobufMessageNameForKeyByTopic option to config. Message keys are deserialized using a protobuf schema if the config is set. Otherwise message keys are treated as strings.

Is there anything you'd like reviewers to focus on?
Name of configuration and unit tests as well as style.

How Has This Been Tested? (put an "x" (case-sensitive!) next to an item)

• No need to
• Manually (please, describe, if necessary)
• Unit checks
• Integration checks
• Covered by existing automation

Checklist (put an "x" (case-sensitive!) next to all the items, otherwise the build will fail)

• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation (e.g. ENVIRONMENT VARIABLES)
• My changes generate no new warnings (e.g. Sonar is happy)
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• Any dependent changes have been merged

Check out Contributing and Code of Conduct

A picture of a cute animal (not mandatory but encouraged)

[Haarolean]

@jdechicchis thanks for the contribution!

[iliax]

Hi @jdechicchis!
Thank you for your PR, please see comments.

In addition to them -> we should also edit serialize method to support sending messages with proto keys via kafka-ui.
Logic should be:

1. if there is no descriptor for topic's k or v -> treat (k or v) as a string - kOrV.getBytes();
2. if there is a descriptor for k/v -> use descriptor for serialization. If there is an serialization error - you should throw ValidationException exception

[jdechicchis]

@iliax Thank you for the review. I was wondering what to do about defaultMessageDescriptor based on your feedback. It's currently only used as a fallback to deserialize the message's value, but based on your feedback I think it may make sense to remove defaultMessageDescriptor and fallback to FALLBACK_FORMATTER (StringMessageFormatter) if the descriptor isn't found (for either key or value)

[iliax]

@jdechicchis Thank you for highlighting it, I actually forgot defaultMessageDescriptor logic... We currently can't remove this, because it will break backward-compatibility.
I think most the most proper way is to add Cluster.protobufMessageNameForKey property and use it as default for topic's keys that have no explicit mapping. If protobufMessageNameForKey is not specified or there was an error during initializing - we should fallback to StringMessageFormatter

[jdechicchis]

@iliax Thanks for the clarification, updated based on the feedback.

By the way, the "Documentation / build-and-test (pull_request)" check is failing with the error:

Done. The following urls did not pass:
https://dev-a63ggcut.auth0.com/

even though I didn't add that URL. What's the best way to resolve it?

[Haarolean]

That build fail is not your fault. Just ignore it for now.

…

On 30 Mar 2022, at 00:23, Joseph DeChicchis ***@***.***> wrote:  @iliax Thanks for the clarification, updated based on the feedback. By the way, the "Documentation / build-and-test (pull_request)" check is failing with the error: Done. The following urls did not pass: https://dev-a63ggcut.auth0.com/ even though I didn't add that URL. What's the best way to resolve it? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were assigned.

[iliax]

Hi @jdechicchis !
Thank you for great job - I added just a couple of small comments - pls fix them and we will be ready to merge

[jdechicchis]

@iliax Fixed log levels and added the @ Nullable annotations!