zstd support for Kafka output

[arnitolog]

Describe the enhancement:
It will be really good to have zstd in between Filebeat and Kafka. And it looks like IBM/sarama#1252 already solved, which unblocks zstd compression implementation in Kafka output.
What else needs to be done to have zstd compression as an option for Kafka output?

[elasticmachine]

Pinging @elastic/agent (Team:Agent)

[arnitolog]

Hello, any ETA when zstd compression might be supported?

[nimarezainia]

Hi @arnitolog due to other priorities facing the team right now, we are not considering adding this capability to the beats. We will keep reviewing this. thanks

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[gaby]

Any updates?

[botelastic]

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

[gaby]

Still valid 👍

[gaby]

Beats already has a dependency that provides support for zstd. Here https://github.com/elastic/beats/blob/main/go.mod#L334C2-L334C31

Dependency: https://github.com/klauspost/compress