Support MSK IAM SASL mechanism ?

[ankush-custiv]

The exporter currently cannot connect to AWS MSK with the IAM SASL mechanism, which is also the recommended way to connect to MSK. Using IAM mechanism, we will only require AWS_SECRET_ACCESS_KEY & AWS_ACCESS_KEY_ID to connect to MSK. Please let us know if it will be supported in future.

[jclRatepay]

I am also very interested in this feature as we would like to make use of the kafka_exporter to get the metrics off of our AWS MSK clusters. I had a first look into the required changes my self - and am afraid that I guess it won't be so easy to achieve (need to mention that I have only poor golang experience yet). The kafka_exporter is making use of "sarama" as a Kafka Client which does not support MSK IAM SASL mechanism yet. There exist issues for implementing this in the client (see e.g. IBM/sarama#1985) which also seemed to be worked on, but this will be some kind of prerequisite before being able to implement that support here, and going through the sarama-library issues they seem to face some problems there.....

[ankush-custiv]

@jclRatepay Right, to support this feature, kafka_exporter must move to "kafka-go"(https://github.com/segmentio/kafka-go) instead of "sarama", atleast that's what we did at our org to use IAM SASL mechanism. There are many other benefits to using kafka-go over sarama & other kafka client libs, which are mentioned on their README. However, I'd wait for @danielqsj's feedback before going in this direction.

[troyApart]

There is now a sarama supported solution: https://github.com/aws/aws-msk-iam-sasl-signer-go
I also see an open PR related: #415

[sabper]

msk already has jmx exporter and node exporter.
It can be activated in msk monitoring settings.
https://docs.aws.amazon.com/ko_kr/msk/latest/developerguide/open-monitoring.html

I activated jmx/node exporter in msk and configured it to collect related metrics using prometheus service monitor.