Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature request] Support for MSK & IAM integration #1985

Closed
bpuertolas opened this issue Jul 12, 2021 · 7 comments
Closed

[Feature request] Support for MSK & IAM integration #1985

bpuertolas opened this issue Jul 12, 2021 · 7 comments
Labels
stale Issues and pull requests without any recent activity

Comments

@bpuertolas
Copy link

Is your feature request related to a problem? Please describe.
Recently, AWS released an integration between Amazon MSK (Amazon Managed Stream for Kafka) and IAM (Identity & Access Management). There is an example of this implementation but they're using a java client and a library named aws-msk-iam-auth in order to match IAM security.

Describe the solution you'd like
Would like to fully support this new integration.

Additional context
Currently, the only security measures that can be added to a Amazon MSK cluster is by protecting it with security groups or TLS. These options are full deny or full allow, there's no way to define a set of access per topic. This new integration with IAM allow this.

@gunturaf
Copy link

There's implementation reference for AWS_MSK_IAM at franz-go in this example https://github.com/twmb/franz-go/tree/master/examples/sasl/aws_msk_iam , I think it's a great addition to Sarama because downstream projects such as otel-contrib and Benthos uses Sarama as their underlying Kafka client, if Sarama supports this then the other projects will benefit too.

Can we sort of copy-pasta your AWS_MSK_IAM code in franz-go to achieve this in Sarama, @twmb ?

@cmaher
Copy link

cmaher commented Nov 29, 2021

kafka-go now has another reference implementation, which uses the aws-sdk to perform the signing, if the maintainers prefer that.

@twmb
Copy link

twmb commented Nov 29, 2021

opentelemetry managed to work around this without any changes to sarama. https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/exporter/kafkaexporter/authentication.go#L127

@MovieStoreGuy
Copy link

This needs to be added to the main project, the otel contribution written by me is currently flawed since sarama does validation checks on the auth mechanism used.

This should be a simple as taking the current version in otel and making it part of the main library.

@github-actions
Copy link

Thank you for taking the time to raise this issue. However, it has not had any activity on it in the past 90 days and will be closed in 30 days if no updates occur.
Please check if the main branch has already resolved the issue since it was raised. If you believe the issue is still valid and you would like input from the maintainers then please comment to ask for it to be reviewed.

@github-actions github-actions bot added the stale Issues and pull requests without any recent activity label Aug 29, 2023
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Sep 28, 2023
@bpalermo
Copy link

@MovieStoreGuy any progress on this?

@dtjm
Copy link

dtjm commented Dec 20, 2023

AWS has released a solution here https://github.com/aws/aws-msk-iam-sasl-signer-go

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stale Issues and pull requests without any recent activity
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants