Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(aws-msk): Support newer versions of Kafka #23899

Closed
1 of 2 tasks
stefanfreitag opened this issue Jan 30, 2023 · 3 comments · Fixed by #23918 · 4 remaining pull requests
Closed
1 of 2 tasks

(aws-msk): Support newer versions of Kafka #23899

stefanfreitag opened this issue Jan 30, 2023 · 3 comments · Fixed by #23918 · 4 remaining pull requests
Labels
@aws-cdk/aws-msk Related to Amazon Managed Streaming for Apache Kafka (Amazon MSK) feature-request A feature should be added or improved.

Comments

@stefanfreitag
Copy link
Contributor

Describe the feature

CDK supports at the moment Kafka versions until v2.8.1 (see packages/@aws-cdk/aws-msk/lib/cluster-version.ts).
MSK itself added support for

  • Apache Kafka version 3.3.1
  • Apache Kafka version 3.2.0
  • Apache Kafka version 3.1.1
  • Amazon MSK tiered storage version 2.8.2.tiered

(see https://docs.aws.amazon.com/msk/latest/developerguide/supported-kafka-versions.html)

These new versions should be enabled also in the CDK.

Use Case

Within the next few month I need to migrate self-managed Kafka clusters to MSK. Here I would like to go with CDK and Kafka v3.x

Proposed Solution

The change could be implemented by enhancing the KafkaVersions class.
There is one additional change as tiered storage was introduced with 2.8.2- not sure how to treat this one.

Other Information

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.62.2

Environment details (OS name and version, etc.)

Ubuntu 22.04
@stefanfreitag stefanfreitag added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Jan 30, 2023
@github-actions github-actions bot added the @aws-cdk/aws-msk Related to Amazon Managed Streaming for Apache Kafka (Amazon MSK) label Jan 30, 2023
@pahud
Copy link
Contributor

pahud commented Jan 30, 2023

Yeah I think we could add the versions in the KafkaVersion class. Thank you for your PR contribution.

@pahud pahud removed the needs-triage This issue or PR still needs to be triaged. label Jan 30, 2023
@stefanfreitag stefanfreitag mentioned this issue Jan 30, 2023
Merged
@stefanfreitag
Copy link
Contributor Author

The tiered storage requires to my understanding an additional property in the construct - at the moment I will leave out the version 2.8.2.tiered and go only with 3.1.1, 3.2.0, and 3.3.1.

@mergify mergify bot closed this as completed in #23918 Feb 22, 2023
mergify bot pushed a commit that referenced this issue Feb 22, 2023
@stefanfreitag
feat(msk): add Kafka versions 3.1.1, 3.2.0, and and 3.3.1 (#23918)
53a1d5f 
Add support for Apache Kafka versions 3.1.1, 3.2.0 and 3.3.1 in Amazon MSK.

Announcements:
- [3.1.1 and 3.2.0](https://aws.amazon.com/about-aws/whats-new/2022/06/amazon-msk-adds-support-apache-kafka-version-3-1-1-3-2-0/) (Posted On: Jun 22, 2022)
- [3.3.1](https://aws.amazon.com/about-aws/whats-new/2022/10/amazon-msk-support-apache-kafka-version-3-3-1) (Posted On: Oct 26, 2022)

Closes #23899 
----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

Naumel pushed a commit that referenced this issue Feb 22, 2023
@stefanfreitag @Naumel
feat(msk): add Kafka versions 3.1.1, 3.2.0, and and 3.3.1 (#23918)
e1b4dec 
Add support for Apache Kafka versions 3.1.1, 3.2.0 and 3.3.1 in Amazon MSK.

Announcements:
- [3.1.1 and 3.2.0](https://aws.amazon.com/about-aws/whats-new/2022/06/amazon-msk-adds-support-apache-kafka-version-3-1-1-3-2-0/) (Posted On: Jun 22, 2022)
- [3.3.1](https://aws.amazon.com/about-aws/whats-new/2022/10/amazon-msk-support-apache-kafka-version-3-3-1) (Posted On: Oct 26, 2022)

Closes #23899 
----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Naumel pushed a commit that referenced this issue Feb 24, 2023
@stefanfreitag @Naumel
feat(msk): add Kafka versions 3.1.1, 3.2.0, and and 3.3.1 (#23918)
e1f4b49 
Add support for Apache Kafka versions 3.1.1, 3.2.0 and 3.3.1 in Amazon MSK.

Announcements:
- [3.1.1 and 3.2.0](https://aws.amazon.com/about-aws/whats-new/2022/06/amazon-msk-adds-support-apache-kafka-version-3-1-1-3-2-0/) (Posted On: Jun 22, 2022)
- [3.3.1](https://aws.amazon.com/about-aws/whats-new/2022/10/amazon-msk-support-apache-kafka-version-3-3-1) (Posted On: Oct 26, 2022)

Closes #23899 
----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
beck3905 pushed a commit to beck3905/aws-cdk that referenced this issue Feb 28, 2023
@stefanfreitag @beck3905
feat(msk): add Kafka versions 3.1.1, 3.2.0, and and 3.3.1 (aws#23918)
2eac2dc 
Add support for Apache Kafka versions 3.1.1, 3.2.0 and 3.3.1 in Amazon MSK.

Announcements:
- [3.1.1 and 3.2.0](https://aws.amazon.com/about-aws/whats-new/2022/06/amazon-msk-adds-support-apache-kafka-version-3-1-1-3-2-0/) (Posted On: Jun 22, 2022)
- [3.3.1](https://aws.amazon.com/about-aws/whats-new/2022/10/amazon-msk-support-apache-kafka-version-3-3-1) (Posted On: Oct 26, 2022)

Closes aws#23899 
----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@stefanfreitag stefanfreitag mentioned this issue Mar 3, 2023
Closed
2 tasks
homakk pushed a commit to homakk/aws-cdk that referenced this issue Mar 13, 2023
@stefanfreitag @homakk
feat(msk): add Kafka versions 3.1.1, 3.2.0, and and 3.3.1 (aws#23918)
8b596d6 
Add support for Apache Kafka versions 3.1.1, 3.2.0 and 3.3.1 in Amazon MSK.

Announcements:
- [3.1.1 and 3.2.0](https://aws.amazon.com/about-aws/whats-new/2022/06/amazon-msk-adds-support-apache-kafka-version-3-1-1-3-2-0/) (Posted On: Jun 22, 2022)
- [3.3.1](https://aws.amazon.com/about-aws/whats-new/2022/10/amazon-msk-support-apache-kafka-version-3-3-1) (Posted On: Oct 26, 2022)

Closes aws#23899 
----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
homakk pushed a commit to homakk/aws-cdk that referenced this issue Mar 28, 2023
@stefanfreitag @homakk
feat(msk): add Kafka versions 3.1.1, 3.2.0, and and 3.3.1 (aws#23918)
f7c86b8 
Add support for Apache Kafka versions 3.1.1, 3.2.0 and 3.3.1 in Amazon MSK.

Announcements:
- [3.1.1 and 3.2.0](https://aws.amazon.com/about-aws/whats-new/2022/06/amazon-msk-adds-support-apache-kafka-version-3-1-1-3-2-0/) (Posted On: Jun 22, 2022)
- [3.3.1](https://aws.amazon.com/about-aws/whats-new/2022/10/amazon-msk-support-apache-kafka-version-3-3-1) (Posted On: Oct 26, 2022)

Closes aws#23899 
----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@chrispidcock chrispidcock mentioned this issue Oct 16, 2023
Closed
2 tasks
This was referenced Sep 13, 2024
Open
Open
Open
@WontonSam WontonSam mentioned this issue Oct 25, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
@aws-cdk/aws-msk Related to Amazon Managed Streaming for Apache Kafka (Amazon MSK) feature-request A feature should be added or improved.
Projects
None yet
Milestone
No milestone
2 participants
@pahud @stefanfreitag