Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Kafka Component #884

Closed
iaingalloway opened this issue Nov 16, 2023 · 0 comments · Fixed by #951
Closed

Add Kafka Component #884

iaingalloway opened this issue Nov 16, 2023 · 0 comments · Fixed by #951
Labels
area-integrations Issues pertaining to Aspire Integrations packages

Comments

@iaingalloway
Copy link

iaingalloway commented Nov 16, 2023 •
edited
Loading

.NET Aspire shows great promise in simplifying the development of distributed systems using .NET for cloud-native applications. However, the absence of support for Kafka (or even similar technologies such as Azure Event Hubs) poses a significant limitation to its broader adoption in the cloud-native ecosystem.

Kafka is central to a lot of modern architecture. It's used by >80% of the Fortune 100, and unlike other systems where there are a lot of vendor-specific alternatives, it's overwhelmingly the most popular option when you're trying to build an event-driven backbone with data in motion at scale.

I recognize the challenges involved in integrating Kafka support, including the absence of a first-party managed Kafka service on Azure and lack of a comprehensive first-party Kafka.NET client. Despite these hurdles, I believe that supporting Kafka could greatly enhance .NET Aspire's relevance in contemporary cloud-native architectures.
@DamianEdwards DamianEdwards added the area-integrations Issues pertaining to Aspire Integrations packages label Nov 16, 2023
@iaingalloway iaingalloway changed the title Support Apache Kafka Add Kafka Component Nov 16, 2023
@g7ed6e g7ed6e mentioned this issue Nov 19, 2023
Merged
@eerhardt eerhardt linked a pull request Nov 28, 2023 that will close this issue
Add Kafka Component #951
Merged
@danmoseley danmoseley mentioned this issue Jan 4, 2024
Closed
eerhardt added a commit that referenced this issue Jan 12, 2024
@g7ed6e @eerhardt
Add Kafka Component (#951)
93cbd9a 
* Add Aspire.Confluent.Kafka component

Contains APIs for both AddKafkaConsumer and AddKafkaProducer. As well as Kafka Hosting APIs to start a Kafka server container.

Update ConfigurationSchema.json using ConfigSchemaGenerator. Cannot be automated until dotnet/runtime#96652 is fixed, so using a local temporary solution for now.

Temporarily add Kafka OTel Metrics logic here until open-telemetry/opentelemetry-dotnet-contrib#1493 is available to use.

Kafka sample can be found in dotnet/aspire-samples#85.

* Address PR feedback

- Make Kafka Settings Config object non-nullable and not settable from user code.
- Use EventId 1 in Logging
- Log response body in Functional tests

* Adjust the initialization of the Settings Config properties because the type can be created publicly.

Fix #884
---------

Co-authored-by: Eric Erhardt <eric.erhardt@microsoft.com>
@github-actions github-actions bot locked and limited conversation to collaborators Apr 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-integrations Issues pertaining to Aspire Integrations packages
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add Kafka Component g7ed6e/aspire
2 participants
@DamianEdwards @iaingalloway