Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: kafka broker now supports all kafka topic config options #3671

Merged
merged 2 commits into from
Feb 9, 2024
Merged

feat: kafka broker now supports all kafka topic config options #3671

merged 2 commits into from
Feb 9, 2024

Conversation

Cali0707
Copy link
Member

@Cali0707 Cali0707 commented Feb 5, 2024

Fixes #1638

Proposed Changes

  • Parse all configmap values with prefix default.topic.config. into the topic config entries

This approach allows us to support any future changes to the kafka topic config options automatically. The user will still get a message when they provide an incorrect config option as the kafka server will reject the create topic request telling them what went wrong:

lastTransitionTime: "2024-02-05T21:25:12Z"
    message: 'kafka server: Configuration is invalid - Unknown topic config name: unknown-config'
    reason: 'Failed to create topic: knative-broker-default-broker-3'
    status: "False"
    type: Ready

Release Note

Auto created topics by brokers now support the full set of topic config options, set them by using the prefix `default.topic.config.` in your kafka-broker-config configmap.

Docs

Will open a docs PR if this approach is accepted

@Cali0707
feat: kafka broker now supports all kafka topic config options
4aeb6bc 
Signed-off-by: Calum Murray <cmurray@redhat.com>
@knative-prow knative-prow bot requested review from matzew and pierDipi February 5, 2024 21:33
@knative-prow knative-prow bot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Feb 5, 2024
@Cali0707
Copy link
Member Author

Cali0707 commented Feb 5, 2024

/cc @pierDipi @matzew @creydr

@knative-prow knative-prow bot requested a review from creydr February 5, 2024 21:33
@codecov Codecov
Copy link

codecov bot commented Feb 5, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (02be94e) 74.39% compared to head (d384ab8) 74.43%.
Report is 6 commits behind head on main.

Additional details and impacted files 
@@             Coverage Diff              @@
##               main    #3671      +/-   ##
============================================
+ Coverage     74.39%   74.43%   +0.03%     
- Complexity      845      846       +1     
============================================
  Files            97       97              
  Lines          3207     3207              
  Branches        273      273              
============================================
+ Hits           2386     2387       +1     
  Misses          654      654              
+ Partials        167      166       -1
Flag Coverage Δ
java-unittests 74.43% <ø> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Cali0707
test: added unit tests, fixed existing tests
d384ab8 
Signed-off-by: Calum Murray <cmurray@redhat.com>
@knative-prow knative-prow bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Feb 5, 2024
pierDipi
pierDipi approved these changes Feb 6, 2024
View reviewed changes
Copy link
Member

@pierDipi pierDipi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm
/approve

@knative-prow knative-prow bot added the lgtm Indicates that a PR is ready to be merged. label Feb 6, 2024
@knative-prow Knative Prow
Copy link

knative-prow bot commented Feb 6, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Cali0707, pierDipi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@pierDipi
Copy link
Member

pierDipi commented Feb 6, 2024

/retest-required

2 similar comments
@Cali0707
Copy link
Member Author

Cali0707 commented Feb 6, 2024

/retest-required

@Cali0707
Copy link
Member Author

Cali0707 commented Feb 6, 2024

/retest-required

@Cali0707
Copy link
Member Author

Cali0707 commented Feb 6, 2024

/cherry-pick release-1.13

@knative-prow-robot
Copy link
Contributor

@Cali0707: once the present PR merges, I will cherry-pick it on top of release-1.13 in a new PR and assign it to you.

In response to this:

/cherry-pick release-1.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@Cali0707
Copy link
Member Author

Cali0707 commented Feb 6, 2024

/cherry-pick release-1.12

@knative-prow-robot
Copy link
Contributor

@Cali0707: once the present PR merges, I will cherry-pick it on top of release-1.12 in a new PR and assign it to you.

In response to this:

/cherry-pick release-1.12

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@pierDipi
Copy link
Member

pierDipi commented Feb 6, 2024

/test reconciler-tests-namespaced-broker

@Cali0707
Copy link
Member Author

Cali0707 commented Feb 6, 2024

/retest-required

1 similar comment
@Cali0707
Copy link
Member Author

Cali0707 commented Feb 6, 2024

/retest-required

@pierDipi
Copy link
Member

pierDipi commented Feb 7, 2024

/test reconciler-tests

@Cali0707 Cali0707 mentioned this pull request Feb 7, 2024
Merged
@Cali0707
Copy link
Member Author

Cali0707 commented Feb 7, 2024

/retest-required

1 similar comment
@pierDipi
Copy link
Member

pierDipi commented Feb 8, 2024

/retest-required

@Cali0707
Copy link
Member Author

Cali0707 commented Feb 8, 2024

/retest-required

3 similar comments
@Cali0707
Copy link
Member Author

Cali0707 commented Feb 8, 2024

/retest-required

@pierDipi
Copy link
Member

pierDipi commented Feb 9, 2024

/retest-required

@pierDipi
Copy link
Member

pierDipi commented Feb 9, 2024

/retest-required

@creydr
Copy link
Contributor

creydr commented Feb 9, 2024

/test upgrade-tests

@knative-prow knative-prow bot merged commit 75194cf into knative-extensions:main Feb 9, 2024
39 checks passed
@knative-prow-robot
Copy link
Contributor

@Cali0707: new pull request created: #3686

In response to this:

/cherry-pick release-1.13

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@knative-prow-robot knative-prow-robot mentioned this pull request Feb 9, 2024
Merged
@knative-prow-robot
Copy link
Contributor

@Cali0707: new pull request created: #3687

In response to this:

/cherry-pick release-1.12

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@knative-prow-robot knative-prow-robot mentioned this pull request Feb 9, 2024
Merged
Cali0707 added a commit to Cali0707/eventing-kafka-broker that referenced this pull request Mar 19, 2024
@Cali0707
feat: kafka broker now supports all kafka topic config options (knati…
60c6834 
…ve-extensions#3671)

* feat: kafka broker now supports all kafka topic config options

Signed-off-by: Calum Murray <cmurray@redhat.com>

* test: added unit tests, fixed existing tests

Signed-off-by: Calum Murray <cmurray@redhat.com>

---------

Signed-off-by: Calum Murray <cmurray@redhat.com>
Cali0707 added a commit to Cali0707/eventing-kafka-broker that referenced this pull request Mar 19, 2024
@Cali0707
feat: kafka broker now supports all kafka topic config options (knati…
f4a716b 
…ve-extensions#3671)

* feat: kafka broker now supports all kafka topic config options

Signed-off-by: Calum Murray <cmurray@redhat.com>

* test: added unit tests, fixed existing tests

Signed-off-by: Calum Murray <cmurray@redhat.com>

---------

Signed-off-by: Calum Murray <cmurray@redhat.com>
openshift-merge-bot bot pushed a commit to openshift-knative/eventing-kafka-broker that referenced this pull request Mar 21, 2024
@Cali0707
feat: kafka broker now supports all kafka topic config options (knati…
2eb9b2f 
…ve-extensions#3671) (#1031)

* feat: kafka broker now supports all kafka topic config options



* test: added unit tests, fixed existing tests



---------

Signed-off-by: Calum Murray <cmurray@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pierDipi pierDipi pierDipi approved these changes

@matzew matzew Awaiting requested review from matzew

@creydr creydr Awaiting requested review from creydr

Assignees

@pierDipi pierDipi

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/control-plane lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Allow further configuration of topics
4 participants
@Cali0707 @pierDipi @knative-prow-robot @creydr