Skip to content
This repository has been archived by the owner on Sep 2, 2024. It is now read-only.

[mtsource] MaxResponseSize considers the number of connected brokers #682

Conversation

lionelvillard
Copy link
Contributor

#668 followup.

This is a quick fix until a proper solution is implemented.

Proposed Changes

  • MaxResponseSize is per broker so lower its value based on the potential number of connected brokers (set to 6)

Release Note


Docs

@knative-prow-robot knative-prow-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jun 2, 2021
@google-cla google-cla bot added the cla: yes Indicates the PR's author has signed the CLA. label Jun 2, 2021
@knative-metrics-robot
Copy link

The following is the coverage report on the affected files.
Say /test pull-knative-sandbox-eventing-kafka-go-coverage to re-run this coverage report

File Old Coverage New Coverage Delta
pkg/source/mtadapter/adapter.go 78.2% 78.8% 0.6

@codecov
Copy link

codecov bot commented Jun 2, 2021

Codecov Report

Merging #682 (4564888) into main (b4481d1) will increase coverage by 0.13%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #682      +/-   ##
==========================================
+ Coverage   75.21%   75.35%   +0.13%     
==========================================
  Files         132      132              
  Lines        5782     5785       +3     
==========================================
+ Hits         4349     4359      +10     
+ Misses       1213     1206       -7     
  Partials      220      220              
Impacted Files Coverage Δ
pkg/source/mtadapter/adapter.go 75.18% <100.00%> (+0.55%) ⬆️
...el/distributed/dispatcher/dispatcher/dispatcher.go 80.53% <0.00%> (+6.19%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b4481d1...4564888. Read the comment docs.

@lionelvillard
Copy link
Contributor Author

/test pull-knative-sandbox-eventing-kafka-integration-test-mt-source

@lionelvillard
Copy link
Contributor Author

/test pull-knative-sandbox-eventing-kafka-upgrade-tests

// maxResponseSize is per connected brokers.
// For now cap the number of connected brokers to maxBrokers until a better solution
// comes along
maxResponseSize = int32(float32(maxResponseSize) / float32(maxBrokers))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maxBrokers is not fixed at 6. If someone uses a Kafka instance with 12 brokers, then this would be a problem. I guess for now, we can hard code it as 6. But this value is really per Kafka source. It should go onto the CR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Contributor

@steven0711dong steven0711dong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@knative-prow-robot knative-prow-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 3, 2021
@knative-prow-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lionelvillard, steven0711dong

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@steven0711dong
Copy link
Contributor

/lgtm

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cla: yes Indicates the PR's author has signed the CLA. lgtm Indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants