Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Include default capacity for cruise control #11212

Closed
azexs opened this issue Mar 3, 2025 · 1 comment
Closed

[Bug]: Include default capacity for cruise control #11212

azexs opened this issue Mar 3, 2025 · 1 comment

Comments

@azexs
Copy link

azexs commented Mar 3, 2025

Bug Description

When I create a kafka cluster with cruisecontrol, the operator creates a configmap with capacity settings for specific brokers managed by strimzi. When connecting an external broker to a strimzi cluster, the capacity for that broker is not in the configmap and cruisecontrol fails because of a null pointer. Default value for brokers should be defined with id “-1” according to cruise control. Would it be correct to extend the capacity.json file with an additional entry with id “-1” with default capacity brokers?

I am testing a specific case of connecting an external cluster to a strimzi cluster to replicate data using cruisecontrol.

Steps to reproduce

  1. Create strimzi kafka cluster with 2 brokers and strimzi
  2. Deploy external apache kafka broker and configure it to connect to strimzi cluster.
  3. Run cruisecontrol.

Expected behavior

capacity.json should include default capacity with id "-1"

Image

https://github.com/linkedin/cruise-control/wiki/Configurations#populating-the-capacity-config-file

Strimzi version

0.45.0

Kubernetes version

1.29.2

Installation method

No response

Infrastructure

No response

Configuration files and logs

2025-03-02 20:54:58 ERROR GoalOptimizer:615 - Proposal precomputation encountered error
com.linkedin.kafka.cruisecontrol.exception.BrokerCapacityResolutionException: Unable to resolve capacity of broker 3. Either (1) adding the default broker capacity (via adding capacity for broker -1 and allowing capacity estimation) or (2) adding missing broker's capacity in file /opt/cruise-control/custom-config/capacity.json.
at com.linkedin.kafka.cruisecontrol.config.BrokerCapacityConfigFileResolver.capacityForBroker(BrokerCapacityConfigFileResolver.java:181) ~[cruise-control-2.5.138.jar:?]
at com.linkedin.kafka.cruisecontrol.monitor.LoadMonitor.populateClusterCapacity(LoadMonitor.java:586) ~[cruise-control-2.5.138.jar:?]
at com.linkedin.kafka.cruisecontrol.monitor.LoadMonitor.clusterModel(LoadMonitor.java:516) ~[cruise-control-2.5.138.jar:?]
at com.linkedin.kafka.cruisecontrol.monitor.LoadMonitor.clusterModel(LoadMonitor.java:472) ~[cruise-control-2.5.138.jar:?]
at com.linkedin.kafka.cruisecontrol.monitor.LoadMonitor.clusterModel(LoadMonitor.java:442) ~[cruise-control-2.5.138.jar:?]
at com.linkedin.kafka.cruisecontrol.analyzer.GoalOptimizer$ProposalCandidateComputer.run(GoalOptimizer.java:603) ~[cruise-control-2.5.138.jar:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
at java.lang.Thread.run(Thread.java:840) ~[?:?]

Additional context

No response

@im-konge
Copy link
Member

im-konge commented Mar 6, 2025

Triaged on 6.3.2025: Based on I am testing a specific case of connecting an external cluster to a strimzi cluster to replicate data using cruisecontrol. it seems that you are doing something that is completely unsupported, as Strimzi doesn't have any control of the external cluster.

@azexs what are you trying to achieve with this?

@strimzi strimzi locked and limited conversation to collaborators Mar 6, 2025
@im-konge im-konge converted this issue into discussion #11230 Mar 6, 2025

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Projects
None yet
Development

No branches or pull requests

2 participants