Skip to content

[STORM-1065] kafka-partition can not find leader in zookeeper #1146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lujinhong wants to merge 3 commits into from
Closed

[STORM-1065] kafka-partition can not find leader in zookeeper #1146

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
556ab52
fix storm-1065: kafka-partition can not find leader in zookeeper
lujinhong Feb 24, 2016
536beee
a simple way to fix storm-1065
lujinhong Feb 26, 2016
58f7fcb
a simple way to fix storm-1065
lujinhong Feb 26, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions external/storm-kafka/src/jvm/org/apache/storm/kafka/DynamicBrokersReader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,15 +156,12 @@ public String brokerPath() {
* @param partition
* @return
*/
private int getLeaderFor(String topic, long partition) {
public int getLeaderFor(String topic, long partition) {
try {
String topicBrokersPath = partitionPath(topic);
byte[] hostPortData = _curator.getData().forPath(topicBrokersPath + "/" + partition + "/state");
Map<Object, Object> value = (Map<Object, Object>) JSONValue.parse(new String(hostPortData, "UTF-8"));
Integer leader = ((Number) value.get("leader")).intValue();
if (leader == -1) {
throw new RuntimeException("No leader found for partition " + partition);
}
return leader;
} catch (RuntimeException e) {
throw e;
Expand Down
5 changes: 5 additions & 0 deletions external/storm-kafka/src/jvm/org/apache/storm/kafka/ZkCoordinator.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,11 @@ public void refresh() {
for (Partition id : newPartitions) {
PartitionManager man = new PartitionManager(_connections, _topologyInstanceId, _state, _stormConf, _spoutConfig, id);
_managers.put(id, man);
if(_reader.getLeaderFor(id.topic, id.partition) != -1) {
_managers.put(id, man);
}else {
LOG.warn("No leader found for partition " + id.topic + " : " + id.partition);
}
}

} catch (Exception e) {
Expand Down