After reconcillication, GUI does not show true state of cluster

[philwinder]

The introduced cluster management code does not update the GUI Task data structure, because some fields are not available when reconciling. E.g. the slave IP address is not available.

We need to consider removing items that are not available to the scheduler at reconcillication time. E.g. the IP address of the slave.

If they are important then we need to implement custom code to pass those details from the executors.

• Scheduler death: GUI will show zero tasks
• Executor death: GUI will show n + 1 tasks, since the task is not removed from the task list.

[philwinder]

Could initialize com.sonian.elasticsearch.zookeeper.discovery.ZooKeeperClusterState in the scheduler to read the ES state written by ES-zookeeper-discovery? Then we should be able to parse the hostname and ports from Elasticsearch's DiscoveryNode.

This is probably the best idea, because the data is already there and required for the ES discovery mechanism. However, this does couple the scheduler code to ES/ES-Zookeeper. Should seperate the code out from the scheduler as much as possible.

This should result in minimal changes to the web GUI.

[a-nldisr]

Is my assumption correct that this is related to when an executor fails and rescheduled, this failed executor is still listed in the dropdown list on the GUI?

[frankscholten]

@a-nldisr Yes

[a-nldisr]

thnx

[philwinder]

Fixed in #302