Kapacitor changed host name but failed to create subscription

[earthnut]

After the test on the same server with influxDB, Kapacitor was moved to another server.
Every time Kapacitor was restarted, the create subscription message showed up in influxDB log.

However after run "show subscriptions", the localhost one was still there.

name: telegraf

retention_policy name mode destinations
default kapacitor ANY [udp://localhost:49771]

And influxDB always send UDP packet to:
[subscriber] 2016/02/24 16:36:40 write udp 127.0.0.1:61098->127.0.0.1:49771: write: connection refused

So do I supposed to drop subscription, before I changed the Kapacitor host name ?

Thanks

[OrangeTux]

I've had a similar problem and delete all subscriptions before starting Kapacitor again.

[nathanielc]

@earthnut Yep, you will need to delete the subscriptions manually if you change the hostname. There is no safe way to know that the old subscriptions do not belong to another Kapacitor instance on localhost so they are not automatically deleted. Should be a rare bit of manual work. Maybe we should use the cluster id in the name in the future but for now there is a simple work around.

[earthnut]

@nathanielc We don't have UDP load balancer.
It seems one influxDB can only have one Kapacitor to subscribe. In this case, if the Kapacitor server went down, we will lose alert messages.

Is there any work around for this situation ?

[nathanielc]

@earthnut Great question, there are two approaches:

1. Configure multiple subscriptions, for this we need to change the subscription name to include a unique ID, should be an easy fix. I'll get on it.
2. Configure a single subscription with multiple destinations. This requires that you setup the subscriptions manually. See the docs on defining subscriptions. You can specify multiple destination and a mode for how InfluxDB will send them data. ANY will pick just one, ALL will send data to all destinations, that way you can have redundant pairs.

Currently Kapacitor does not have good HA support. This is on the roadmap.

[earthnut]

@nathanielc
thanks

I like the 1st approach. we will try the the 2nd approach first.