inconsistent behaviour between influxdb subscriptions and /write endpoint

[forestjohnsonpeoplenet]

We are using the TICK stack with the InfluxDb relay according to the diagram at https://github.com/influxdata/influxdb-relay#description

(Well technically it's the "TRIGK" stack since we are using grafana instead of chronograf :P)

We thought that the cleanest way to configure kapacitor would be to have the relays forward data to it directly, rather than having kapacitor subscribe to one of the influxdb replicas or subscribe to a load balancer representing the replicas. We wanted to do this so we could deploy two versions of Kapacitor at the same time, and then switch traffic between them at the relay or load balancer level. To do "blue-green" or "canary" deployments of new Kapacitor configurations.

We were able to configure that, but we hit a snag. When data is ingested by influxdb, if no retention policy is specified, influxdb will automatically put it in the "autogen" retention policy. However, it appears that when data is forwarded to Kapacitor, it doesn't. It appears to have a retention policy of "empty string" instead. This really threw me for a loop and I had no idea what was going on until I stumbled across the following thread on the influxdata mailing list:

https://groups.google.com/forum/#!searchin/influxdb/kapacitor$20write$20|sort:relevance/influxdb/mnomTKVUK98/fYnMoP3sCgAJ

In my opinion either InfluxDb should require a retention policy to be specified like Kapacitor does, or Kapacitor should automatically shovel metrics with null retention policies into one called autogen just like InfluxDb.

For now we can change our task definitions to subscribe to the emptystring retention policy as a work-around.

So, in the POST /tasks endpoint, we would use:

"dbrps": [
  {
    "db": "my_db",
    "rp": ""
  }
],

instead of what we used when we were using influxdb subscriptions:

"dbrps": [
  {
    "db": "my_db",
    "rp": "autogen"
  }
],

[forestjohnsonpeoplenet]

After a bit more googling I found this PR which fixes the issue in exactly the way I am suggesting, but on the influxdb-relay rather than on Kapacitor:

https://github.com/influxdata/influxdb-relay/pull/22/files

So, it's possible to set it at the relay, which is nice. It's a little bit hard to find though. Either that should be documented somewhere in the Kapacitor documentation or the same configuration option should be added to Kapacitor.

What about people with smaller projects who want to send directly from telegraf into kapacitor?

[nathanielc]

@forestjohnsonpeoplenet I think the same option should be added to Kapacitor.