[init] add optional client deregisration on stop

[cwjohnston]

Provided that CLIENT_DEREGISTER_ON_STOP environment variable is set to true and CLIENT_DEREGISTER_HANDLER is set to deregister, the init script will send the following event to sensu-client on port 3030:

{ 
  "name": "api_call", 
  "output": "delete", 
  "process": "init", 
  "user": "root", 
  "status": 2, 
  "handler": "deregister" 
}

This event is intended to be processed by the "deregister" handler.

[cwjohnston]

The deregistration handler was released in version 0.1.0 of sensu-plugins-sensu (see sensu-plugins/sensu-plugins-sensu#1)

[portertech]

Can we make this a warning (1)? As the event will temporarily display via the API/Dashboard.

[cwjohnston]

@johntdyer Uchiwa allows event view to be filtered by status (e.g. warning, critical, unknown, all). You're right that using the warning status here does not fully hide these events.

[johntdyer]

Yea, this is less then ideal for is because we want to see general warnings but we don't want to see deregister events cluttering up the dashboard

Sent from mobile

On Feb 16, 2016, at 23:59, Cameron Johnston notifications@github.com wrote:

@johntdyer Uchiwa allows event view to be filtered by status (e.g. warning, critical, unknown, all). You're right that using the warning status here does not fully hide these events.

—
Reply to this email directly or view it on GitHub.

[johntdyer]

@cwjohnston so how do you clear the warning alert on uchiwa ?

[cwjohnston]

@johntdyer This change to the init script only emits this deregistration event when the value of CLIENT_DEREGISTER_ON_STOP is true. As there is no default value for CLIENT_DEREGISTER_ON_STOP, the expected default behavior is for client deregistration to be disabled .

If you have enabled this behavior, you should also configure your Sensu server to process such events using the deregistration handler (shipped in sensu-plugins-sensu version 0.1.0). When this handler processes an event, it deletes the associated client via the Sensu API, which will, in turn, resolve any events associated with that client.

[johntdyer]

@cwjohnston sorry to be so "dense" here but here is what I have

I have added the handler in Sensu

{
  "handlers": {
    "deregister": {
      "type": "pipe",
      "severities": [
        "warning"
      ],
      "command": "handler-sensu-deregister.rb"
    }
  }
}

and then I degister an client and I see the following in the logs

{"timestamp":"2016-04-20T15:24:21.562171+0000","level":"info","message":"processing event","event":{"id":"465a7def-3993-4fb8-a7cb-a668772eee15","client":{"name":"rt2.prod.wdc.sl.xxxxxx","address":"198.11.254.103","subscriptions":["base","rt2.prod.wdc.sl.xxxxxx","hosted_wdc","prism_as","prism_ms","bind","runtime","runtime_wdc"],"keepalive":{"playbook":"https://xxxxxx.atlassian.net/browse/OPS-3848","thresholds":{"warning":120,"critical":180},"handlers":["default","pagerduty"],"refresh":1800},"version":"0.21.0","timestamp":1461165844},"check":{"name":"api_call","output":"delete","process":"init","user":"root","status":1,"handler":"deregister","executed":1461165861,"issued":1461165861,"history":["1","1"],"total_state_change":0},"occurrences":2,"action":"create","timestamp":1461165861}}
{"timestamp":"2016-04-20T15:24:22.200706+0000","level":"info","message":"handler output","handler":{"type":"pipe","severities":["warning"],"command":"handler-sensu-deregister.rb","name":"deregister"},"output":["only handling every 60 occurrences: rt2.prod.wdc.sl.xxxxxx/api_call\n"]}

[keitap]

I had the same issue as @johntdyer.
This issue is because 'filter_repeated' method ( https://github.com/sensu-plugins/sensu-plugin/blob/master/lib/sensu-handler.rb#L119 ) filters out a deregister event.
In my case, to restart sensu-server resolved it but not sure this is right way.

More better solution is that to add 'interval: 1, refresh: 1' to deregister event json here: https://github.com/sensu/sensu-build/pull/148/files#diff-32a7b82f0e4d5b8e858da79847107666R202
so that it will never filtered out by 'filter_repeated'.

Hope this will help.