Implement alerts clearing

[dannycohen]

As Opie, I want to view alerts details and mark an alert as "cleared" after I took corrective actions

Visualizations:

1. An alert can be set to be "active" or "cleared"
2. Alert list displays only alerts whose status is "active"
3. To Clear an alert - select one or more alerts (by checking the checkbox in the first column of each row) and click the "Clear Alerts" button
4. If no heartbeat alerts remain active for an endpoint, the relevant indicators revers to green
5. It may be possible that an indicator is green but it has an active alert.

• See case 2 below which covers the following flow of events:
  1. Heartbeat failure causes the creation of an alert and the indicator is red
  2. The endpoint recuperates after a few minutes and heartbeat messages are received
  3. The previously created alert is still active - indicating there was a failure at some point in the past
  4. The indicator is green - indicating that in the heartbeat messages are received in the the present

Notes:

• This is a follow-up to Implement Heartbeat indicator #40, Implement Heartbeat indicator per Endpoint #41 and Implement Heartbeat alerts viewing #42, but applies to all alert types

Demo / Acceptance tests:

Case 1:

1. Deploy the heartbeat plugin in 3 of the 5 Video Store sample endpoints (in all except the ContentManagement and Operations endpoints)
2. Run the Video Store sample
3. Kill the "Sales" endpoint
4. Indicator should turn red within 1 minute
5. Number below Heartbeat Indicator should be 2 in green and 1 in red
6. Click on heartbeat indicator
7. 3 endpoint heartbeat indicators should be displayed, 2 green, 1 red
8. The name of each endpoint and the number seconds elapsed since last heartbeat messages was received noted next to each indicator
9. Click on the endpoint name "Sales"
10. The alerts list is displayed, filtered to show only the heartbeat alerts for the "Sales" endpoint
11. Select all the alerts for the "Sales" endpoint
12. Click the "Clear Alerts" button
13. The cleared alerts are removed from the alerts list
14. The endpoint heartbeat indicator for "Sales" endpoint is green (i.e. 3 green indicators) and so is the overall heartbeat indicator (Implement Heartbeat indicator #40)
15. Since we did not revive the "Sales" endpoint, the "Sales" heartbeat Indicator should turn red within 1 minute (and a new heartbeat alert should be created)

Case 2:

1. Deploy the heartbeat plugin in 3 of the 5 Video Store sample endpoints (in all except the ContentManagement and Operations endpoints)
2. Run the Video Store sample
3. Kill the "Sales" endpoint
4. Indicator should turn red within 1 minute
5. Number below Heartbeat Indicator should be 2 in green and 1 in red
6. Click on heartbeat indicator
7. 3 endpoint heartbeat indicators should be displayed, 2 green, 1 red
8. The name of each endpoint and the number seconds elapsed since last heartbeat messages was received noted next to each indicator
9. Click on the endpoint name "Sales"
10. The alerts list is displayed, filtered to show only the heartbeat alerts for the "Sales" endpoint
11. An active heartbeat alerts exists for the "Sales" endpoint
12. Create a new instance of the "Sales" endpoint
13. "Sales" endpoint Indicator should turn green within 1 minute (since it starts receiving heartbeat messages from the endpoint)
14. Next to the "Sales" endpoint indicator there is a small exclamation mark indicating there is an active alert
15. Click on the endpoint name "Sales"
16. The alerts list is displayed, filtered to show only the heartbeat alerts for the "Sales" endpoint
17. The same active heartbeat alerts exists for the "Sales" endpoint (see step 9 above)
18. Select the alert and click the "Clear Alerts" button
19. The active alert is removed from the alerts list
20. In the "Sales" endpoint indicator the small exclamation mark is no longer visible (indicating there are no longer any active alert)

[dannycohen]

Moved to Particular/ServicePulse#13