Sentry

We use Sentry to detect, triage and momnitor application errors. As part of the joining process, as a developer, you should be given access to our Sentry account: https://govuk-notify-gds.sentry.io/

PagerDuty alerts

Some issues may be significant enough that we want to have PagerDuty report on them, which can in turn send a message to Slack or call a developer if the issue is severe enough.

This is currently configured by (ab)using Sentry teams.

There are two Sentry teams for PagerDuty integrations:

• govuk-notify-pagerduty-p1
• govuk-notify-pagerduty-warnings

We have configured an alerting rule on notifications-api that will send any issues which go from 'resolved' -> 'unresolved' to the 'GOV.UK Notify - Warnings' PagerDuty service. This is currently used for our slow SMS delivery detection.

If you want a new issue to trigger a PagerDuty alert, then do the following:

• assign the issue to the appropriate Sentry team (govuk-notify-pagerduty-p1 or govuk-notify-pagerduty-warnings).
• check the existing alert rules for the project, eg https://govuk-notify-gds.sentry.io/alerts/rules
• add a new alert rule if an appropriate one doesn't already exist.
  • WHEN: The issue changes state from resolved to unresolved
  • IF: The issue is assigned to team
  • THEN: Send a notification to PagerDuty account GDS and service .
  • Set an appropriate name and assign the rule to the same sentry team.

We should only need to set up the alert if we want to trigger PagerDuty from a new project (repo). If the alert already exists for the project, you should just be able to assign the issue to the appropriate govuk-notify-pagerduty-x team.