Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adjust Pager Duty Alarms #3108

Closed
Jkd-eth opened this issue Dec 5, 2024 · 0 comments · Fixed by passportxyz/passport-scorer#751 or #3137
Closed

Adjust Pager Duty Alarms #3108

Jkd-eth opened this issue Dec 5, 2024 · 0 comments · Fixed by passportxyz/passport-scorer#751 or #3137
Assignees

Comments

@Jkd-eth
Copy link
Contributor

Jkd-eth commented Dec 5, 2024

User Story:

As the Passport Engineering team
We want to adjust the pager duty alarms
So that we can reduce the 'noise' of false positive alarms

Acceptance Criteria

GIVEN the existing Pager Duty Alarms
WHEN we've made updates
THEN we significantly reduce the number of alarms that clear within 10 or 15 minutes without intervention

Update Pager Duty Alarms & Conditions by doing:

  • Look at the latest incidents we have in PagerDuty
  • Increase the monitoring windows so we reduce the 5min auto resolve false positives

Once this is 'done' let's move this to 'blocked' instead of Product review to let it soak for 1 week before completing the work

Product & Design Links:

Tech Details:

Open Questions:

Notes/Assumptions:

@Jkd-eth Jkd-eth moved this to Prioritized in Passport New Dec 5, 2024
@lucianHymer lucianHymer moved this from Prioritized to In Progress (WIP) in Passport New Dec 12, 2024
@lucianHymer lucianHymer self-assigned this Dec 12, 2024
@lucianHymer lucianHymer moved this from In Progress (WIP) to Blocked in Passport New Dec 13, 2024
@lucianHymer lucianHymer moved this from Blocked to Code Complete in Passport New Dec 13, 2024
@lucianHymer lucianHymer moved this from Code Complete to Blocked in Passport New Dec 13, 2024
@lucianHymer lucianHymer moved this from Blocked to Code Complete in Passport New Dec 13, 2024
@nutrina nutrina linked a pull request Dec 16, 2024 that will close this issue
@larisa17 larisa17 moved this from Code Complete to Ready to Deploy in Passport New Dec 17, 2024
@Jkd-eth Jkd-eth moved this from Ready to Deploy to Done in Passport New Dec 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
2 participants