Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

2372 alert for excessive 5xx responses; descriptions for alerts #2373

Merged
merged 23 commits into from
Dec 29, 2023
Merged

2372 alert for excessive 5xx responses; descriptions for alerts #2373

merged 23 commits into from
Dec 29, 2023

Conversation

Matthew-Grayson
Copy link
Contributor

@Matthew-Grayson Matthew-Grayson commented Nov 9, 2023 •
edited
Loading

🗣 Description

Add log metric alarm for when the percentage of 5xx responses exceeds 5% from the API gateway.
Add extended description for all alerts.

💭 Motivation and context

Closes issue #2372

🧪 Testing

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • Changes are limited to a single goal - eschew scope creep!
  • All future TODOs are captured in issues, which are referenced
    in code comments.
  • All relevant type-of-change labels have been added.
  • I have read the CONTRIBUTING document.
  • These code changes follow cisagov code standards.
  • All relevant repo and/or project documentation has been updated
    to reflect the changes in this PR.
  • Tests have been added and/or modified to cover the changes in this PR.
  • All new and existing tests pass.

✅ Pre-merge checklist

  • Revert dependencies to default branches.
  • Finalize version.

✅ Post-merge checklist

  • Create a release.

Matthew-Grayson and others added 15 commits October 19, 2023 15:42
@Matthew-Grayson
Add log metric filters and alarms for remaining CloudWatch controls.
83f5f9e
@Matthew-Grayson
Fix log_metric_namespace definition in prod.tfvars.
c89d2ac
@Matthew-Grayson
Remove tags from log_filters.
f9f968f
@Matthew-Grayson
Add statistic field to alarms.
4d67c64
@Matthew-Grayson
Merge branch 'master' into 2328-add-log-metric-filtersalarms-for-aws-…
bbc537f 
…cis-benchmarks
@Matthew-Grayson
Add system shutdown filter and alarm; refactor log metric names to be…
a20074d 
… more descriptive.
@Matthew-Grayson
Merge branch 'master' into 2328-add-log-metric-filtersalarms-for-aws-…
8354649 
…cis-benchmarks
@Matthew-Grayson
Merge branch 'master' into 2328-add-log-metric-filtersalarms-for-aws-…
85f3f2a 
…cis-benchmarks
@Matthew-Grayson
Merge branch 'master' into 2328-add-log-metric-filtersalarms-for-aws-…
d28de8b 
…cis-benchmarks
@Matthew-Grayson
Fix capitalization of tags for resources in cloudwatch.tf, log_alarms…
39d6bb9 
….tf, and sns.tf.
@Matthew-Grayson
Add severity tag to alarms; refactor alarm name for ec2 instance shut…
4d6a6cf 
…down.
@Matthew-Grayson
Fix formatting.
cc2786c
@Matthew-Grayson
Refactor ec2 shutdown metric vars.
8f9251a
@Matthew-Grayson
Add filters/alarms for db instance shutdowns and deletions.
674b00f
@Matthew-Grayson
Add api_error_rate alert and vars.
b3d1e4f
@Matthew-Grayson Matthew-Grayson linked an issue Nov 9, 2023 that may be closed by this pull request
API Gateway Alert Threshold #2372
Closed
1 task
@Matthew-Grayson Matthew-Grayson self-assigned this Nov 9, 2023
@Matthew-Grayson Matthew-Grayson changed the title 2372 api gateway alert threshold 2372 alert for excessive 5xx responses; descriptions for alerts Nov 22, 2023
@Matthew-Grayson Matthew-Grayson marked this pull request as ready for review November 22, 2023 15:20
aloftus23
aloftus23 approved these changes Dec 27, 2023
View reviewed changes
Copy link
Contributor

@aloftus23 aloftus23 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

cduhn17
cduhn17 requested changes Dec 28, 2023
View reviewed changes
Copy link
Contributor

@cduhn17 cduhn17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Matt, See suggestions, otherwise this PR looks good.

infrastructure/log_alarms.tf Outdated Show resolved Hide resolved
@cduhn17 cduhn17 self-requested a review December 28, 2023 18:09
cduhn17
cduhn17 approved these changes Dec 28, 2023
View reviewed changes
Copy link
Contributor

@cduhn17 cduhn17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for taking a look at the suggestions

@cduhn17 cduhn17 merged commit a32ee59 into master Dec 29, 2023
7 of 9 checks passed
@cduhn17 cduhn17 deleted the 2372-api-gateway-alert-threshold branch December 29, 2023 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aloftus23 aloftus23 aloftus23 approved these changes

@schmelz21 schmelz21 schmelz21 approved these changes

@cduhn17 cduhn17 cduhn17 approved these changes

@nickviola nickviola Awaiting requested review from nickviola nickviola is a code owner

@edujosemena edujosemena Awaiting requested review from edujosemena

Assignees

@Matthew-Grayson Matthew-Grayson

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

API Gateway Alert Threshold
4 participants
@Matthew-Grayson @aloftus23 @schmelz21 @cduhn17