added detector for s3 5xxErrors

.github/workflows/generator.yml

@@ -66,7 +66,7 @@ jobs:
 
  - name: Enable destroy
  if: env.SFX_AUTH_TOKEN != null
- run: echo ::set-output name=enabled::1
+ run: echo "enabled=1" >> $GITHUB_ENV
  id: destroy
 
  - name: Terraform Apply

docs/severity.md

@@ -26,6 +26,7 @@
 - [integration_aws-rds-aurora-postgresql](#integration_aws-rds-aurora-postgresql)
 - [integration_aws-rds-common](#integration_aws-rds-common)
 - [integration_aws-redshift](#integration_aws-redshift)
+- [integration_aws-s3](#integration_aws-s3)
 - [integration_aws-sqs](#integration_aws-sqs)
 - [integration_aws-vpn](#integration_aws-vpn)
 - [integration_azure-api-management-service](#integration_azure-api-management-service)
@@ -334,6 +335,13 @@
 |AWS Redshift storage usage|X|X|-|-|-|
 
 
+## integration_aws-s3
+
+|Detector|Critical|Major|Minor|Warning|Info|
+|---|---|---|---|---|---|
+|AWS S3 s3 errors and requests|X|X|-|-|-|
+
+
 ## integration_aws-sqs
 
 |Detector|Critical|Major|Minor|Warning|Info|

modules/integration_aws-s3/README.md

@@ -0,0 +1,106 @@
+# AWS-S3 SignalFx detectors
+
+<!-- START doctoc generated TOC please keep comment here to allow auto update -->
+<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
+:link: **Contents**
+
+- [How to use this module?](#how-to-use-this-module)
+- [What are the available detectors in this module?](#what-are-the-available-detectors-in-this-module)
+- [How to collect required metrics?](#how-to-collect-required-metrics)
+ - [Metrics](#metrics)
+- [Related documentation](#related-documentation)
+
+<!-- END doctoc generated TOC please keep comment here to allow auto update -->
+
+## How to use this module?
+
+This directory defines a [Terraform](https://www.terraform.io/)
+[module](https://www.terraform.io/language/modules/syntax) you can use in your
+existing [stack](https://github.com/claranet/terraform-signalfx-detectors/wiki/Getting-started#stack) by adding a
+`module` configuration and setting its `source` parameter to URL of this folder:
+
+```hcl
+module "signalfx-detectors-integration-aws-s3" {
+ source = "github.com/claranet/terraform-signalfx-detectors.git//modules/integration_aws-s3?ref={revision}"
+
+ environment = var.environment
+ notifications = local.notifications
+}
+```
+
+Note the following parameters:
+
+* `source`: Use this parameter to specify the URL of the module. The double slash (`//`) is intentional and required.
+ Terraform uses it to specify subfolders within a Git repo (see [module
+ sources](https://www.terraform.io/language/modules/sources)). The `ref` parameter specifies a specific Git tag in
+ this repository. It is recommended to use the latest "pinned" version in place of `{revision}`. Avoid using a branch
+ like `master` except for testing purpose. Note that every modules in this repository are available on the Terraform
+ [registry](https://registry.terraform.io/modules/claranet/detectors/signalfx) and we recommend using it as source
+ instead of `git` which is more flexible but less future-proof.
+
+* `environment`: Use this parameter to specify the
+ [environment](https://github.com/claranet/terraform-signalfx-detectors/wiki/Getting-started#environment) used by this
+ instance of the module.
+ Its value will be added to the `prefixes` list at the start of the [detector
+ name](https://github.com/claranet/terraform-signalfx-detectors/wiki/Templating#example).
+ In general, it will also be used in the `filtering` internal sub-module to [apply
+ filters](https://github.com/claranet/terraform-signalfx-detectors/wiki/Guidance#filtering) based on our default
+ [tagging convention](https://github.com/claranet/terraform-signalfx-detectors/wiki/Tagging-convention) by default.
+
+* `notifications`: Use this parameter to define where alerts should be sent depending on their severity. It consists
+ of a Terraform [object](https://www.terraform.io/language/expressions/type-constraints#object) where each key represents an available
+ [detector rule severity](https://docs.splunk.com/observability/alerts-detectors-notifications/create-detectors-for-alerts.html#severity)
+ and its value is a list of recipients. Every recipients must respect the [detector notification
+ format](https://registry.terraform.io/providers/splunk-terraform/signalfx/latest/docs/resources/detector#notification-format).
+ Check the [notification binding](https://github.com/claranet/terraform-signalfx-detectors/wiki/Notifications-binding)
+ documentation to understand the recommended role of each severity.
+
+These 3 parameters alongs with all variables defined in [common-variables.tf](common-variables.tf) are common to all
+[modules](../) in this repository. Other variables, specific to this module, are available in
+[variables.tf](variables.tf) and [variables-gen.tf](variables-gen.tf).
+In general, the default configuration "works" but all of these Terraform
+[variables](https://www.terraform.io/language/values/variables) make it possible to
+customize the detectors behavior to better fit your needs.
+
+Most of them represent usual tips and rules detailled in the
+[guidance](https://github.com/claranet/terraform-signalfx-detectors/wiki/Guidance) documentation and listed in the
+common [variables](https://github.com/claranet/terraform-signalfx-detectors/wiki/Variables) dedicated documentation.
+
+Feel free to explore the [wiki](https://github.com/claranet/terraform-signalfx-detectors/wiki) for more information about
+general usage of this repository.
+
+## What are the available detectors in this module?
+
+This module creates the following SignalFx detectors which could contain one or multiple alerting rules:
+
+|Detector|Critical|Major|Minor|Warning|Info|
+|---|---|---|---|---|---|
+|AWS S3 s3 errors and requests|X|X|-|-|-|
+
+## How to collect required metrics?
+
+This module deploys detectors using metrics reported by the
+[AWS integration](https://docs.splunk.com/Observability/gdi/get-data-in/connect/aws/aws.html) configurable
+with [this Terraform module](https://github.com/claranet/terraform-signalfx-integrations/tree/master/cloud/aws).
+
+
+Check the [Related documentation](#related-documentation) section for more detailed and specific information about this module dependencies.
+
+
+
+### Metrics
+
+
+Here is the list of required metrics for detectors in this module.
+
+* `5xxErrors`
+* `AllRequests`
+
+
+
+
+## Related documentation
+
+* [Terraform SignalFx provider](https://registry.terraform.io/providers/splunk-terraform/signalfx/latest/docs)
+* [Terraform SignalFx detector](https://registry.terraform.io/providers/splunk-terraform/signalfx/latest/docs/resources/detector)
+* [Splunk Observability integrations](https://docs.splunk.com/Observability/gdi/get-data-in/integrations.html)

modules/integration_aws-s3/common-filters.tf

@@ -0,0 +1 @@
+../../common/module/filters-integration-aws.tf

modules/integration_aws-s3/common-locals.tf

@@ -0,0 +1 @@
+../../common/module/locals.tf

modules/integration_aws-s3/common-modules.tf

@@ -0,0 +1 @@
+../../common/module/modules.tf

modules/integration_aws-s3/common-variables.tf

@@ -0,0 +1 @@
+../../common/module/variables.tf

modules/integration_aws-s3/common-versions.tf

@@ -0,0 +1 @@
+../../common/module/versions.tf

modules/integration_aws-s3/conf/00-5xxErrors.yaml

@@ -0,0 +1,36 @@
+module: AWS S3
+name: S3 Errors and Requests
+id: s3_5xxErrors
+
+transformation: true
+aggregation: true
+filtering: "filter('namespace', 'AWS/S3')"
+
+value_unit: "%"
+
+signals:
+ errors:
+ metric: '5xxErrors'
+ extrapolation: zero
+ rollup: sum
+ requests:
+ metric: 'AllRequests'
+ extrapolation: zero
+ rollup: sum
+ signal:
+ formula: (errors/requests).scale(100).fill(value=0)
+
+rules:
+ critical:
+ threshold: 10
+ comparator: ">"
+ lasting_duration: 5m
+ lasting_at_least: 0.9
+ append_condition: and when(requests > ${var.minimum_traffic})
+ major:
+ threshold: 5
+ comparator: ">"
+ dependency: critical
+ lasting_duration: 5m
+ lasting_at_least: 0.9
+ append_condition: and when(requests > ${var.minimum_traffic})

modules/integration_aws-s3/conf/readme.yaml

@@ -0,0 +1,3 @@
+documentations:
+
+source_doc:

modules/integration_aws-s3/detectors-gen.tf

@@ -0,0 +1,48 @@
+resource "signalfx_detector" "s3_5xxerrors" {
+ name = format("%s %s", local.detector_name_prefix, "AWS S3 s3 errors and requests")
+
+ authorized_writer_teams = var.authorized_writer_teams
+ teams = try(coalescelist(var.teams, var.authorized_writer_teams), null)
+ tags = compact(concat(local.common_tags, local.tags, var.extra_tags))
+
+ viz_options {
+ label = "signal"
+ value_suffix = "%"
+ }
+
+ program_text = <<-EOF
+ base_filtering = filter('namespace', 'AWS/S3')
+ errors = data('5xxErrors', filter=base_filtering and ${module.filtering.signalflow}, rollup='sum', extrapolation='zero')${var.s3_5xxerrors_aggregation_function}${var.s3_5xxerrors_transformation_function}
+ requests = data('AllRequests', filter=base_filtering and ${module.filtering.signalflow}, rollup='sum', extrapolation='zero')${var.s3_5xxerrors_aggregation_function}${var.s3_5xxerrors_transformation_function}
+ signal = (errors/requests).scale(100).fill(value=0).publish('signal')
+ detect(when(signal > ${var.s3_5xxerrors_threshold_critical}, lasting=%{if var.s3_5xxerrors_lasting_duration_critical == null}None%{else}'${var.s3_5xxerrors_lasting_duration_critical}'%{endif}, at_least=${var.s3_5xxerrors_at_least_percentage_critical}) and when(requests > ${var.minimum_traffic})).publish('CRIT')
+ detect(when(signal > ${var.s3_5xxerrors_threshold_major}, lasting=%{if var.s3_5xxerrors_lasting_duration_major == null}None%{else}'${var.s3_5xxerrors_lasting_duration_major}'%{endif}, at_least=${var.s3_5xxerrors_at_least_percentage_major}) and when(requests > ${var.minimum_traffic}) and (not when(signal > ${var.s3_5xxerrors_threshold_critical}, lasting=%{if var.s3_5xxerrors_lasting_duration_critical == null}None%{else}'${var.s3_5xxerrors_lasting_duration_critical}'%{endif}, at_least=${var.s3_5xxerrors_at_least_percentage_critical}) and when(requests > ${var.minimum_traffic}))).publish('MAJOR')
+EOF
+
+ rule {
+ description = "is too high > ${var.s3_5xxerrors_threshold_critical}%"
+ severity = "Critical"
+ detect_label = "CRIT"
+ disabled = coalesce(var.s3_5xxerrors_disabled_critical, var.s3_5xxerrors_disabled, var.detectors_disabled)
+ notifications = try(coalescelist(lookup(var.s3_5xxerrors_notifications, "critical", []), var.notifications.critical), null)
+ runbook_url = try(coalesce(var.s3_5xxerrors_runbook_url, var.runbook_url), "")
+ tip = var.s3_5xxerrors_tip
+ parameterized_subject = var.message_subject == "" ? local.rule_subject : var.message_subject
+ parameterized_body = var.message_body == "" ? local.rule_body : var.message_body
+ }
+
+ rule {
+ description = "is too high > ${var.s3_5xxerrors_threshold_major}%"
+ severity = "Major"
+ detect_label = "MAJOR"
+ disabled = coalesce(var.s3_5xxerrors_disabled_major, var.s3_5xxerrors_disabled, var.detectors_disabled)
+ notifications = try(coalescelist(lookup(var.s3_5xxerrors_notifications, "major", []), var.notifications.major), null)
+ runbook_url = try(coalesce(var.s3_5xxerrors_runbook_url, var.runbook_url), "")
+ tip = var.s3_5xxerrors_tip
+ parameterized_subject = var.message_subject == "" ? local.rule_subject : var.message_subject
+ parameterized_body = var.message_body == "" ? local.rule_body : var.message_body
+ }
+
+ max_delay = var.s3_5xxerrors_max_delay
+}
+

modules/integration_aws-s3/outputs.tf

@@ -0,0 +1,5 @@
+output "s3_5xxerrors" {
+ description = "Detector resource for s3_5xxerrors"
+ value = signalfx_detector.s3_5xxerrors
+}
+

modules/integration_aws-s3/tags.tf

@@ -0,0 +1,4 @@
+locals {
+ tags = ["integration", "aws-s3"]
+}
+

modules/integration_aws-s3/variables-gen.tf

@@ -0,0 +1,90 @@
+# s3_5xxerrors detector
+
+variable "s3_5xxerrors_notifications" {
+ description = "Notification recipients list per severity overridden for s3_5xxerrors detector"
+ type = map(list(string))
+ default = {}
+}
+
+variable "s3_5xxerrors_aggregation_function" {
+ description = "Aggregation function and group by for s3_5xxerrors detector (i.e. \".mean(by=['host'])\")"
+ type = string
+ default = ""
+}
+
+variable "s3_5xxerrors_transformation_function" {
+ description = "Transformation function for s3_5xxerrors detector (i.e. \".mean(over='5m')\")"
+ type = string
+ default = ""
+}
+
+variable "s3_5xxerrors_max_delay" {
+ description = "Enforce max delay for s3_5xxerrors detector (use \"0\" or \"null\" for \"Auto\")"
+ type = number
+ default = null
+}
+
+variable "s3_5xxerrors_tip" {
+ description = "Suggested first course of action or any note useful for incident handling"
+ type = string
+ default = ""
+}
+
+variable "s3_5xxerrors_runbook_url" {
+ description = "URL like SignalFx dashboard or wiki page which can help to troubleshoot the incident cause"
+ type = string
+ default = ""
+}
+
+variable "s3_5xxerrors_disabled" {
+ description = "Disable all alerting rules for s3_5xxerrors detector"
+ type = bool
+ default = null
+}
+
+variable "s3_5xxerrors_disabled_critical" {
+ description = "Disable critical alerting rule for s3_5xxerrors detector"
+ type = bool
+ default = null
+}
+
+variable "s3_5xxerrors_disabled_major" {
+ description = "Disable major alerting rule for s3_5xxerrors detector"
+ type = bool
+ default = null
+}
+
+variable "s3_5xxerrors_threshold_critical" {
+ description = "Critical threshold for s3_5xxerrors detector in %"
+ type = number
+ default = 10
+}
+
+variable "s3_5xxerrors_lasting_duration_critical" {
+ description = "Minimum duration that conditions must be true before raising alert"
+ type = string
+ default = "5m"
+}
+
+variable "s3_5xxerrors_at_least_percentage_critical" {
+ description = "Percentage of lasting that conditions must be true before raising alert (>= 0.0 and <= 1.0)"
+ type = number
+ default = 0.9
+}
+variable "s3_5xxerrors_threshold_major" {
+ description = "Major threshold for s3_5xxerrors detector in %"
+ type = number
+ default = 5
+}
+
+variable "s3_5xxerrors_lasting_duration_major" {
+ description = "Minimum duration that conditions must be true before raising alert"
+ type = string
+ default = "5m"
+}
+
+variable "s3_5xxerrors_at_least_percentage_major" {
+ description = "Percentage of lasting that conditions must be true before raising alert (>= 0.0 and <= 1.0)"
+ type = number
+ default = 0.9
+}

modules/integration_aws-s3/variables.tf

@@ -0,0 +1,7 @@
+# Module specific
+
+variable "minimum_traffic" {
+ description = "Minimum required traffic to evaluate rate of errors detectors"
+ type = number
+ default = 4
+}