Skip to content

Commit

Permalink
Added pytest and logging (based on #27) (#54)
Browse files Browse the repository at this point in the history
  • Loading branch information
antonbabenko authored Dec 20, 2019
1 parent 4c407c2 commit 3d19cf1
Show file tree
Hide file tree
Showing 8 changed files with 192 additions and 77 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,4 @@ terraform.tfstate
*.tfstate*
terraform.tfvars
functions/*.zip
functions/pytest.ini
30 changes: 18 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# AWS Notify Slack Terraform module

This module creates SNS topic (or use existing one) and a AWS Lambda function which sends notifications to Slack using [incoming webhooks API](https://api.slack.com/incoming-webhooks).
This module creates an SNS topic (or uses an existing one) and an AWS Lambda function that sends notifications to Slack using the [incoming webhooks API](https://api.slack.com/incoming-webhooks).

Start by setting up an [incoming webhook integration](https://my.slack.com/services/new/incoming-webhook/) in your Slack workspace.

Expand All @@ -17,8 +17,10 @@ Terraform 0.11. Pin module version to `~> v1.0`. Submit pull-requests to `terraf
- [x] Support plaintext and encrypted version of Slack webhook URL
- [x] Most of Slack message options are customizable
- [x] Support different types of SNS messages:
- [x] AWS Cloudwatch
- [x] AWS CloudWatch Alarms
- [x] AWS CloudWatch LogMetrics Alarms
- [ ] [Send pull-request to add support of other message types](https://github.com/terraform-aws-modules/terraform-aws-notify-slack/pulls)
- [x] Local pytest driven testing of the lambda to a Slack sandbox channel

## Usage

Expand All @@ -37,22 +39,25 @@ module "notify_slack" {

## Use existing SNS topic or create new

If you want to subscribe AWS Lambda Function created by this module to an existing SNS topic you should specify `create_sns_topic = false` as argument and specify name of existing SNS topic name in `sns_topic_name`.
If you want to subscribe the AWS Lambda Function created by this module to an existing SNS topic you should specify `create_sns_topic = false` as an argument and specify the name of existing SNS topic name in `sns_topic_name`.

## Import existing Cloudwatch Log Group
## Examples

Since `v2.3.0` of this module AWS Cloudwatch Log group is created also by this module.
* [notify-slack-simple](https://github.com/terraform-aws-modules/terraform-aws-notify-slack/tree/master/examples/notify-slack-simple) - Creates SNS topic which sends messages to Slack channel.
* [cloudwatch-alerts-to-slack](https://github.com/terraform-aws-modules/terraform-aws-notify-slack/tree/master/examples/cloudwatch-alerts-to-slack) - End to end example which shows how to send AWS Cloudwatch alerts to Slack channel and use KMS to encrypt webhook URL.

If you are updating from previous version of this module and you don't want to recreate log group, you need to import it like this (change `MODULE_NAME` as necessary):
## Testing with pytest

```
$ terraform import module.MODULE_NAME.aws_cloudwatch_log_group.lambda /aws/lambda/notify_slack
```
To run the tests:

## Examples
1. Set up a dedicated slack channel as a test sandbox with it's own webhook. See [Slack Incoming Webhooks docs](https://api.slack.com/messaging/webhooks) for details.
2. Make a copy of the sample pytest configuration and edit as needed.

* [notify-slack-simple](https://github.com/terraform-aws-modules/terraform-aws-notify-slack/tree/master/examples/notify-slack-simple) - Creates SNS topic which sends messages to Slack channel.
* [cloudwatch-alerts-to-slack](https://github.com/terraform-aws-modules/terraform-aws-notify-slack/tree/master/examples/cloudwatch-alerts-to-slack) - End to end example which shows how to send AWS Cloudwatch alerts to Slack channel and use KMS to encrypt webhook URL.
cp functions/pytest.ini.sample functions/pytest.ini

3. Run the tests:

pytest functions/notify_slack_test.py

<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Inputs
Expand All @@ -68,6 +73,7 @@ $ terraform import module.MODULE_NAME.aws_cloudwatch_log_group.lambda /aws/lambd
| kms\_key\_arn | ARN of the KMS key used for decrypting slack webhook url | string | `""` | no |
| lambda\_function\_name | The name of the Lambda function to create | string | `"notify_slack"` | no |
| lambda\_function\_tags | Additional tags for the Lambda function | map(string) | `{}` | no |
| log\_events | Boolean flag to enabled/disable logging of incoming events | string | `"false"` | no |
| reserved\_concurrent\_executions | The amount of reserved concurrent executions for this lambda function. A value of 0 disables lambda from being triggered and -1 removes any concurrency limitations | number | `"-1"` | no |
| slack\_channel | The name of the channel in Slack for notifications | string | n/a | yes |
| slack\_emoji | A custom emoji that will appear on Slack messages | string | `":aws:"` | no |
Expand Down
144 changes: 79 additions & 65 deletions functions/notify_slack.py
Original file line number Diff line number Diff line change
@@ -1,88 +1,102 @@
from __future__ import print_function
from urllib.error import HTTPError
import os, boto3, json, base64
import urllib.request, urllib.parse
import logging


# Decrypt encrypted URL with KMS
def decrypt(encrypted_url):
region = os.environ['AWS_REGION']
try:
kms = boto3.client('kms', region_name=region)
plaintext = kms.decrypt(CiphertextBlob=base64.b64decode(encrypted_url))['Plaintext']
return plaintext.decode()
except Exception:
logging.exception("Failed to decrypt URL with KMS")
region = os.environ['AWS_REGION']
try:
kms = boto3.client('kms', region_name=region)
plaintext = kms.decrypt(CiphertextBlob=base64.b64decode(encrypted_url))['Plaintext']
return plaintext.decode()
except Exception:
logging.exception("Failed to decrypt URL with KMS")


def cloudwatch_notification(message, region):
states = {'OK': 'good', 'INSUFFICIENT_DATA': 'warning', 'ALARM': 'danger'}

return {
"color": states[message['NewStateValue']],
"fallback": "Alarm {} triggered".format(message['AlarmName']),
"fields": [
{ "title": "Alarm Name", "value": message['AlarmName'], "short": True },
{ "title": "Alarm Description", "value": message['AlarmDescription'], "short": False},
{ "title": "Alarm reason", "value": message['NewStateReason'], "short": False},
{ "title": "Old State", "value": message['OldStateValue'], "short": True },
{ "title": "Current State", "value": message['NewStateValue'], "short": True },
{
"title": "Link to Alarm",
"value": "https://console.aws.amazon.com/cloudwatch/home?region=" + region + "#alarm:alarmFilter=ANY;name=" + urllib.parse.quote(message['AlarmName']),
"short": False
}
]
}
states = {'OK': 'good', 'INSUFFICIENT_DATA': 'warning', 'ALARM': 'danger'}

return {
"color": states[message['NewStateValue']],
"fallback": "Alarm {} triggered".format(message['AlarmName']),
"fields": [
{ "title": "Alarm Name", "value": message['AlarmName'], "short": True },
{ "title": "Alarm Description", "value": message['AlarmDescription'], "short": False},
{ "title": "Alarm reason", "value": message['NewStateReason'], "short": False},
{ "title": "Old State", "value": message['OldStateValue'], "short": True },
{ "title": "Current State", "value": message['NewStateValue'], "short": True },
{
"title": "Link to Alarm",
"value": "https://console.aws.amazon.com/cloudwatch/home?region=" + region + "#alarm:alarmFilter=ANY;name=" + urllib.parse.quote(message['AlarmName']),
"short": False
}
]
}


def default_notification(subject, message):
return {
"fallback": "A new message",
"fields": [{"title": subject if subject else "Message", "value": json.dumps(message), "short": False}]
}
return {
"fallback": "A new message",
"fields": [{"title": subject if subject else "Message", "value": json.dumps(message), "short": False}]
}


# Send a message to a slack channel
def notify_slack(subject, message, region):
slack_url = os.environ['SLACK_WEBHOOK_URL']
if not slack_url.startswith("http"):
slack_url = decrypt(slack_url)

slack_channel = os.environ['SLACK_CHANNEL']
slack_username = os.environ['SLACK_USERNAME']
slack_emoji = os.environ['SLACK_EMOJI']

payload = {
"channel": slack_channel,
"username": slack_username,
"icon_emoji": slack_emoji,
"attachments": []
}
if type(message) is str:
try:
message = json.loads(message)
except json.JSONDecodeError as err:
logging.exception(f'JSON decode error: {err}')
if "AlarmName" in message:
notification = cloudwatch_notification(message, region)
payload['text'] = "AWS CloudWatch notification - " + message["AlarmName"]
payload['attachments'].append(notification)
else:
payload['text'] = "AWS notification"
payload['attachments'].append(default_notification(subject, message))

data = urllib.parse.urlencode({"payload": json.dumps(payload)}).encode("utf-8")
req = urllib.request.Request(slack_url)
urllib.request.urlopen(req, data)
slack_url = os.environ['SLACK_WEBHOOK_URL']
if not slack_url.startswith("http"):
slack_url = decrypt(slack_url)

slack_channel = os.environ['SLACK_CHANNEL']
slack_username = os.environ['SLACK_USERNAME']
slack_emoji = os.environ['SLACK_EMOJI']

payload = {
"channel": slack_channel,
"username": slack_username,
"icon_emoji": slack_emoji,
"attachments": []
}

if type(message) is str:
try:
message = json.loads(message)
except json.JSONDecodeError as err:
logging.exception(f'JSON decode error: {err}')

if "AlarmName" in message:
notification = cloudwatch_notification(message, region)
payload['text'] = "AWS CloudWatch notification - " + message["AlarmName"]
payload['attachments'].append(notification)
else:
payload['text'] = "AWS notification"
payload['attachments'].append(default_notification(subject, message))

data = urllib.parse.urlencode({"payload": json.dumps(payload)}).encode("utf-8")
req = urllib.request.Request(slack_url)

try:
result = urllib.request.urlopen(req, data)
return json.dumps({"code": result.getcode(), "info": result.info().as_string()})

except HTTPError as e:
logging.error("{}: result".format(e))
return json.dumps({"code": e.getcode(), "info": e.info().as_string()})


def lambda_handler(event, context):
subject = event['Records'][0]['Sns']['Subject']
message = event['Records'][0]['Sns']['Message']
region = event['Records'][0]['Sns']['TopicArn'].split(":")[3]
notify_slack(subject, message, region)
if 'LOG_EVENTS' in os.environ and os.environ['LOG_EVENTS'] == 'True':
logging.warning('Event logging enabled: `{}`'.format(json.dumps(event)))

subject = event['Records'][0]['Sns']['Subject']
message = event['Records'][0]['Sns']['Message']
region = event['Records'][0]['Sns']['TopicArn'].split(":")[3]
response = notify_slack(subject, message, region)

return message
if json.loads(response)["code"] != 200:
logging.error("Error: received status `{}` using event `{}` and context `{}`".format(json.loads(response)["info"], event, context))

#notify_slack({"AlarmName":"Example","AlarmDescription":"Example alarm description.","AWSAccountId":"000000000000","NewStateValue":"ALARM","NewStateReason":"Threshold Crossed","StateChangeTime":"2017-01-12T16:30:42.236+0000","Region":"EU - Ireland","OldStateValue":"OK"}, "eu-west-1")
return response
76 changes: 76 additions & 0 deletions functions/notify_slack_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
#!/usr/bin/env pytest

import notify_slack
import pytest
from json import loads
from os import environ

events = (
(
{
"Records": [
{
"EventSource": "aws:sns",
"EventVersion": "1.0",
"EventSubscriptionArn": "arn:aws:sns:eu-west-2:735598076380:service-updates:d29b4e2c-6840-9c4e-ceac-17128efcc337",
"Sns": {
"Type": "Notification",
"MessageId": "f86e3c5b-cd17-1ab8-80e9-c0776d4f1e7a",
"TopicArn": "arn:aws:sns:eu-west-2:735598076380:service-updates",
"Subject": "OK: \"DBMigrationRequired\" in EU (London)",
"Message": "{\"AlarmName\":\"DBMigrationRequired\",\"AlarmDescription\":\"App is reporting \\\"A JPA error occurred (Unable to build EntityManagerFactory)\\\"\",\"AWSAccountId\":\"735598076380\",\"NewStateValue\":\"OK\",\"NewStateReason\":\"Threshold Crossed: 1 datapoint [1.0 (12/02/19 15:44:00)] was not less than the threshold (1.0).\",\"StateChangeTime\":\"2019-02-12T15:45:24.006+0000\",\"Region\":\"EU (London)\",\"OldStateValue\":\"ALARM\",\"Trigger\":{\"MetricName\":\"DBMigrationRequired\",\"Namespace\":\"LogMetrics\",\"StatisticType\":\"Statistic\",\"Statistic\":\"SUM\",\"Unit\":null,\"Dimensions\":[],\"Period\":60,\"EvaluationPeriods\":1,\"ComparisonOperator\":\"LessThanThreshold\",\"Threshold\":1.0,\"TreatMissingData\":\"- TreatMissingData: NonBreaching\",\"EvaluateLowSampleCountPercentile\":\"\"}}",
"Timestamp": "2019-02-12T15:45:24.091Z",
"SignatureVersion": "1",
"Signature": "WMYdVRN7ECNXMWZ0faRDD4fSfALW5MISB6O//LMd/LeSQYNQ/1eKYEE0PM1SHcH+73T/f/eVHbID/F203VZaGECQTD4LVA4B0DGAEY39LVbWdPTCHIDC6QCBV5ScGFZcROBXMe3UBWWMQAVTSWTE0eP526BFUTecaDFM4b9HMT4NEHWa4A2TA7d888JaVKKdSVNTd4bGS6Q2XFG1MOb652BRAHdARO7A6//2/47JZ5COM6LR0/V7TcOYCBZ20CRF6L5XLU46YYL3I1PNGKbEC1PIeVDVJVPcA17NfUbFXWYBX8LHfM4O7ZbGAPaGffDYLFWM6TX1Y6fQ01OSMc21OdUGV6HQR01e%==",
"SigningCertUrl": "https://sns.eu-west-2.amazonaws.com/SimpleNotificationService-7dd85a2b76adaa8dd603b7a0c9150589.pem",
"UnsubscribeUrl": "https://sns.eu-west-2.amazonaws.com/?Action=Unsubscribe&SubscriptionArn=arn:aws:sns:eu-west-2:735598076380:service-updates:d29b4e2c-6840-9c4e-ceac-17128efcc337",
"MessageAttributes": {}
}
}
]
}
),
(
{
"AlarmName": "Example",
"AlarmDescription": "Example alarm description.",
"AWSAccountId": "000000000000",
"NewStateValue": "ALARM",
"NewStateReason": "Threshold Crossed",
"StateChangeTime": "2017-01-12T16:30:42.236+0000",
"Region": "EU - Ireland",
"OldStateValue": "OK"
}
),
(
{
"AlarmType": "Unsupported alarm type",
"AWSAccountId": "000000000000",
"NewStateValue": "ALARM",
}
)
)


@pytest.fixture(scope='module', autouse=True)
def check_environment_variables():
required_environment_variables = ("SLACK_CHANNEL", "SLACK_EMOJI", "SLACK_USERNAME", "SLACK_WEBHOOK_URL")
missing_environment_variables = []
for k in required_environment_variables:
if k not in environ:
missing_environment_variables.append(k)

if len(missing_environment_variables) > 0:
pytest.exit('Missing environment variables: {}'.format(", ".join(missing_environment_variables)))


@pytest.mark.parametrize("event", events)
def test_lambda_handler(event):
if 'Records' in event:
response = notify_slack.lambda_handler(event, 'self-context')

else:
response = notify_slack.notify_slack('subject', event, 'eu-west-1')

response = loads(response)
assert response['code'] == 200
8 changes: 8 additions & 0 deletions functions/pytest.ini.sample
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[pytest]
addopts = --disable-pytest-warnings
env =
SLACK_CHANNEL=slack_testing_sandbox
SLACK_EMOJI=:aws:
SLACK_USERNAME=notify_slack_test
SLACK_WEBHOOK_URL=https://hooks.slack.com/services/YOUR/WEBOOK/URL

4 changes: 4 additions & 0 deletions functions/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
boto3

pytest
pytest-env
1 change: 1 addition & 0 deletions main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,7 @@ resource "aws_lambda_function" "notify_slack" {
SLACK_CHANNEL = var.slack_channel
SLACK_USERNAME = var.slack_username
SLACK_EMOJI = var.slack_emoji
LOG_EVENTS = var.log_events ? "True" : "False"
}
}

Expand Down
5 changes: 5 additions & 0 deletions variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,11 @@ variable "kms_key_arn" {
default = ""
}

variable "log_events" {
description = "Boolean flag to enabled/disable logging of incoming events"
default = false
}

variable "reserved_concurrent_executions" {
description = "The amount of reserved concurrent executions for this lambda function. A value of 0 disables lambda from being triggered and -1 removes any concurrency limitations"
type = number
Expand Down

0 comments on commit 3d19cf1

Please sign in to comment.