Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS EC2 health codebundle #3

Open
wants to merge 57 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
63adb4f
add codebundles/aws-c7n-ebs-health/sli.robot
saurabh3460 Nov 20, 2024
7a205bd
add c7n ebs policies
saurabh3460 Nov 20, 2024
9ee4894
add script to create test infra
saurabh3460 Nov 20, 2024
b86124b
added runbook.robot with List Unattached EBS Volumes task
saurabh3460 Nov 20, 2024
e1bd6e0
Merge branch 'runwhen-contrib:main' into main
saurabh3460 Nov 20, 2024
774460f
added parse_ebs_results func in Core.py
saurabh3460 Nov 22, 2024
9d5dd28
change name of unused-ebs-snapshots policy
saurabh3460 Nov 22, 2024
3dd7314
change secret__aws_account_id -> secret__aws_access_key_id
saurabh3460 Nov 22, 2024
b9505d0
updated create/delete snapshot script in .test
saurabh3460 Nov 22, 2024
aa77f67
added List Unused EBS Snapshots and List Unencrypted EBS Volumes task…
saurabh3460 Nov 22, 2024
780854e
add runwhen generation rule and template yaml
saurabh3460 Nov 22, 2024
3455556
clean cc lib
saurabh3460 Nov 25, 2024
ecc92ff
replace ebs test script with terraform
saurabh3460 Nov 27, 2024
cfb684b
remove volume check and add encrypted false in ebs.tf
saurabh3460 Nov 27, 2024
e9f4513
added taskfile in ebs health codebundle
saurabh3460 Nov 27, 2024
6102c59
add account_id in ebs gen rule qualifiers
saurabh3460 Nov 27, 2024
4c59821
add check-rwp-config task in ebs cb test's taskfile
saurabh3460 Nov 27, 2024
90b306b
update ebs cb test README
saurabh3460 Nov 27, 2024
7f81bd3
add encrypted filed in ebs tf file
saurabh3460 Nov 28, 2024
9602af8
add suite variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in eb…
saurabh3460 Nov 28, 2024
becadf9
add rw-cli-keywords dependency in requirements.txt
saurabh3460 Nov 29, 2024
37fd8b7
fix sli locations filed in both ebs and s3 cb
saurabh3460 Nov 29, 2024
5672dc9
update Author in sli
saurabh3460 Nov 29, 2024
53c55bc
fix Add Issue and change AWS_ACCOUNT_NAME -> AWS_ACCOUNT_ID
saurabh3460 Nov 29, 2024
406e2fd
ebs Taskfile: add custom field and terraform/cb.secret
saurabh3460 Nov 29, 2024
b0d6153
ebs sli: fix score logic
saurabh3460 Nov 29, 2024
4580a3b
ebs runbook: update next steps string and task title
saurabh3460 Nov 29, 2024
d7fce89
EBS CB: fix typo and update image url in templates
saurabh3460 Dec 4, 2024
59c5463
update intervalSeconds 300 -> 600
saurabh3460 Dec 6, 2024
d1caa2e
EBS CB: update Metadata and thresholds defaults 1->0
saurabh3460 Dec 6, 2024
e4df799
ebs cb: rename gereration rule file
saurabh3460 Dec 6, 2024
b41e23d
ebs cb: update Metadata, statements in add issues in runbook.robot
saurabh3460 Dec 6, 2024
5d5f2c8
ebs cb: fix score logic
saurabh3460 Dec 11, 2024
7f557d0
ebs cb: update REDME.md with how to test steps
saurabh3460 Dec 12, 2024
1b2699e
ebs cb: update REDME.md with how to test steps
saurabh3460 Dec 12, 2024
c78fbe9
Merge remote-tracking branch 'upstream/main'
saurabh3460 Dec 13, 2024
d7d837a
EC2 cb: added test infra tf and taskfile
saurabh3460 Dec 2, 2024
062324b
added old-ec2-instances jinja template
saurabh3460 Dec 2, 2024
d16b95e
added generate_policy, find_value_recursive func and ec2 support in p…
saurabh3460 Dec 2, 2024
637a5fc
added GenerationRules and SLx templates
saurabh3460 Dec 2, 2024
c162e63
EC2 CB: added runbook
saurabh3460 Dec 4, 2024
667b8e1
EC2 CB: added sli.robot
saurabh3460 Dec 4, 2024
2310372
ec2 cb: intervalSeconds 300 -> 600
saurabh3460 Dec 9, 2024
e089330
ec2 cb: updated Metadata and issue statments in runbook.robot
saurabh3460 Dec 9, 2024
ea708b5
ec2 cb: updated Metadata and default in sli.robot
saurabh3460 Dec 9, 2024
beebae3
ec2 cb: fix score logic
saurabh3460 Dec 11, 2024
875b071
ec2 cb: add README.md files
saurabh3460 Dec 18, 2024
279b553
ec2 cb: add vpc resource in test terraform
saurabh3460 Dec 18, 2024
00be7ae
ec2 cb: update sli.yaml
saurabh3460 Dec 18, 2024
2c3e6fa
ec2 cb: update doc section in sli.robot
saurabh3460 Dec 18, 2024
f0f818c
ec2 cb: update doc and clean runbook.robot
saurabh3460 Dec 18, 2024
0c65a84
ec2 cb: update spec.alias and spec.asMeasuredBy in slx.yaml
saurabh3460 Dec 18, 2024
b1befae
ec2 cb: added unpatched and unused cloudcustodian policies
saurabh3460 Dec 19, 2024
fa9b73e
ebs cb: update tasks in runbook.robot
saurabh3460 Dec 19, 2024
cb46b27
ec2 cb: update rw templates
saurabh3460 Dec 19, 2024
a19f13a
ec2 cb: update root README.md SLI section
saurabh3460 Dec 19, 2024
f404f62
ebs cb: update tasks in sli.robot
saurabh3460 Dec 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
apiVersion: runwhen.com/v1
kind: GenerationRules
spec:
platform: aws
generationRules:
- resourceTypes:
- aws_ec2_instances
matchRules:
- type: pattern
pattern: ".+"
properties: [name]
mode: substring
slxs:
- baseName: aws-c7n-ec2-health
qualifiers: ["account_id", "region"]
baseTemplateName: aws-c7n-ec2-health
levelOfDetail: basic
outputItems:
- type: slx
- type: sli
- type: runbook
templateName: aws-c7n-ec2-health-taskset.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
apiVersion: runwhen.com/v1
kind: ServiceLevelIndicator
metadata:
name: {{slx_name}}
labels:
{% include "common-labels.yaml" %}
annotations:
{% include "common-annotations.yaml" %}
spec:
displayUnitsLong: OK
displayUnitsShort: ok
locations:
- {{default_location}}
description: Measures securitiy and health of AWS EC2 Instnaces in this AWS region {{match_resource.resource.region}} and account {{match_resource.resource.account_id}}
codeBundle:
{% if repo_url %}
repoUrl: {{repo_url}}
{% else %}
repoUrl: https://github.com/runwhen-contrib/rw-c7n-codecollection.git
{% endif %}
{% if ref %}
ref: {{ref}}
{% else %}
ref: main
{% endif %}
pathToRobot: codebundles/aws-c7n-ec2-health/sli.robot
intervalStrategy: intermezzo
intervalSeconds: 600
configProvided:
- name: AWS_REGION
value: "{{match_resource.resource.region}}"
- name: AWS_ACCOUNT_ID
value: "{{match_resource.resource.account_id}}"
secretsProvided:
- name: AWS_ACCESS_KEY_ID
workspaceKey: {{custom.aws_access_key_id}}
- name: AWS_SECRET_ACCESS_KEY
workspaceKey: {{custom.aws_secret_access_key}}
alerts:
warning:
operator: <
threshold: '1'
for: '20m'
ticket:
operator: <
threshold: '1'
for: '40m'
page:
operator: '=='
threshold: '0'
for: ''
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
apiVersion: runwhen.com/v1
kind: ServiceLevelX
metadata:
name: {{slx_name}}
labels:
{% include "common-labels.yaml" %}
annotations:
{% include "common-annotations.yaml" %}
spec:
imageURL: https://PLACEHOLDER
alias: AWS EC2 Health For Region {{match_resource.resource.region}} in Account {{match_resource.resource.account_id}}
asMeasuredBy: The number of unused and unpatched EC2 instances in region {{match_resource.resource.region}} and account {{match_resource.resource.account_id}}
configProvided:
- name: SLX_PLACEHOLDER
value: SLX_PLACEHOLDER
owners:
- {{workspace.owner_email}}
statement: Helps identify unused and unpatched EC2 instances in the region {{match_resource.resource.region}} for proactive resource cleanup and security.
additionalContext:
region: "{{match_resource.resource.region}}"
account_id: "{{match_resource.resource.account_id}}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
apiVersion: runwhen.com/v1
kind: Runbook
metadata:
name: {{slx_name}}
labels:
{% include "common-labels.yaml" %}
annotations:
{% include "common-annotations.yaml" %}
spec:
location: {{default_location}}
description: Runs tasks to identify and triage unused and unpatched AWS EC2 Instances.
codeBundle:
{% if repo_url %}
repoUrl: {{repo_url}}
{% else %}
repoUrl: https://github.com/runwhen-contrib/rw-c7n-codecollection.git
{% endif %}
{% if ref %}
ref: {{ref}}
{% else %}
ref: main
{% endif %}
pathToRobot: codebundles/aws-c7n-ec2-health/runbook.robot
configProvided:
- name: AWS_REGION
value: "{{match_resource.resource.region}}"
- name: AWS_ACCOUNT_ID
value: "{{match_resource.resource.account_id}}"
secretsProvided:
- name: AWS_ACCESS_KEY_ID
workspaceKey: {{custom.aws_access_key_id}}
- name: AWS_SECRET_ACCESS_KEY
workspaceKey: {{custom.aws_secret_access_key}}
100 changes: 100 additions & 0 deletions codebundles/aws-c7n-ec2-health/.test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
### How to test this codebundle?

#### IAM User Configuration

We create two distinct AWS IAM users with carefully scoped access:

**CloudCustodian IAM User**

Purpose: Service Level Indicator (SLI) monitoring and runbook automation and configured with least privilege access principles

With the following policy:

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"tag:GetResources",
"ec2:DescribeInstances",
"ec2:DescribeTags",
"autoscaling:DescribeTags",
"autoscaling:DescribeLaunchConfigurations",
"ec2:DescribeRegions",
"ec2:DescribeSnapshots",
"ec2:DescribeVolumeAttribute",
"ec2:DescribeSecurityGroups",
"ec2:DescribeImages",
"ec2:DescribeVolumeStatus",
"autoscaling:DescribeAutoScalingGroups",
"ec2:DescribeVpcs",
"ec2:DescribeVolumes"
],
"Resource": "*"
}
]
}
```

**Infrastructure Deployment User**

Purpose: Cloud infrastructure provisioning and management using Terraform

#### Credential Setup

Navigate to the `.test/terraform` directory and configure two secret files for authentication:

`cb.secret` - CloudCustodian and RunWhen Credentials

Create this file with the following environment variables:

```sh
export RW_PAT=""
export RW_WORKSPACE=""
export RW_API_URL="papi.beta.runwhen.com"

export AWS_DEFAULT_REGION="us-west-2"
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
export AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)
```


`tf.secret` - Terraform Deployment Credentials

Create this file with the following environment variables:

```sh
export AWS_DEFAULT_REGION=""
export AWS_ACCESS_KEY_ID=""
export AWS_SECRET_ACCESS_KEY=""
export AWS_SESSION_TOKEN="" # Optional: Include if using temporary credentials
```

#### Testing Workflow

1. Build test infra:
```sh
task build-infra
```

2. Generate RunWhen Configurations
```sh
tasks
```

3. Upload generated SLx to RunWhen Platform

```sh
task upload-slxs
```

4. At last, after testing, clean up the test infrastructure.

```sh
task clean
```

Loading