Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AS8-6579] Initial checkin #1

Merged
merged 8 commits into from
Jul 12, 2021
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# ------------------------------------------------------------------------------
# Terraform
# ------------------------------------------------------------------------------

# Local .terraform directories
**/.terraform/*
**/.terragrunt-cache/*

# .tfstate files
*.tfstate
*.tfstate.*

# lock files
.terraform.lock.hcl

# Crash log files
crash.log

# Ignore any .tfvars files that are generated automatically for each Terraform run. Most
# .tfvars files are managed as part of configuration and so should be included in
# version control.
#
# example.tfvars

# Ignore override files as they are usually used to override resources locally and so
# are not checked in
override.tf
override.tf.json
*_override.tf
*_override.tf.json

# Include override files you do wish to add to version control using negated pattern
#
# !example_override.tf

# Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan
# example: *tfplan*

# macOS system files
**/.DS_Store

# Exclude IDE generated folders/files
.idea/
.vscode/
14 changes: 14 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.2.0
hooks:
- id: end-of-file-fixer
- id: trailing-whitespace
- id: check-merge-conflict
- repo: git://github.com/antonbabenko/pre-commit-terraform
rev: v1.50.0
hooks:
- id: terraform_fmt
- id: terraform_docs
args: ['-a --required=false']
- id: terraform_validate
107 changes: 105 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,105 @@
# cloud-build-slack-notifier
Terraform Module to add Slack notifications to Cloud Build
# terraform-google-cloud-build-slack-notifier

A Terraform module to enable Slack notifications for Cloud Build events.

**Note - This will add the following resources to your project:**

- Google Cloud Storage Bucket for storing the notifier configuration
- Google Pub/Sub for events emitted from Cloud Build
- Google Cloud Run for processing the events emitted from Cloud Build
cjonesy marked this conversation as resolved.
Show resolved Hide resolved

This module is based on the instructions found in GCP's [Configuring Slack notifications](https://cloud.google.com/build/docs/configuring-notifications/configure-slack) guide.

## Setup

You will need a Slack app incoming webhook url stored in a Google Secret Manager
secret for this to work.

- Create a [Slack app](https://api.slack.com/apps?new_app=1) for your desired Slack workspace.
- Activate [incoming webhooks](https://api.slack.com/messaging/webhooks) to post messages from Cloud Build to Slack.
- Create a new secret in Google Secret Manager and store the webhook url in it.

## Pre-commit Hooks

[Pre-commit](https://pre-commit.com/) hooks have been configured for this repo.

The enabled hooks check for a variety of common problems in Terraform code, and
will run any time you commit to your branch.

Pre-commit (and dependencies) can be installed by running:
`brew install pre-commit coreutils terraform-docs`

To enable the hooks locally, run the following from the root of this repo:
`pre-commit install`

To uninstall the hooks, run the following from the root of this repo:
`pre-commit uninstall`

To skip running the hooks when you commit:
`git commit -n` aka `git commit --no-verify`

**Currently enabled plugins:**

- [pre-commit-terraform](https://github.com/antonbabenko/pre-commit-terraform)
- `terraform_fmt`: Rewrites all Terraform configuration files to a canonical format
- `terraform_docs`: Inserts input and output documentation into `README.md`
- `terraform_validate`: Validates all Terraform configuration files
- [pre-commit-hooks](https://github.com/pre-commit/pre-commit-hooks)
- `end-of-file-fixer`: Makes sure files end in a newline and only a newline
- `trailing-whitespace`: Trims trailing whitespace
- `check-merge-conflict`: Check for files that contain merge conflict strings

<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Requirements

No requirements.

## Providers

| Name | Version |
|------|---------|
| <a name="provider_google"></a> [google](#provider\_google) | n/a |
| <a name="provider_google-beta"></a> [google-beta](#provider\_google-beta) | n/a |
| <a name="provider_random"></a> [random](#provider\_random) | n/a |

## Modules

No modules.

## Resources

| Name | Type |
|------|------|
| [google-beta_google_cloud_run_service.cloud_build_notifier](https://registry.terraform.io/providers/hashicorp/google-beta/latest/docs/resources/google_cloud_run_service) | resource |
| [google-beta_google_project_service_identity.pubsub](https://registry.terraform.io/providers/hashicorp/google-beta/latest/docs/resources/google_project_service_identity) | resource |
| [google_project_iam_member.notifier_project_roles](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/project_iam_member) | resource |
| [google_project_iam_member.pubsub_invoker_roles](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/project_iam_member) | resource |
| [google_project_iam_member.pubsub_project_roles](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/project_iam_member) | resource |
| [google_project_service.apis](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/project_service) | resource |
| [google_pubsub_subscription.cloud_builds](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/pubsub_subscription) | resource |
| [google_pubsub_topic.cloud_builds](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/pubsub_topic) | resource |
| [google_secret_manager_secret_iam_member.notifier_secret_accessor](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/secret_manager_secret_iam_member) | resource |
| [google_service_account.notifier](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/service_account) | resource |
| [google_service_account.pubsub_invoker](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/service_account) | resource |
| [google_storage_bucket.cloud_build_notifier](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket) | resource |
| [google_storage_bucket_object.cloud_build_notifier_config](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_object) | resource |
| [random_id.cloud_build_notifier](https://registry.terraform.io/providers/hashicorp/random/latest/docs/resources/id) | resource |
| [google_project.slack_webhook_url_secret_project](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/project) | data source |
| [google_secret_manager_secret_version.slack_webhook_url](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/secret_manager_secret_version) | data source |

## Inputs

| Name | Description | Type | Default |
|------|-------------|------|---------|
| <a name="input_cloud_build_event_filter"></a> [cloud\_build\_event\_filter](#input\_cloud\_build\_event\_filter) | The CEL filter to apply to incoming Cloud Build events. | `string` | `"build.substitutions['BRANCH_NAME'] == 'main' && build.status in [Build.Status.SUCCESS, Build.Status.FAILURE, Build.Status.TIMEOUT]"` |
| <a name="input_cloud_build_notifier_image"></a> [cloud\_build\_notifier\_image](#input\_cloud\_build\_notifier\_image) | The image to use for the notifier. | `string` | `"us-east1-docker.pkg.dev/gcb-release/cloud-build-notifiers/slack:latest"` |
| <a name="input_name"></a> [name](#input\_name) | The name to use on all resources created. | `string` | n/a |
| <a name="input_project_id"></a> [project\_id](#input\_project\_id) | Project ID of the project in which Cloud Build is running. | `string` | n/a |
| <a name="input_region"></a> [region](#input\_region) | The region in which to deploy the notifier service. | `string` | `"us-central1"` |
| <a name="input_slack_webhook_url_secret_id"></a> [slack\_webhook\_url\_secret\_id](#input\_slack\_webhook\_url\_secret\_id) | The ID of an existing Google Secret Manager secret, containing a Slack webhook URL. | `string` | n/a |
| <a name="input_slack_webhook_url_secret_project"></a> [slack\_webhook\_url\_secret\_project](#input\_slack\_webhook\_url\_secret\_project) | The project ID containing the slack\_webhook\_url\_secret\_id. | `string` | n/a |

## Outputs

No outputs.
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
218 changes: 218 additions & 0 deletions main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,218 @@
# Cloud Build Notifier

locals {
base_name = "cbnotify-${var.name}"
}


# ------------------------------------------------------------------------------
# Project
# ------------------------------------------------------------------------------

# Enable required APIs
resource "google_project_service" "apis" {
for_each = toset([
# Ensure cloudbuild API is enabled (it should already be though)
"cloudbuild.googleapis.com",
# Compute is used by Cloud Run, which in turn runs the notifier
"compute.googleapis.com",
# Pub/Sub is used to handle events from Cloud Build
"pubsub.googleapis.com",
# Cloud Run is used to run the notifier
"run.googleapis.com",
])
project = var.project_id
service = each.key

disable_dependent_services = true
}

# Lookup the slack_webhook_url_secret_project so we can access the project number
data "google_project" "slack_webhook_url_secret_project" {
project_id = var.slack_webhook_url_secret_project
}


# ------------------------------------------------------------------------------
# Secrets
# ------------------------------------------------------------------------------

data "google_secret_manager_secret_version" "slack_webhook_url" {
project = data.google_project.slack_webhook_url_secret_project.number
secret = var.slack_webhook_url_secret_id
}


# ------------------------------------------------------------------------------
# Service Accounts
# ------------------------------------------------------------------------------

# Create cloud build notifier service account
resource "google_service_account" "notifier" {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be good to output this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

never mind on this, I was confused on what it was used for.

account_id = "${local.base_name}-notifier"
project = var.project_id
}

# Give the service account required project permissions
resource "google_project_iam_member" "notifier_project_roles" {
for_each = toset([
"roles/storage.objectViewer",
"roles/iam.serviceAccountTokenCreator",
"roles/logging.logWriter"
])

project = var.project_id
role = each.key
member = "serviceAccount:${google_service_account.notifier.email}"
}

# Give the notifier service account access to the secret
resource "google_secret_manager_secret_iam_member" "notifier_secret_accessor" {
secret_id = data.google_secret_manager_secret_version.slack_webhook_url.secret
role = "roles/secretmanager.secretAccessor"
member = "serviceAccount:${google_service_account.notifier.email}"

depends_on = [
google_project_service.apis
]
}

# Look up the pubsub SA
resource "google_project_service_identity" "pubsub" {
provider = google-beta
project = var.project_id
service = "pubsub.googleapis.com"
}

# Grant the Pub/Sub SA permission to create auth tokens in your project
resource "google_project_iam_member" "pubsub_project_roles" {
project = var.project_id
role = "roles/iam.serviceAccountTokenCreator"
member = "serviceAccount:${google_project_service_identity.pubsub.email}"
}

# Create a pub/sub invoker service account
resource "google_service_account" "pubsub_invoker" {
account_id = "${local.base_name}-pubsub"
project = var.project_id
}

# Give the pub/sub invoker service account the Cloud Run Invoker permission
resource "google_project_iam_member" "pubsub_invoker_roles" {
project = var.project_id
role = "roles/run.invoker"
member = "serviceAccount:${google_service_account.pubsub_invoker.email}"
}


# ------------------------------------------------------------------------------
# GCS Bucket
# ------------------------------------------------------------------------------

# Create bucket
resource "random_id" "cloud_build_notifier" {
byte_length = 4
}

resource "google_storage_bucket" "cloud_build_notifier" {
project = var.project_id
name = "${local.base_name}-${random_id.cloud_build_notifier.hex}"
force_destroy = true
}

resource "google_storage_bucket_object" "cloud_build_notifier_config" {
name = "${local.base_name}-config.yaml"
bucket = google_storage_bucket.cloud_build_notifier.name

content = jsonencode({
apiVersion = "cloud-build-notifiers/v1"
kind = "SlackNotifier"
metadata = {
name = local.base_name
}
spec = {
notification = {
filter = var.cloud_build_event_filter
delivery = {
webhookUrl = {
secretRef = "webhook-url"
}
}
}
secrets = [
{
name = "webhook-url"
value = data.google_secret_manager_secret_version.slack_webhook_url.name
}
]
}
})
}


# ------------------------------------------------------------------------------
# Cloud Run
# ------------------------------------------------------------------------------

resource "google_cloud_run_service" "cloud_build_notifier" {
provider = google-beta

#FIXME: Is there no way to tell a cloud run service to restart when it's config changes?
name = "${local.base_name}-${lower(regex("[0-9A-Za-z]+", google_storage_bucket_object.cloud_build_notifier_config.crc32c))}" # HACK To make the cloud run job change when the config changes
Copy link
Contributor Author

@cjonesy cjonesy Jul 8, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels really gross to me, but I can't find a better way to handle this.

When we make a change to the configuration in a Cloud Run service it doesn't notice and just continues to run with the old configuration.

I read through all the options in the google_cloud_run_service docs, but could not find anything to trigger it to deploy a new revision on config change.

The only way I could come up with is appending the configs crc32 to the end of the service's name... which triggers terraform to destroy and recreate the service.

If anyone has ideas on how to get around this I'd love to hear them!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hashicorp/terraform#11418
It seems like this approach is the best way (assuming it actually redeploys when the name changes?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I verified that it does redeploy when the config changes.

location = var.region
project = var.project_id

template {
spec {
service_account_name = google_service_account.notifier.email

containers {
image = var.cloud_build_notifier_image

env {
name = "CONFIG_PATH"
value = "${google_storage_bucket.cloud_build_notifier.url}/${local.base_name}-config.yaml"
}

env {
name = "PROJECT_ID"
value = var.project_id
}
}
}
}

autogenerate_revision_name = true

lifecycle {
# Ignored because Cloud Run may add annotations outside of this config
ignore_changes = [
metadata.0.annotations,
]
}
}


# ------------------------------------------------------------------------------
# Pub/Sub
# ------------------------------------------------------------------------------

# Create the cloud-builds topic to receive build update messages for your notifier
resource "google_pubsub_topic" "cloud_builds" {
project = var.project_id
name = "cloud-builds"
}

resource "google_pubsub_subscription" "cloud_builds" {
name = local.base_name
topic = google_pubsub_topic.cloud_builds.name
project = var.project_id

push_config {
push_endpoint = google_cloud_run_service.cloud_build_notifier.status[0].url

oidc_token {
service_account_email = google_service_account.pubsub_invoker.email
}
}
}
Empty file added outputs.tf
Empty file.
Loading