Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial module addition and tests #2

Merged
Merged
Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
f6d3079
ran cookiecutter to generate module
Oct 21, 2019
4a4c1d5
initial demo generation, code copy, and test setup. WIP.
Oct 22, 2019
a8d74f7
local tests work now
Oct 22, 2019
6e7a56f
kitchen doesn't need to run these controls anymore
Oct 23, 2019
c47d728
update module description
Oct 23, 2019
6ddea6f
bump dev tools image version
Oct 23, 2019
eb5e4d8
update readme with required service account and api
Oct 23, 2019
cdb8cec
erase trailing whitespace
Oct 23, 2019
ccbf37b
bump developer tools to fix linting
Oct 23, 2019
c89b89b
show module outputs in tests
Oct 23, 2019
b135a42
updated readme, ran generate_docs
Oct 24, 2019
a7ddc6c
use a null_data_source for the outputs instead of a hacky way to wait…
Oct 24, 2019
799ba7b
move null data source over to main with proper dependency
Oct 28, 2019
9722b00
don't need these destroy trues anymore
Oct 28, 2019
eeb817b
spaces for tabs
Oct 28, 2019
d65165b
allow module invokers to pass in create/destroy commands instead of u…
Oct 28, 2019
d6691cf
don't require users to use an output to run a command
Oct 29, 2019
cc40d77
remove old outputs & regenerate docs
Oct 29, 2019
93d0cff
remove old outputs from test outputs
Oct 30, 2019
6bceb2f
update README and remove input variables passing in defaults
Oct 30, 2019
e891d0e
revert null data source to locals workaround, see issue #1
Nov 8, 2019
e69fee3
bump suite timeout
Nov 8, 2019
684e86e
allow this module to optionally be disabled
Nov 8, 2019
eb8c817
make sure the run command doesn't go before compress
Nov 8, 2019
0f8aedb
update README
Nov 8, 2019
5013d6e
add a couple outputs for this module
Nov 8, 2019
8867bd9
no type on outputs
Nov 8, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.git
.terraform
.terraform.d
.kitchen
terraform.tfstate.d
test/fixtures/*/.terraform
test/fixtures/*/terraform.tfstate.d
examples/.kitchen
examples/*/.terraform
examples/*/terraform.tfstate.d
47 changes: 47 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# OSX leaves these everywhere on SMB shares
._*

# OSX trash
.DS_Store

# Python
*.pyc

# Emacs save files
*~
\#*\#
.\#*

# Vim-related files
[._]*.s[a-w][a-z]
[._]s[a-w][a-z]
*.un~
Session.vim
.netrwhist

### https://raw.github.com/github/gitignore/90f149de451a5433aebd94d02d11b0e28843a1af/Terraform.gitignore

# Local .terraform directories
**/.terraform/*

# .tfstate files
*.tfstate
*.tfstate.*

# Crash log files
crash.log

# Kitchen files
**/inspec.lock
**/.kitchen
**/kitchen.local.yml
**/Gemfile.lock

# Ignore any .tfvars files that are generated automatically for each Terraform run. Most
# .tfvars files are managed as part of configuration and so should be included in
# version control.
**/*.tfvars

google-cloud-sdk/
google-cloud-sdk.staging/
credentials.json
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on
[Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to
[Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [0.1.0] - 20XX-YY-ZZ

### Added

- Initial release

[Unreleased]: https://github.com/terraform-google-modules/terraform-google-gcloud/compare/v0.1.0...HEAD
[0.1.0]: https://github.com/terraform-google-modules/terraform-google-gcloud/releases/tag/v0.1.0
99 changes: 99 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Contributing

This document provides guidelines for contributing to the module.

## Dependencies

The following dependencies must be installed on the development system:

- [Docker Engine][docker-engine]
- [Google Cloud SDK][google-cloud-sdk]
- [make]

## Generating Documentation for Inputs and Outputs

The Inputs and Outputs tables in the READMEs of the root module,
submodules, and example modules are automatically generated based on
the `variables` and `outputs` of the respective modules. These tables
must be refreshed if the module interfaces are changed.

### Execution

Run `make generate_docs` to generate new Inputs and Outputs tables.

## Integration Testing

Integration tests are used to verify the behaviour of the root module,
submodules, and example modules. Additions, changes, and fixes should
be accompanied with tests.

The integration tests are run using [Kitchen][kitchen],
[Kitchen-Terraform][kitchen-terraform], and [InSpec][inspec]. These
tools are packaged within a Docker image for convenience.

The general strategy for these tests is to verify the behaviour of the
[example modules](./examples/), thus ensuring that the root module,
submodules, and example modules are all functionally correct.

### Test Environment
The easiest way to test the module is in an isolated test project. The setup for such a project is defined in [test/setup](./test/setup/) directory.

To use this setup, you need a service account with Project Creator access on a folder. Export the Service Account credentials to your environment like so:

```
export SERVICE_ACCOUNT_JSON=$(< credentials.json)
```

You will also need to set a few environment variables:
```
export TF_VAR_org_id="your_org_id"
export TF_VAR_folder_id="your_folder_id"
export TF_VAR_billing_account="your_billing_account_id"
```

With these settings in place, you can prepare a test project using Docker:
```
make docker_test_prepare
```

### Noninteractive Execution

Run `make docker_test_integration` to test all of the example modules
noninteractively, using the prepared test project.

### Interactive Execution

1. Run `make docker_run` to start the testing Docker container in
interactive mode.

1. Run `kitchen_do create <EXAMPLE_NAME>` to initialize the working
directory for an example module.

1. Run `kitchen_do converge <EXAMPLE_NAME>` to apply the example module.

1. Run `kitchen_do verify <EXAMPLE_NAME>` to test the example module.

1. Run `kitchen_do destroy <EXAMPLE_NAME>` to destroy the example module
state.

## Linting and Formatting

Many of the files in the repository can be linted or formatted to
maintain a standard of quality.

### Execution

Run `make docker_test_lint`.

[docker-engine]: https://www.docker.com/products/docker-engine
[flake8]: http://flake8.pycqa.org/en/latest/
[gofmt]: https://golang.org/cmd/gofmt/
[google-cloud-sdk]: https://cloud.google.com/sdk/install
[hadolint]: https://github.com/hadolint/hadolint
[inspec]: https://inspec.io/
[kitchen-terraform]: https://github.com/newcontext-oss/kitchen-terraform
[kitchen]: https://kitchen.ci/
[make]: https://en.wikipedia.org/wiki/Make_(software)
[shellcheck]: https://www.shellcheck.net/
[terraform-docs]: https://github.com/segmentio/terraform-docs
[terraform]: https://terraform.io/
1 change: 1 addition & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
Expand Down
117 changes: 117 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
# Copyright 2019 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# Please note that this file was generated from [terraform-google-module-template](https://github.com/terraform-google-modules/terraform-google-module-template).
# Please make sure to contribute relevant changes upstream!

# Make will use bash instead of sh
SHELL := /usr/bin/env bash

GCLOUD_SDK_VERSION:=268.0.0
DOCKER_TAG_VERSION_DEVELOPER_TOOLS := 0.4.6
DOCKER_IMAGE_DEVELOPER_TOOLS := cft/developer-tools
REGISTRY_URL := gcr.io/cloud-foundation-cicd

# Enter docker container for local development
.PHONY: docker_run
docker_run:
docker run --rm -it \
-e SERVICE_ACCOUNT_JSON \
-v "$(CURDIR)":/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/bin/bash

# Execute prepare tests within the docker container
.PHONY: docker_test_prepare
docker_test_prepare:
docker run --rm -it \
-e SERVICE_ACCOUNT_JSON \
-e TF_VAR_org_id \
-e TF_VAR_folder_id \
-e TF_VAR_billing_account \
-v "$(CURDIR)":/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/usr/local/bin/execute_with_credentials.sh prepare_environment

# Clean up test environment within the docker container
.PHONY: docker_test_cleanup
docker_test_cleanup:
docker run --rm -it \
-e SERVICE_ACCOUNT_JSON \
-e TF_VAR_org_id \
-e TF_VAR_folder_id \
-e TF_VAR_billing_account \
-v "$(CURDIR)":/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/usr/local/bin/execute_with_credentials.sh cleanup_environment

# Execute integration tests within the docker container
.PHONY: docker_test_integration
docker_test_integration:
docker run --rm -it \
-e SERVICE_ACCOUNT_JSON \
-v "$(CURDIR)":/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/usr/local/bin/test_integration.sh

# Execute lint tests within the docker container
.PHONY: docker_test_lint
docker_test_lint:
docker run --rm -it \
-v "$(CURDIR)":/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/usr/local/bin/test_lint.sh

# Generate documentation
.PHONY: docker_generate_docs
docker_generate_docs:
docker run --rm -it \
-v "$(CURDIR)":/workspace \
$(REGISTRY_URL)/${DOCKER_IMAGE_DEVELOPER_TOOLS}:${DOCKER_TAG_VERSION_DEVELOPER_TOOLS} \
/bin/bash -c 'source /usr/local/bin/task_helper_functions.sh && generate_docs'

# Alias for backwards compatibility
.PHONY: generate_docs
generate_docs: docker_generate_docs

.PHONY: all
all: reset
all:
$(MAKE) gcloud.darwin
$(MAKE) gcloud.linux

.PHONY: gcloud.darwin
gcloud.darwin: OS_ARCH=darwin
gcloud.darwin: gcloud.download

.PHONY: gcloud.linux
gcloud.linux: OS_ARCH=linux
gcloud.linux: gcloud.download

.PHONY: gcloud.download
gcloud.download:
mkdir -p cache/${OS_ARCH}/
cd cache/${OS_ARCH}/ && \
curl -sL -o google-cloud-sdk.tar.gz https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-${GCLOUD_SDK_VERSION}-${OS_ARCH}-x86_64.tar.gz

.PHONY: clean
clean: ## Clean caches of decompressed SDKs
rm -rf cache/darwin/google-cloud-sdk/
rm -rf cache/linux/google-cloud-sdk/
rm -rf cache/darwin/google-cloud-sdk.staging/
rm -rf cache/linux/google-cloud-sdk.staging/

.PHONY: reset
reset:
rm -rf cache
86 changes: 85 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,85 @@
# terraform-google-gcloud
# terraform-google-gcloud

This module allows you to use gcloud, gsutil, and any gcloud component in Terraform. Sometimes, there isn't Terraform GCP support for a particular feature, or you'd like to do something each time Terraform runs (ie: upload a file to a Kubernetes pod) that lacks Terraform support.

This module *does not* create any resources on GCP itself, rather exposes the GCP SDK to you for usage in null resources & external data resources.

## Usage

Basic usage of this module is as follows:

```hcl
module "gcloud" {
source = "terraform-google-modules/gcloud/google"
version = "~> 0.1"

platform = "linux"
additional_components = ["kubectl", "beta"]

create_command = "${module.gcloud.gcloud} version"
milesmatthias marked this conversation as resolved.
Show resolved Hide resolved
destroy_command = "${module.gcloud.kubectl} cp ..."
}
```

Functional examples are included in the
[examples](./examples/) directory.

<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|:----:|:-----:|:-----:|
| additional\_components | Additional gcloud CLI components to install. Defaults to none. Valid value are components listed in `gcloud components list` | list | `<list>` | no |
| create\_command | On create, the command you'd like to run with the GCP SDK. | string | `"true"` | no |
| destroy\_command | On destroy, the command you'd like to run with the GCP SDK. | string | `"true"` | no |
| platform | Platform CLI will run on. Defaults to linux. Valid values: linux, darwin | string | `"linux"` | no |
| service\_account\_key\_file | Path to service account key file to run `gcloud auth activate-service-account` with. Optional. | string | `""` | no |
| use\_tf\_google\_credentials\_env\_var | Use GOOGLE_CREDENTIALS environment variable to run `gcloud auth activate-service-account` with. Optional. | string | `"false"` | no |

## Outputs

| Name | Description |
|------|-------------|
| bq | Path to bq CLI |
| gcloud | Path to gcloud CLI |
| gcloud\_bin\_path | Path to gcloud bin path for use to locate any other components |
| gsutil | Path to gsutil CLI |
| kubectl | Path to kubectl CLI. Must be installed first using additional_components |

<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->

## Requirements

These sections describe requirements for using this module.

### Software

The following dependencies must be available:

- [Terraform][terraform] v0.12
- [Terraform Provider for GCP][terraform-provider-gcp] plugin v2.0

### Service Account
milesmatthias marked this conversation as resolved.
Show resolved Hide resolved

A service account must be created, along with a key, to use this module.
The service account must have the proper IAM roles for whatever
commands you're running with this module.

### APIs
milesmatthias marked this conversation as resolved.
Show resolved Hide resolved

A project is not required to host resources of this module, since
this module does not create any resources.

However you will likely need a project for your service account
and any resources you'd like to interact with while using this module.

## Contributing

Refer to the [contribution guidelines](./CONTRIBUTING.md) for
information on contributing to this module.

[iam-module]: https://registry.terraform.io/modules/terraform-google-modules/iam/google
[project-factory-module]: https://registry.terraform.io/modules/terraform-google-modules/project-factory/google
[terraform-provider-gcp]: https://www.terraform.io/docs/providers/google/index.html
[terraform]: https://www.terraform.io/downloads.html

Loading