Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for set env vars inside hook runtime #408

Merged
merged 4 commits into from
Jul 5, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 20 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ If you are using `pre-commit-terraform` already or want to support its developme
* [Available Hooks](#available-hooks)
* [Hooks usage notes and examples](#hooks-usage-notes-and-examples)
* [All hooks: Usage of environment variables in `--args`](#all-hooks-usage-of-environment-variables-in---args)
* [All hooks: Set env vars inside hook at runtime](#all-hooks-set-env-vars-inside-hook-at-runtime)
* [checkov (deprecated) and terraform_checkov](#checkov-deprecated-and-terraform_checkov)
* [infracost_breakdown](#infracost_breakdown)
* [terraform_docs](#terraform_docs)
Expand Down Expand Up @@ -283,6 +284,22 @@ Config example:
If for config above set up `export CONFIG_NAME=.tflint; export CONFIG_EXT=hcl` before `pre-commit run`, args will be expanded to `--config=.tflint.hcl --module`.

### All hooks: Set env vars inside hook at runtime

> All, except deprecated hooks: `checkov`, `terraform_docs_replace`

You can specify environment variables that will be passed to the hook at runtime.

Config example:

```yaml
- id: terraform_validate
args:
- --envs=AWS_DEFAULT_REGION="us-west-2"
- --envs=AWS_ACCESS_KEY_ID="anaccesskey"
- --envs=AWS_SECRET_ACCESS_KEY="asecretkey"
Comment on lines +298 to +300
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed privately we'll get back to reconsidering this option's name later (e.g. to --env-vars or something like that)

```

### checkov (deprecated) and terraform_checkov

> `checkov` hook is deprecated, please use `terraform_checkov`.
Expand Down Expand Up @@ -614,25 +631,15 @@ Example:
- --args=-no-color
```

2. `terraform_validate` also supports custom environment variables passed to the pre-commit runtime:

```yaml
- id: terraform_validate
args:
- --envs=AWS_DEFAULT_REGION="us-west-2"
- --envs=AWS_ACCESS_KEY_ID="anaccesskey"
- --envs=AWS_SECRET_ACCESS_KEY="asecretkey"
```

3. `terraform_validate` also supports passing custom arguments to its `terraform init`:
2. `terraform_validate` also supports passing custom arguments to its `terraform init`:

```yaml
- id: terraform_validate
args:
- --tf-init-args=-lockfile=readonly
```

4. It may happen that Terraform working directory (`.terraform`) already exists but not in the best condition (eg, not initialized modules, wrong version of Terraform, etc.). To solve this problem, you can find and delete all `.terraform` directories in your repository:
3. It may happen that Terraform working directory (`.terraform`) already exists but not in the best condition (eg, not initialized modules, wrong version of Terraform, etc.). To solve this problem, you can find and delete all `.terraform` directories in your repository:

```bash
echo "
Expand All @@ -648,7 +655,7 @@ Example:

**Warning:** If you use Terraform workspaces, DO NOT use this workaround ([details](https://github.com/antonbabenko/pre-commit-terraform/issues/203#issuecomment-918791847)). Wait to [`force-init`](https://github.com/antonbabenko/pre-commit-terraform/issues/224) option implementation.

5. `terraform_validate` in a repo with Terraform module, written using Terraform 0.15+ and which uses provider `configuration_aliases` ([Provider Aliases Within Modules](https://www.terraform.io/language/modules/develop/providers#provider-aliases-within-modules)), errors out.
4. `terraform_validate` in a repo with Terraform module, written using Terraform 0.15+ and which uses provider `configuration_aliases` ([Provider Aliases Within Modules](https://www.terraform.io/language/modules/develop/providers#provider-aliases-within-modules)), errors out.

When running the hook against Terraform code where you have provider `configuration_aliases` defined in a `required_providers` configuration block, terraform will throw an error like:
>
Expand Down
32 changes: 31 additions & 1 deletion hooks/_common.sh
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ function common::initialize {
# ARGS (array) arguments that configure wrapped tool behavior
# HOOK_CONFIG (array) arguments that configure hook behavior
# TF_INIT_ARGS (array) arguments for `terraform init` command
# ENVS (array) environment variables will be available
# for all 3rd-party tools executed by a hook.
# FILES (array) filenames to check
# Arguments:
# $@ (array) all specified in `hooks.[].args` in
Expand All @@ -37,9 +39,11 @@ function common::parse_cmdline {
ARGS=() HOOK_CONFIG=() FILES=()
# Used inside `common::terraform_init` function
TF_INIT_ARGS=()
# Used inside `common::export_provided_env_vars` function
ENVS=()

local argv
argv=$(getopt -o a:,h:,i: --long args:,hook-config:,init-args:,tf-init-args: -- "$@") || return
argv=$(getopt -o a:,h:,i:,e: --long args:,hook-config:,init-args:,tf-init-args:,envs: -- "$@") || return
eval "set -- $argv"

for argv; do
Expand All @@ -60,6 +64,11 @@ function common::parse_cmdline {
TF_INIT_ARGS+=("$1")
shift
;;
-e | --envs)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@antonbabenko Any ideas on how it can be better named?

Main idea:

You can specify environment variables that will be passed to the hook runtime.

Config example for now

- id: terraform_validate
  args:
    - --envs=AWS_DEFAULT_REGION="us-west-2"
    - --envs=AWS_ACCESS_KEY_ID="anaccesskey"
    - --envs=AWS_SECRET_ACCESS_KEY="asecretkey"

Copy link
Collaborator Author

@MaxymVlasov MaxymVlasov Jul 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yermulnik proposed --env-vars, but I don't think that much better option, to introduce it and then remove --envs in 2.0.0

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

env-vars sounds perfect to me

shift
ENVS+=("$1")
shift
;;
--)
shift
# shellcheck disable=SC2034 # Variable is used
Expand Down Expand Up @@ -270,3 +279,24 @@ function common::terraform_init {

return $exit_code
}

#######################################################################
# Export provided K/V as environment variables.
# Arguments:
# env_vars (array) environment variables will be available
# for all 3rd-party tools executed by a hook.
#######################################################################
function common::export_provided_env_vars {
local -a -r env_vars=("$@")

local var
local var_name
local var_value

for var in "${env_vars[@]}"; do
var_name="${var%%=*}"
var_value="${var#*=}"
# shellcheck disable=SC2086
export $var_name="$var_value"
done
}
1 change: 1 addition & 0 deletions hooks/infracost_breakdown.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
infracost_breakdown_ "${HOOK_CONFIG[*]}" "${ARGS[*]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_checkov.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# Support for setting PATH to repo root.
# shellcheck disable=SC2178 # It's the simplest syntax for that case
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_docs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# Support for setting relative PATH to .terraform-docs.yml config.
# shellcheck disable=SC2178 # It's the simplest syntax for that case
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_fmt.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_providers_lock.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_tflint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# Support for setting PATH to repo root.
# shellcheck disable=SC2178 # It's the simplest syntax for that case
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_tfsec.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# Support for setting PATH to repo root.
# shellcheck disable=SC2178 # It's the simplest syntax for that case
Expand Down
67 changes: 2 additions & 65 deletions hooks/terraform_validate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,73 +12,13 @@ export AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION:-us-east-1}

function main {
common::initialize "$SCRIPT_DIR"
parse_cmdline_ "$@"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars

# Export provided env var K/V pairs to environment
local var var_name var_value
for var in "${ENVS[@]}"; do
var_name="${var%%=*}"
var_value="${var#*=}"
# shellcheck disable=SC2086
export $var_name="$var_value"
done

# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
}

#######################################################################
# Parse args and filenames passed to script and populate respective
# global variables with appropriate values
# Globals (init and populate):
# ARGS (array) arguments that configure wrapped tool behavior
# HOOK_CONFIG (array) arguments that configure hook behavior
# TF_INIT_ARGS (array) arguments to `terraform init` command
# ENVS (array) environment variables that will be used with
# `terraform` commands
# FILES (array) filenames to check
# Arguments:
# $@ (array) all specified in `hooks.[].args` in
# `.pre-commit-config.yaml` and filenames.
#######################################################################
function parse_cmdline_ {
declare argv
argv=$(getopt -o e:i:a:h: --long envs:,tf-init-args:,init-args:,args: -- "$@") || return
eval "set -- $argv"

for argv; do
case $argv in
-a | --args)
shift
ARGS+=("$1")
shift
;;
-h | --hook-config)
shift
HOOK_CONFIG+=("$1;")
shift
;;
# TODO: Planned breaking change: remove `--init-args` as not self-descriptive
-i | --init-args | --tf-init-args)
shift
TF_INIT_ARGS+=("$1")
shift
;;
-e | --envs)
shift
ENVS+=("$1")
shift
;;
--)
shift
FILES=("$@")
break
;;
esac
done
}

#######################################################################
# Unique part of `common::per_dir_hook`. The function is executed in loop
# on each provided dir path. Run wrapped tool with specified arguments
Expand Down Expand Up @@ -120,7 +60,4 @@ function per_dir_hook_unique_part {
return $exit_code
}

# global arrays
declare -a ENVS

[ "${BASH_SOURCE[0]}" != "$0" ] || main "$@"
1 change: 1 addition & 0 deletions hooks/terraform_wrapper_module_for_each.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars

check_dependencies
Expand Down
1 change: 1 addition & 0 deletions hooks/terragrunt_fmt.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/terragrunt_validate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/terrascan.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/tfupdate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down