Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add support for set env vars inside hook runtime #408

Merged
merged 4 commits into from
Jul 5, 2022
Merged

feat: Add support for set env vars inside hook runtime #408

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4da99c6
feat: Add support for set env vars inside hook runtime
MaxymVlasov Jul 5, 2022
950b809
Remove debug
MaxymVlasov Jul 5, 2022
3583656
Apply suggestions from code review
MaxymVlasov Jul 5, 2022
1545bd7
pre-commit fixes
Jul 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 20 additions & 13 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ If you are using `pre-commit-terraform` already or want to support its developme
* [Available Hooks](#available-hooks)
* [Hooks usage notes and examples](#hooks-usage-notes-and-examples)
* [All hooks: Usage of environment variables in `--args`](#all-hooks-usage-of-environment-variables-in---args)
* [All hooks: Set env vars inside hook runtime](#all-hooks-set-env-vars-inside-hook-runtime)
MaxymVlasov marked this conversation as resolved.
Show resolved Hide resolved
* [checkov (deprecated) and terraform_checkov](#checkov-deprecated-and-terraform_checkov)
* [infracost_breakdown](#infracost_breakdown)
* [terraform_docs](#terraform_docs)
Expand Down Expand Up @@ -283,6 +284,22 @@ Config example:

If for config above set up `export CONFIG_NAME=.tflint; export CONFIG_EXT=hcl` before `pre-commit run`, args will be expanded to `--config=.tflint.hcl --module`.

### All hooks: Set env vars inside hook runtime
MaxymVlasov marked this conversation as resolved.
Show resolved Hide resolved

> All, except deprecated hooks: `checkov`, `terraform_docs_replace`

You can specify environment variables that will be passed to the hook runtime.
MaxymVlasov marked this conversation as resolved.
Show resolved Hide resolved

Config example:

```yaml
- id: terraform_validate
args:
- --envs=AWS_DEFAULT_REGION="us-west-2"
- --envs=AWS_ACCESS_KEY_ID="anaccesskey"
- --envs=AWS_SECRET_ACCESS_KEY="asecretkey"
Comment on lines +298 to +300
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed privately we'll get back to reconsidering this option's name later (e.g. to --env-vars or something like that)

```

### checkov (deprecated) and terraform_checkov

> `checkov` hook is deprecated, please use `terraform_checkov`.
Expand Down Expand Up @@ -614,25 +631,15 @@ Example:
- --args=-no-color
```

2. `terraform_validate` also supports custom environment variables passed to the pre-commit runtime:

```yaml
- id: terraform_validate
args:
- --envs=AWS_DEFAULT_REGION="us-west-2"
- --envs=AWS_ACCESS_KEY_ID="anaccesskey"
- --envs=AWS_SECRET_ACCESS_KEY="asecretkey"
```

3. `terraform_validate` also supports passing custom arguments to its `terraform init`:
2. `terraform_validate` also supports passing custom arguments to its `terraform init`:

```yaml
- id: terraform_validate
args:
- --tf-init-args=-lockfile=readonly
```

4. It may happen that Terraform working directory (`.terraform`) already exists but not in the best condition (eg, not initialized modules, wrong version of Terraform, etc.). To solve this problem, you can find and delete all `.terraform` directories in your repository:
3. It may happen that Terraform working directory (`.terraform`) already exists but not in the best condition (eg, not initialized modules, wrong version of Terraform, etc.). To solve this problem, you can find and delete all `.terraform` directories in your repository:

```bash
echo "
Expand All @@ -648,7 +655,7 @@ Example:

**Warning:** If you use Terraform workspaces, DO NOT use this workaround ([details](https://github.com/antonbabenko/pre-commit-terraform/issues/203#issuecomment-918791847)). Wait to [`force-init`](https://github.com/antonbabenko/pre-commit-terraform/issues/224) option implementation.

5. `terraform_validate` in a repo with Terraform module, written using Terraform 0.15+ and which uses provider `configuration_aliases` ([Provider Aliases Within Modules](https://www.terraform.io/language/modules/develop/providers#provider-aliases-within-modules)), errors out.
4. `terraform_validate` in a repo with Terraform module, written using Terraform 0.15+ and which uses provider `configuration_aliases` ([Provider Aliases Within Modules](https://www.terraform.io/language/modules/develop/providers#provider-aliases-within-modules)), errors out.

When running the hook against Terraform code where you have provider `configuration_aliases` defined in a `required_providers` configuration block, terraform will throw an error like:
>
Expand Down
33 changes: 32 additions & 1 deletion hooks/_common.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ function common::initialize {
# ARGS (array) arguments that configure wrapped tool behavior
# HOOK_CONFIG (array) arguments that configure hook behavior
# TF_INIT_ARGS (array) arguments for `terraform init` command
# ENVS (array) environment variables that will be with all
# 3rd-party tools that will read them.
MaxymVlasov marked this conversation as resolved.
Show resolved Hide resolved
# FILES (array) filenames to check
# Arguments:
# $@ (array) all specified in `hooks.[].args` in
Expand All @@ -37,9 +39,11 @@ function common::parse_cmdline {
ARGS=() HOOK_CONFIG=() FILES=()
# Used inside `common::terraform_init` function
TF_INIT_ARGS=()
# Used inside `common::export_provided_env_vars` function
ENVS=()

local argv
argv=$(getopt -o a:,h:,i: --long args:,hook-config:,init-args:,tf-init-args: -- "$@") || return
argv=$(getopt -o a:,h:,i:,e: --long args:,hook-config:,init-args:,tf-init-args:,envs: -- "$@") || return
eval "set -- $argv"

for argv; do
Expand All @@ -60,6 +64,11 @@ function common::parse_cmdline {
TF_INIT_ARGS+=("$1")
shift
;;
-e | --envs)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@antonbabenko Any ideas on how it can be better named?

Main idea:

You can specify environment variables that will be passed to the hook runtime.

Config example for now

- id: terraform_validate
  args:
    - --envs=AWS_DEFAULT_REGION="us-west-2"
    - --envs=AWS_ACCESS_KEY_ID="anaccesskey"
    - --envs=AWS_SECRET_ACCESS_KEY="asecretkey"

Copy link
Collaborator Author

@MaxymVlasov MaxymVlasov Jul 5, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@yermulnik proposed --env-vars, but I don't think that much better option, to introduce it and then remove --envs in 2.0.0

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

env-vars sounds perfect to me

shift
ENVS+=("$1")
shift
;;
--)
shift
# shellcheck disable=SC2034 # Variable is used
Expand Down Expand Up @@ -270,3 +279,25 @@ function common::terraform_init {

return $exit_code
}

#######################################################################
# Export provided K/V as environment variables.
# Arguments:
# env_vars (array) environment variables that will be with all
# 3rd-party tools that will read them.
MaxymVlasov marked this conversation as resolved.
Show resolved Hide resolved
#######################################################################
function common::export_provided_env_vars {
local -a -r env_vars=("$@")

local var
local var_name
local var_value

for var in "${env_vars[@]}"; do
var_name="${var%%=*}"
var_value="${var#*=}"
# shellcheck disable=SC2086
export $var_name="$var_value"
done
exit 1
MaxymVlasov marked this conversation as resolved.
Show resolved Hide resolved
}
1 change: 1 addition & 0 deletions hooks/infracost_breakdown.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
infracost_breakdown_ "${HOOK_CONFIG[*]}" "${ARGS[*]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_checkov.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# Support for setting PATH to repo root.
# shellcheck disable=SC2178 # It's the simplest syntax for that case
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_docs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# Support for setting relative PATH to .terraform-docs.yml config.
# shellcheck disable=SC2178 # It's the simplest syntax for that case
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_fmt.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_providers_lock.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_tflint.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# Support for setting PATH to repo root.
# shellcheck disable=SC2178 # It's the simplest syntax for that case
Expand Down
1 change: 1 addition & 0 deletions hooks/terraform_tfsec.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# Support for setting PATH to repo root.
# shellcheck disable=SC2178 # It's the simplest syntax for that case
Expand Down
67 changes: 2 additions & 65 deletions hooks/terraform_validate.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,73 +12,13 @@ export AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION:-us-east-1}

function main {
common::initialize "$SCRIPT_DIR"
parse_cmdline_ "$@"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars

# Export provided env var K/V pairs to environment
local var var_name var_value
for var in "${ENVS[@]}"; do
var_name="${var%%=*}"
var_value="${var#*=}"
# shellcheck disable=SC2086
export $var_name="$var_value"
done

# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
}

#######################################################################
# Parse args and filenames passed to script and populate respective
# global variables with appropriate values
# Globals (init and populate):
# ARGS (array) arguments that configure wrapped tool behavior
# HOOK_CONFIG (array) arguments that configure hook behavior
# TF_INIT_ARGS (array) arguments to `terraform init` command
# ENVS (array) environment variables that will be used with
# `terraform` commands
# FILES (array) filenames to check
# Arguments:
# $@ (array) all specified in `hooks.[].args` in
# `.pre-commit-config.yaml` and filenames.
#######################################################################
function parse_cmdline_ {
declare argv
argv=$(getopt -o e:i:a:h: --long envs:,tf-init-args:,init-args:,args: -- "$@") || return
eval "set -- $argv"

for argv; do
case $argv in
-a | --args)
shift
ARGS+=("$1")
shift
;;
-h | --hook-config)
shift
HOOK_CONFIG+=("$1;")
shift
;;
# TODO: Planned breaking change: remove `--init-args` as not self-descriptive
-i | --init-args | --tf-init-args)
shift
TF_INIT_ARGS+=("$1")
shift
;;
-e | --envs)
shift
ENVS+=("$1")
shift
;;
--)
shift
FILES=("$@")
break
;;
esac
done
}

#######################################################################
# Unique part of `common::per_dir_hook`. The function is executed in loop
# on each provided dir path. Run wrapped tool with specified arguments
Expand Down Expand Up @@ -120,7 +60,4 @@ function per_dir_hook_unique_part {
return $exit_code
}

# global arrays
declare -a ENVS

[ "${BASH_SOURCE[0]}" != "$0" ] || main "$@"
1 change: 1 addition & 0 deletions hooks/terraform_wrapper_module_for_each.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars

check_dependencies
Expand Down
1 change: 1 addition & 0 deletions hooks/terragrunt_fmt.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/terragrunt_validate.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/terrascan.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down
1 change: 1 addition & 0 deletions hooks/tfupdate.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
function main {
common::initialize "$SCRIPT_DIR"
common::parse_cmdline "$@"
common::export_provided_env_vars "${ENVS[@]}"
common::parse_and_export_env_vars
# shellcheck disable=SC2153 # False positive
common::per_dir_hook "${ARGS[*]}" "$HOOK_ID" "${FILES[@]}"
Expand Down