Variable precedence

[cdsre]

Most framework tools allow variables to be set in several way, inside the configuration, from a vars file, from an env var, from a var command line flag etc.

Examples of this are Terraform, Ansible, and Gitlab

Most frameworks also provide a way to set configuration values from ENV vars by prefixing the var with a string such as AWS_, TERRAFORM_, PACKER_, ANSIBLE_, so maybe its work adopting a similar approach for var, or command line switches that drive behaviour. Something like CTK_ or CHAOSTOOLKIT_ I prefer the latter as its more verbose for anyone looking at the system or runtime env vars.

In addition to the above vars which really drive overall behaviour of the process, another strong choice for dynamic env vars is a shorthand prefix. Terraform is another good example of this. For chaos toolkit that could look something like CTK_VAR_some_value=abcde or CTK_SECRET_another_value=shhh. this would allow vars to be set programmatically at the environment level. While I like configuration files and they for sure still have a place, there will be some values that will change from env to env that will want to be set dynamically by the CICD.

some of the above might fundamentally change how configuration is currently set when using a var file. However I think that might need to be reviewed anyway based on this chaostoolkit/chaostoolkit-lib#257 where someone might want to set configuration for control files where that configuration doesn't apply to the experiment its self. Currently the code will only iterate through the experiments configuration and update the values from the external source. There may be a solid design choice reason for this but does feel limiting or at least with controls.

Allow complex values in ENV vars or command line vars. Things like terraform vars and ansible vars. Both of these popular frameworks allow variables to be passed at the command line either as a plain value or a complex value

terraform plan -var 'name=["a", "b", "c"]'
ansible-playbook arcade.yml --extra-vars '{"pacman":"mrs","ghosts":["inky","pinky","clyde","sue"]}'

This discussion is to consider how variables could be set and potentially allow variables to set that are not just in the experiment configuration, this would also help address previously raised issues where you might want a config value like aws_region that only applies to an external control thats not part of the experiment.