Allow defining relationships in config schemas

[radeksimko]

Given the following piece of code in helper/schema/schema.go

func (m schemaMap) Input(
    input terraform.UIInput,
    c *terraform.ResourceConfig) (*terraform.ResourceConfig, error) {
    keys := make([]string, 0, len(m))
    for k, _ := range m {
        keys = append(keys, k)
    }
    sort.Strings(keys)

    for _, k := range keys {
...

all config options are sorted alphabetically and treated independently in the prompt.
I can imagine cases where I would like the user to enter some config options first and eventually not ask for some other options based on the entered data.

Example?
Imagine following AWS provider config:

• credentials_provider (detect | iam | env | static)
• credentials_file_path
• credentials_file_profile_name
• access_key
• secret_key
• security_token

It is obvious that:

1. It would be annoying to ask user for all of these
2. Some options with certain values are making other ones superfluous (e.g. credentials_provider = detect, iam or env means that I should ignore all other options)

Would there be interest in such feature? If not, how would you suggest solving the problem with dependent config options above?

Related: hashicorp/terraform#1049

[ketzacoatl]

In SaltStack, state files (formula, sort of like .tf resource definitions) are rendered as a template before being applied. In reading the source as a template, we can use all the power provided by the package rendering the template. For example, by default SaltStack uses the jinja renderer, so templates have all the power of jinja in them. In this way, we can easily create and manipulate data structures internal to the template and otherwise used in rendering the template. Using Terraform's count makes me want conditionals, iteration, and more powerful capabilities such as those in jinja, in Terraform's resource definitions. FWIW.

[radeksimko]

@ketzacoatl How does Jinja solution apply to this problem of dependencies between config options? Any example you can provide so we can be inspired?

[phinze]

@radeksimko reviewing some old issues here. We still don't have a facility for this, but I feel like the "user enters input interactively" use case is more or less a "first Terraform experience" sort of feature. Once you're using Terraform in anger, you're almost definitely configuring provider in some other way.

I wonder if it'd be easier to just make the most common provider configuration the interactive ones, rather than try to represent the tree of possible options in a sane way interactively. What do you think?

[radeksimko]

use case is more or less a "first Terraform experience" sort of feature. Once you're using Terraform in anger, you're almost definitely configuring provider in some other way.

I wouldn't be so sure. It may be the case now purely because of the way we deal with entered data (just throw it away after apply/destroy basically). If we'd be saving this anywhere, I think the story would differ (i.e. people would be using tfvars and defaults less than they do now).

Related: hashicorp/terraform#2550

I wonder if it'd be easier to just make the most common provider configuration the interactive ones, rather than try to represent the tree of possible options in a sane way interactively.

So you mean basically having a very specific codified tree for provider configuration, but not try to solve the problem in runtime? Sounds like an interesting idea and probably solves the example I gave above. 👍

[phinze]

If we'd be saving this anywhere, I think the story would differ (i.e. people would be using tfvars and defaults less than they do now).

Good point! I hadn't thought about this. 👍

So yeah it's easy to picture how a prompt opt-in or opt-out config boolean would work, but I can't picture how a "subset A or subset B" prompt config would work.

Today's behavior is that we prompt for the superset yeah? It'd be tricky to take away that behavior without surprising users who might be pressing <enter> a lot to get to the combination of attributes they use.

Hmm... more [thinking] necessary. 😀

[radeksimko]

Agreed it's not trivial, otherwise I would have came up with a PR instead of an issue already 😃
I don't expect this to be solved in two lines of code.

Maybe we should think of inputs similar way we think about resources? i.e. in graphs? I'm not sure whether it helps or whether it makes the problem more complicated... I guess both.

either way... https://twitter.com/mitchellh/status/648506349807013888

[phinze]

Maybe we should think of inputs similar way we think about resources? i.e. in graphs?

It's become a running HashiCorp joke that given any problem, eventually a graph will find its way somewhere into the solution. 😀

Your points are well taken though - will continue to think about this.

[apparentlymart]

Just to throw in my two cents: in my world we've reduced user input to a bare minimum by deriving almost everything from data obtained from data-fetching resources like terraform_remote_state and consul_keys.

I'm personally more interested in fixing the warts surrounding that workflow and de-emphasizing direct configuration, so I agree with what @phinze said earlier about the user inputs being more of a "welcome to Terraform" flow than something I'd want to use in a "real" scenario, because I find it much more powerful to compute/derive these values than to hand-wire them, even if my hand wiring would end up stored in a file somewhere.

The only case where we ever use per-run user-supplied variables is to allow a single application config to be deployed separately to multiple environments, like I was describing in my comment over in hashicorp/terraform#1964 a while back. We just have some root configs that have the provider configurations hard-coded (we use environment variables for credentials, so this just includes location-ish things like AWS region) and then all other configurations descend in some way from those root configs.

By all of this I'm not meaning to argue that we shouldn't improve the UX around user inputs like @radeksimko originally suggested... just hoping to throw another perspective into the mix in case it helps to think about the overall problem.

[scalp42]

The issue is that you don't want to rely on Consul to have Terraform working though (chicken/egg issue).

[apparentlymart]

Indeed... we use terraform_remote_state exclusively for the first bunch of layers until we get to the layer that creates the consul cluster. After that we use a mixture of consul_keys and terraform_remote_state, depending on whether the configuration also needs to be visible to the nodes at runtime... the nodes can of course only see what's persisted in Consul, not what was in the Terraform config.

[bflad]

Hey @radeksimko 👋 Do you think we should still keep this lingering issue around? I think Terraform by convention has generally leaned towards non-interactive usage and validating configuration "all at once" rather than piecemeal.

Regarding the proposal, I'm not sure it necessarily belongs in the SDK repository unless there's a more detailed proposal involving the likely protocol changes to support provider schema information offering certain types of suggestions back to CLI, which could then be potentially used in the user interface.

[radeksimko]

Hello to Radek 2015 and Brian 2021 👋🏻
The transfer of the issue to the SDK may have been slightly misguided from my side - you're right that gRPC/SDK isn't responsible for UI aspects at all, Terraform CLI/Core is.

The use case I described above with AWS authentication probably isn't as strong anymore - the overall auth UX has probably improved significantly and most methods involve ENV variables which entirely avoids the UI/input problem. That said there's still semi-related problem with (expiring) MFA.

I can still imagine the underlying problem being valid, just in other use cases. The SDK already has a mechanism such as ConflictsWith which isn't communicated over the gRPC protocol and therefore can't be used anywhere outside of provider/SDK.

The definition of "required" fields is somewhat blurry as a result of this, which also affects the UX in editors. We recently introduced prefilling of required fields in hashicorp/vscode-terraform#799 and users have noticed that even after the initial pre-fill there may still be fields which were not pre-filled because they are marked as "Optional" which is all we know from the protocol perspective.

Maybe this is less related to the original issue here and maybe there's a better place to discuss it, but I do believe we need some way of expressing "conditionally required fields".

I will let you judge the best course of actions here, but hopefully the above helps you make the right decision.