"terraform validate" should not fail on implicit provider config when provider arguments are required #21408
Labels
Comments
|
Hi @plum117! Thanks for reporting this. Indeed, the documentation is missing a note to say that The option is removed from Terraform v0.12 because So with that said, for any situation where you would've run We'll get the documentation updated to mention that |
|
Thanks for the info I tried New behavior Old behavior: |
|
Thanks for the additional context, @plum117. Given the error message you saw here, it looks like you've found a bug in From the wording of the message it sounds like the problem may be inside a |
|
@apparentlymart below is the output (with trace) of the validate commands between 0.11.4 and 0.12.0. On 0.11.14 version, the command do not seems to do that much... See next post for the remaining log |
|
Remaining log of |
|
Let me know if there is other information that could help you |
|
It sounds like there was a missing validation rule for this in Terraform v0.11 and so it was failing to detect it. Terraform v0.12 validate is a more complete validation with a full syntax check and type check, so it will find additional errors that Terraform v0.11 might not have caught. As far as I can tell, from what you've shared so far it seems like the new behavior is correct and the old behavior was incorrect: your configuration contains an error which Terraform v0.11 was incorrectly failing to report. If you can share your configuration then I can try to reproduce it myself and confirm whether that is the case. |
|
Hi. Is it appropriate to be running For all of my modules, I'm getting the same error: I can show you any of the modules. OOI, if I take my 0.12 upgraded module and place it in the It would be nice to be able to validate modules as part of the gitlab pipeline for the module (it was like that for 0.11, so we'd like to carry on with this). |
|
Hi @rquadling! I'm going to answer your terminology questions first because I think that'll make my subsequent answer easier to follow:
When it comes to re-usable modules that are used only as child modules, the developer of such a module will be working on it outside of the context of any particular caller, so during development the child module is treated like a root module of a configuration: check out its VCS repository separately, run Running For modules that have complex dependencies that also need to be created, a more advanced pattern is to create a You can see that, for example, in the I don't have any automated CI set up for that repository right now, but if I did I'd run |
|
Thanks for all of the context @apparentlymart, it's super helpful especially as we work to upgrade to v0.12! I'm experiencing the same errors as @plum117 and @rquadling with the terraform {
required_version = ">= 0.12.0"
}
resource "aws_lb" "test" {}and I run It appears that this is due to the fact that the AWS provider requires a region to be set. If I add the following to the provider "aws" {
version = "~> 2.12.0"
region = "us-east-1"
}then While this workaround gets the job done, it's a little confusing as the documentation on modules says the following:
but it appears that we can't successfully use |
|
Ahh, okay! I was going down the wrong path because we were talking about Terraform is in a bit of a tough spot here because it's trying to validate the provider configurations against the provider schema, and the provider schema says I think the trick here will be to introduce a new subtlety into the validation: the There are a few tricks to this we need to watch out for:
For the moment, the workaround I'd suggest is to set the (Sorry @plum117 that I was misunderstanding the source of the problem you were having here. The change I described above should solve the problem you are seeing too.) |
apparentlymart
changed the title
|
Ahhh, makes sense. Thanks @apparentlymart! |
|
Thank you for the details. Extremely useful for us. Inherited a working system from a defunct provider. Learning a lot as we go. |
3 tasks
This was referenced Jun 12, 2019
namusyaka
added a commit
to terraform-google-modules/terraform-google-pubsub
that referenced
this issue
Jun 15, 2019
Now the validator behaves as if this were `false`. See: hashicorp/terraform#21408 (comment) However, as a result of my investigation, the validator uses the source module to check whether the syntax is valid and types/values are being used correctly. To react with the changes, this commit adds PWD=directory as an temporary environment variable on the command execution, because validate command seems to refer to the current directory.
|
I don't understand why a required variable in a provider is required for validation in the first place. This sounds like a provider issue, where e.g. the AWS provider should not require a region if only being used for "terraform validate". As it stands, this type of configuration would never work with an IDE for "running on save". It makes no sense to require live variables for a provider if that provider is not going to be doing anything that would require that variable. e.g. if I'm on a box doing development, I'm not in a situation where I could run terraform plan / apply at all. The last thing I want is for the providers to get initialized in a way that uses real values. |
|
IMO missing required provider arguments should be a WARNING not an ERROR. The difference is simple: If something may not go perfectly unless the run time environment remedies it, that's a warning. Error's should be things that will always be wrong. |
maartenvanderhoef
added a commit
to doingcloudright/terraform-aws-ecr-cross-account
that referenced
this issue
Sep 23, 2019
namusyaka
added a commit
to terraform-google-modules/terraform-google-pubsub
that referenced
this issue
Oct 9, 2019
Now the validator behaves as if this were `false`. See: hashicorp/terraform#21408 (comment) However, as a result of my investigation, the validator uses the source module to check whether the syntax is valid and types/values are being used correctly. To react with the changes, this commit adds PWD=directory as an temporary environment variable on the command execution, because validate command seems to refer to the current directory.
aj-jaswanth-ee
added a commit
to PredictiveHire/terraform-aws-wireguard
that referenced
this issue
Oct 14, 2019
6 tasks
|
There doesn't seem to be a workaround if the provider does not allow using env vars to override/set arguments. |
This was referenced Jan 17, 2020
|
@apparentlymart any update on when we could see this issue fixed ? the new AzureRM provider release introduced a new required provider setting which is causing a lot of headache to users / developper who's modules are relying on azurerm resources. |
|
It is quite annoying having to pass in API Keys and access tokens to a Please change this "error" to a warning! |
|
I've posted a proposed fix in #24896. A lot of the comments here informed what I've written up there, particularly v2 of the Azure provider which I'm seeing for the first time. I do want to emphasize that even if accepted this would mean that you'd have the option to specify:
However, once you specify any provider configuration that's not an alias or version, you'll have to satisfy the whole schema. Required fields will still be required. Providers play a role here too. Essential configuration should all have corresponding environment variables. This helps ensure that most provider blocks are empty/undeclared for users who are on Terraform Cloud or really any server environment where it's convenient/required to pass hostnames and credentials to a provider. Providers can make credentials optional and check them in This works in all the private projects/modules I work on with ~10 different providers but depending on how you configure providers you may need to make some updates. |
|
@bendrucker That's a good direction definitely. It would also be great notation for actually documenting how our providers are configured for new people :) |
|
Thanks, I think it’s extremely unlikely that you’ll see an API like that for specifying that provider config will come from the environment. Terraform has no visibility into whether a provider field uses environment variables for default nor which variables it’ll use, that’s all part of a function on the provider side. The number of changes required to make this possible are significant, both in providers and core. That’s why I’m suggesting that providers play an important role here in ensuring that “all or nothing” config is possible. For AWS, for example, no credentials by the provider until runtime, just a |
9 tasks
This was referenced Jun 5, 2020
This was referenced Oct 2, 2020
|
As @apparentlymart has outlined specifying But what's the equivalent for Azure modules which require the Meanwhile I've created a workaround we use in Bitbucket Pipelines. Maybe this is suitable for one of you. image: hashicorp/terraform:light
options:
docker: true
definitions:
steps:
- step: &terraform-validate
name: Terraform validate
caches:
- docker
script:
# Workaround for error: "features": required field is not set
- echo -e '\nprovider "azurerm" { \n features {} \n}' >> versions.tf
- terraform init -backend=false
- terraform validate
pipelines:
default: &default
- step: *terraform-validate |
|
@pselle The team that maintains the I was just wondering if you knew - since this issue has now been merged/closed, when can we expect it to be released? cc @manicminer @tombuildsstuff - Is there anything else that needs to be done on the |
|
This fix was merged into 0.15 #24896 (comment), you can test with the 0.15 alpha builds if you'd like to help triple check that it does ameliorate the issue, or we can re-open if it doesn't. |
|
I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. |
ghost
locked as resolved and limited conversation to collaborators
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
I ran the command
terraform validate -check-variables=falseand I got the below output. It seems like the command is not well format.However, when I run
terraform validatethen the command seems fine and I have the expected output :Error: Missing required argument. The argument "region" is required, but was not set.Does the validate command actually changed since
terraform validate --helpdo not show-check-variables=falseas being a possible option. At the same time, the website seems saying otherwise: https://www.terraform.io/docs/commands/validate.htmlMaybe the doc on the web site needs a small update. Thanks
terraform validate -check-variables=false
2019/05/23 10:32:07 [INFO] Terraform version: 0.12.0
2019/05/23 10:32:07 [INFO] Go runtime version: go1.12.4
2019/05/23 10:32:07 [INFO] CLI args: []string{"/home/dprevost/.local/bin/terraform", "validate", "-check-variables=false"}
2019/05/23 10:32:07 [DEBUG] Attempting to open CLI config file: /home/dprevost/.terraformrc
2019/05/23 10:32:07 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/05/23 10:32:07 [INFO] CLI command args: []string{"validate", "-check-variables=false"}
Usage: terraform validate [options] [dir]
Validate the configuration files in a directory, referring only to the
configuration and not accessing any remote services such as remote state,
provider APIs, etc.
Validate runs checks that verify whether a configuration is
internally-consistent, regardless of any provided variables or existing
state. It is thus primarily useful for general verification of reusable
modules, including correctness of attribute names and value types.
It is safe to run this command automatically, for example as a post-save
check in a text editor or as a test step for a re-usable module in a CI
system.
Validation requires an initialized working directory with any referenced
plugins and modules installed. To initialize a working directory for
validation without accessing any configured remote backend, use:
terraform init -backend=false
If dir is not specified, then the current directory will be used.
To verify configuration in the context of a particular run (a particular
target workspace, operation variables, etc), use the terraform plan
subcommand instead, which includes an implied validation check.
Options:
-json Produce output in a machine-readable JSON format, suitable for
use in e.g. text editor integrations.
The text was updated successfully, but these errors were encountered: