SkipFunc and more helpful errors for Terraform versions during testing

[kmoe]

ProtoV6ProviderFactories added in hashicorp/terraform-plugin-sdk#761 requires Terraform v0.15.4 or later. Using earlier versions of Terraform will result in an unhelpful error from the Terraform CLI.

Check the Terraform version being used during acc testing and give a helpful error if incompatible provider factory versions are used.

Also provide a built-in SkipFunc for skipping tests based on Terraform versions.

[alexsomesan]

I'd love to see this too. I need to selectively skip acc tests based on TF version.

I our case, this would have to be available via terraform-plugin-test / terraform-plugin-go

[bflad]

Hi folks 👋

With the introduction of the terraform-plugin-testing module (migration guide), we are transferring feature requests relating to the helper/resource "testing framework" in the terraform-plugin-sdk repository over to the new terraform-plugin-testing repository.

Thanks again for this feature request.

[bflad]

Since SkipFunc does not have access to the terraform-exec being used by the rest of the TestCase, it might be a little awkward to shoe-horn a solution into there to call the proper Terraform binary to call terraform version for that information.

We could consider creating something like:

type TerraformVersionFunc func(version.Version) (bool, error) // returns: whether to t.Skip(), error

type TestCase struct {
  TerraformVersion TerraformVersionFunc
}

If defined, this could enable the testing code to explicitly execute terraform version and pass that information to the function. Since the function signature is generic, it should enable any of these use cases:

• Skipping testing based on version constraints
• Failing testing based on version constraints

Where those version constraints could be written with helpers such as:

• tfversion.All(resource.TerraformVersionFunc...)
• tfversion.SkipAbove(string)
• tfversion.SkipBelow(string)
• tfversion.SkipBetween(string, string)
• tfversion.RequireAbove(string)
• tfversion.RequireBelow(string)
• tfversion.RequireBetween(string, string)

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.