fix issues with version-specific tests when run against a local binary #359
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When running e2e tests against a locally built Terraform binary, the runTestWithVersions helper has some surprising behavior: it throws away the selected versions and runs the test against the local binary instead. This seems fairly undesirable, since the tests using that helper are testing version-specific behavior (and thus fail when a local bin path is set). This commit clarifies some naming and ensures that version override behavior is confined to `runTest`, which makes no assumptions about which versions it's using.
radditude
changed the title
kmoe
approved these changes
Feb 9, 2023
There was a problem hiding this comment.
Thanks @radditude, this is much clearer!
I'm going to merge this now as I think the e2e failures against Terraform main/1.4 are resolved by #361.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When running e2e tests against a locally built Terraform binary set with
TFEXEC_E2ETEST_TERRAFORM_PATH, therunTestWithVersionshelper (introduced in #354) has some surprising behavior: it throws away the list of versions that was passed in and runs the test against the local binary instead. This seems fairly undesirable and was likely not an intended effect of that prior PR, since the tests using that helper are testing version-specific behavior (and thus are prone to fail when a local bin path is set!).This PR proposes moving that version override behavior into
runTest, where it was previously, and combiningrunTestWithVersionsandrunTestVersionsinto one test helper that runs the given test function against only the versions passed in.Closes hashicorp/terraform#32639
Potential future enhancement:
If it would be valuable to have the ability to run some tests against a specified list of versions as well as the local binary if it exists, probably a clearer way to do that would be to add something like a
runTestWithVersionsAndLatesthelper that reads fromTFEXEC_E2ETEST_TERRAFORM_PATHand appends the local binary's version instead of overwriting the others.