ext/dynblock: Preserve marks from for_each expression into result #679
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously if the
for_each
expression was marked then expansion would fail because marked expressions are never directly iterable. (The failure also had a very confusing error message, because it wasn't directly testing whether the value was marked and was just assuming that any non-iterable value is non-iterable because it's of the wrong type.)Now instead we'll allow marked for_each and preserve the marks into the values produced by the resulting block as much as we can.
This runs into the classic problem that HCL blocks are not values themselves and so cannot carry marks directly, but we can at least make sure that the values of any leaf arguments end up marked. We also ensure that applications that use
hcldec
(which includes Terraform) will see marks on the objects that represent the blocks themselves, thereby improving the marking accuracy.This also fixes a nearby bug that I found along the way:
dynblock
would previously panic if the value specified for a block label was marked. It will now return an error message, although the error message is not very good quality because HCL doesn't understand what any marks represent. If this becomes a problem in practice then perhaps in future we'll allow applications to specify their own label value validator functions so that they can customize the error messages, but dynamic labels are so rarely used right now that it doesn't seem worth that complexity.Terraform-specific note: Terraform currently only uses its "sensitive" mark and configures
dynblock
to reject anyfor_each
value that carries that mark, so this change effectively does nothing for today's Terraform because the new codepaths are unreachable anyway.I've implemented this for hashicorp/terraform#35078, because that introduces a new mark called "ephemeral" which represents that the value isn't required to be consistent between plan and apply and is never persisted in the state. It isn't appropriate to reject values with that mark in dynamic block
for_each
, because we want to be able to use ephemeral values in provider configuration blocks and several heavily-used providers use nested blocks in their configuration schemas that might need to vary based on an ephemeral value.(For example, consider an ephemeral input value for providing a JWT to use for AWS's "AssumeRoleWithWebIdentity". A JWT is a good example of something that's likely to vary between plan and apply and so would be declared as ephemeral, but it would be reasonable to use the nullness of that value to choose dynamically whether or not to include the
assume_role_with_web_identity
block, which requires that we allow thefor_each
result to be derived from the ephemeral value and therefore be ephemeral itself.)