enable use of provider functions in the Terraform language #34394
Conversation
jbardin
force-pushed
the
jbardin/provider-functions-lang
branch
from
8267098
to
789f31c
Compare
jbardin
force-pushed
the
jbardin/provider-functions-lang
branch
from
789f31c
to
95763a2
Compare
This essentially doubles up the registrations for all of the built-in functions to have both non-namespaced and core::-namespaced versions of each function. This is in preparation for later commits introducing other namespaces, such as a provider:: namespace which could hold functions that were contributed by providers that are currently in scope.
The jsonfunction package has special cases for some internal functions, and wee need to add the namespaced equivalents to those lists.
This is intended as a bridge to functions defined outside of the core language, which will live in other namespaces in order to avoid ambiguity with any built-in functions. For now we only accept provider-contributed external functions. The lang package doesn't really know anything special about providers, so its only responsibility here is to enforce the naming scheme that any provider-contributed functions always live under the namespace prefix "provider::NAME::", where NAME is a name decided by whatever codepath instantiated the state as a unique identifier for a particular provider. This commit also includes updates to the core package to make it possible to pass in ExternalFuncs in principle, although in practice we don't yet have any codepaths which do so, and so this is effectively a no-op commit.
We need to be able to reuse providers which are used for function calls, so cache them in the EvalContext. This does require a way to close the provider however, which we can do via the normal root module close node since it already does the same for provisioners. Update the `CloseProvisioners` method to be a more general `ClosePlugins` method, and use that to call Close on the cached providers too.
This establishes all of the wiring necessary for a provider's declared functions to appear in the hcl.EvalContext when we evaluate expressions inside a module that depends on a provider that contributes functions. This does not yet make any attempts to guarantee that the provider-contributed functions correctly honor the contracts such as behaving as pure functions. Properly checking this is important because if a function doesn't uphold Terraform's expectations then it will cause confusing errors reported downstream, incorrectly blaming other components for the inconsistency.
Now that we dynamically add scoped versions to the global functions we can't check the static Descriptions list. This should be OK since the function descriptions are taken directly from Descriptions list, and if any other scoped functions were added in the future, they may not be coming from the current global set of functions anyway.
jbardin
force-pushed
the
jbardin/provider-functions-lang
branch
from
95763a2
to
1644c2f
Compare
| @@ -12,7 +12,7 @@ import ( | |||
| ) | |||
|
|
|||
| var ( | |||
| ignoredFunctions = []string{"map", "list"} | |||
| ignoredFunctions = []string{"map", "list", "core::map", "core::list"} | |||
There was a problem hiding this comment.
Just noting here because this change reminded me of this command: do we intend in a later PR to expand metadata functions to include the functions offered by installed providers?
There was a problem hiding this comment.
metadata functions doesn't require an initialized working directory, so unfortunately provider functions are not going to be part of that output. Since that is mainly for the language server integration, it's yet to be determined if a separate command to list providers functions is needed (they are fetching schemas anyways if providers are available, which include functions), or maybe having inconsistent output from metadata is actually OK.
| condition = (local.result == 3) | ||
| error_message = "Wrong number of Es in my cheese." | ||
| } | ||
| } |
There was a problem hiding this comment.
Very good example of a useful provider function.
|
Reminder for the merging maintainer: if this is a user-visible change, please update the changelog on the appropriate release branch. |
|
I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. |
Enable the use of provider-defined functions within the
provider::scope.Providers can now implement functions which can be used from within the Terraform configuration language. The schema for each function is defined via the provider schema, using a layout similar to that of
ctyfunction definitions already in use internally. All provider functions found via their schemas are added to theprovider::namespace, using the provider's local name as defined in the module'srequired_providers. This means that anoopfunction declared by theawsprovider would be access asprovider::aws::noop(). The use of the::delimiter as a function scoping mechanism was already added to the updatedhclpackage.While providers must be trusted to ensure their functions only operate "offline", Terraform helps enforce this by only calling provider functions on an un-configured provider instance. This special provider instance will be started on demand, and cached for the remainder of the evaluation run. Having a separate provider is also required because normal resource providers have a lifecycle tied to their respective provider graph node, however function evaluation can happen from any node in the configuration so a separate instance must be waiting for any possible calls.
One important validation is not yet implemented, which is checking the purity of the function implementations. Checking for side effects of course is not possible, so again we must trust the providers implementations, but we do intend to validate the result values for consistency.