nodeExpandApplyableResource only does expansion related tasks, so remove
the Execute method and move the logic all into DynamicExpand.

Ephemeral resource behavior is a strange mix of temporary values and
existing resource types, and their operation during destroy is going to
be more difficult to manage than normal resources.

The pruneUnusedNodesTransformer is probably no longer equipped to handle
the situation, and will probably need to be build in a different manner,
but this minor change allows us to move forward for  now.

Just like Descendants, Ancestors doesn't need to return an error.

backend/s3: s3 native locking changelog entry

Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>

Ephemeral expansion and add tests for ephemeral values

Apply and destroy evaluation of ephemeral resources (partial)

)

* reflect the Kubernetes 'secret_suffix' validation fix

Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>

* rephrase changelog entry

Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>

---------

Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>

ephemeral variables need to be re-supplied

…g is printed

…n terrafom test

Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>

…l-values

Add ancestor equivalents of FirstDescendantsWith and MatchDescendant

In configurations with long chains of dependencies, we spend a lot of
time recording redundant dependency edges, and writing all those
dependencies in the state.

Rather than record every possible dependency edge, only record the first
managed resource in any lineage, such that the resulting graph is
topologically equivalent having removed many transitive edges. For some
configurations this can save considerable time and space when handling
the state.

Use the more efficient FirstAncestorsWith when connecting dependencies
cause by `depends_on`.

* moved: error if not targeting a resource address

* address comments

core: use more efficient methods for finding ancestors

…viders (#35877)

Co-authored-by: Daniel Schmidt <danielmschmidt92@gmail.com>

With the missing destroy node types sorted out, this transformer can now
be designed around finding the nodes we need to keep for destroy
operations, rather than the whack-a-mole game of detecting nodes we
don't want to keep.

When executing a destroy operation, we can never be certain that the
state will contain all the data required to evaluate the entire
configuration. This could be because of a previously failed apply
operation, or external changes made out of band. This means that once we
process the configuration to get the providers hooked up, we need to
remove any nodes that correspond to configuration values which may not
be available to evaluate.

In a destroy operation we primarily need three categories of nodes to
complete the plan and apply:
 - Nodes corresponding to values in the state
 - Providers required to destroy those values obtained from the state
 - Any node which a provider depends on, to ensure its configuration can
   be fully evaluated

Other than that minimal set of nodes, the rest are no longer necessary
and should be skipped to avoid errors which could block the apply
operation.

* check root output values when determining plan applicability in refresh mode

In refresh-only mode, we do not anticipate proposing any actions; however, a plan is marked as “applyable” if there are changes in the state between runs. Currently, a plan is considered “applyable” only when there are differences in managed resources. This approach seems to overlook changes in root output values. As a result, a plan can be marked as non-applyable, even when there are changes in the root output value, due to the lack of detected changes since only managed resources were checked.

* set 'applyable' to true

While in most cases a missing instance during evaluation would be a bug
in terraform, the insertion of postconditions into instance planning has
created a situation where not all instances will be available when
GetResource is called. We do have a special context for evaluating self
alone, but postconditions also need to have the module context available
too.

We may be able to come up with some other way to create an evaluation
context which handles self more selectively, but that looks like a large
change for a situation which should not otherwise happen. If this were
to mask a legitimate error, the fact that an unexpected DyanamicVal were
encountered will raise the problem again, and we can correlate that with
the logged warning.

don't error on eval of missing instances

…troy

core: update destroy graph pruning to handle ephemeral resources

Before a timestamp is assigned during plan, the `time.Time` value will
be the zero value for that type. This represents a case where the
timestamp is not yet known, but the handling of that timestamp isn't
using our usual cty value system for representing unknowns. Rather than
refactor the use of the timestamp itself throughout the code, we can
simply have the function return an unknown if it encounters a zero
value.

`plantimestamp` can be unknown during validation

When key-value tags are enabled and used in the workspace, users may
define the tags attribute as a map of strings in the cloud block in order
to more precicely match workspaces using those tags.

The console eval walk is a special case where only state is evaluated,
however since ephemeral resources will never have stored state we need
to make sure that their evaluation always returns an unknown value.

The modules command provides a holistic view of all installed modules
for a given root module. It also includes information on whether a
installed module is currently referenced by configuration. As of now,
this command only supports a machine-readable view format.

This commit makes several modifications to the modules command. It
downcases the field names in the resulting JSON string, adds a format
version to the result. The result of this command will now only contain
entries which have been declared in configuration, as opposed to the
internal manifest which shows all installed modules for a working dir.

fix ephemeral evaluation during with console

Fix broken link for describe providers

---------

Co-authored-by: Brian McClain <brianmmcclain@gmail.com>

`terraform modules` command

Co-authored-by: Radek Simko <radeksimko@users.noreply.github.com>

core: check preconditions and postconditions for ephemeral resources

…_block

cloud: Allows object tag schema for selecting key/value tagged workspaces

…5919)

* Merge to main

* fix

* Apply suggestions from code review

Co-authored-by: Robin Norwood <robin.norwood@gmail.com>

* Apply suggestions from code review

Co-authored-by: Robin Norwood <robin.norwood@gmail.com>

* Terraform format

* Small fix

* Add ephemeral resources page

* Update website/docs/language/values/locals.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/values/locals.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/values/locals.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/values/locals.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/ephemeral.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Updates from review feedback

* Add the finishing touches

* Add terraform applying

* Halfway through

* Fix the links

* One more pass of resources

* Almost done only inputs outputs and locals to go

* Finish inputs and outputs

* Apply suggestions from code review

Co-authored-by: Robin Norwood <rnorwood@hashicorp.com>

* Add feedback

* Make example small for now

* Apply suggestions from code review

Co-authored-by: Austin Valle <austinvalle@gmail.com>

* Update website/docs/language/state/sensitive-data.mdx

* Add example to ephemeral resource page

* Reference ephemeral values in provisioner and connection blocks

* Correct output command docs

* Update website/docs/language/resources/provisioners/syntax.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Update website/docs/language/resources/provisioners/syntax.mdx

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>

* Reference provisioners and connection blocks

* Apply suggestions from code review

Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>

---------

Co-authored-by: Robin Norwood <robin.norwood@gmail.com>
Co-authored-by: Robin Norwood <rnorwood@hashicorp.com>
Co-authored-by: Austin Valle <austinvalle@gmail.com>
Co-authored-by: Tu Nguyen <im2nguyen@users.noreply.github.com>

It turns out the reuse of the plan nodes for ephemerals during apply was
interfering with how checks are handled in the plan. Since checks
already have an established pattern of use, we can avoid the whole issue
by using only the resource apply nodes to kick of ephemeral values.

This copies the subset of expansion logic needed from the resource plan
node into the apply node. Since the apply time evaluation is much
simpler, duplicated code is not overly complex, but we may want to
revisit this for refactoring in the future.

We can't enforce the check of the close flag in mocked ephemeral
resources, since multiple instances could call the close method while
other instances are running.

The cloud backend includes a retry hook that relies on a go-tfe implementation. Since these retry messages were not rendered through a view, the `-json` flag was not respected, leading to mixed (non-pure JSON) output in the terminal whenever retry messages were printed.

Before passing the init view to the cloud backend, we ensure the retryLog hook is included in the init view.

Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>

Signed-off-by: Bruno Schaatsbergen <git@bschaatsbergen.com>

Ephemeral instances are only removed as whole resources, so there should
not be missing instances from an ephemeral resource. We can't error on
missing instances, because evaluation of checks may be pulling only a
single instance for evaluation before other sibling instances have been
opened.

Co-authored-by: Rose M Koron <32436232+rkoron007@users.noreply.github.com>