capture output of provisioners into variables

[piavlo]

so that they could be referenced in other TF resources

[mitchellh]

Interesting idea... and completely possible. I think we need an overhaul of the state a bit to better support longform text to support this. We plan on making this change at some point. I'll tag this for now.

[lalloni]

+1!

We're needing this feature right now for creating and sharing cluster discovery tokens.

[lalloni]

Here's how it's used in Ansible.

[yruss972]

+1

[supernomad]

So I just ran into the need for this as well, want to save the output from an inline script call and have it available after its been created.

[james-gonzalez]

I second @supernomad. We are currently taking the state, parsing it into an ini file and then querying it that way into variables (via the Ansible lookup function). It would be great if this could be made easier.

[timothy-spencer]

I would love something like this as well. I've got a fair number of things that create accounts or keys or whatever on an instance that I'd love to be able to capture and then stick into other instances. I know this is probably tricky, but man, it would be cool to remove these manual steps from the process. :-)

[tomwganem]

I use terraform to stand up mesos clusters with a newly provisioned vault instance that has a randomly generated set of passwords for that cluster. In addition, I generate access tokens for our internal deployment tools and our devops engineers to access these secrets when they need them. I would like to be able to save these access tokens to our permanent on-premise vault installation. If terraform provided me a way to capture output from a provisioner, I would be able to do that.

[maus-]

+1

[drpaneas]

+1 I need this functionality also.

[deepblue-phoenix]

+1 this would be great

[anasinnyk]

+1 it's block my swarm cluster. I have a token in master node and I need it randomly token from provisioner to use for include different nodes to cluster.

[holms]

Any status on this? Why such a basic feature still not implemented? How can I join slaves of cluster if I can't memorize token...?

[smastrorocco]

+1 helpful with AWS provider as well to fetch EC2 Windows password and pass to communicator

[zarbis]

Interested in this also, mainly to register cluster join tokens.

[rmihael]

It is a roadblock for many different scenarios. Is there any workaround to get command's output as a variable?

[matti]

Meanwhile I did this that works for me in my use case: https://github.com/matti/terraform-shell-resource

[aldoborrero]

Yeah, I see this as very useful feature to have as some commented for creating Docker Swarms or other valid use cases...

[dmitrypol]

+1

[ravidbro]

+1

[bazmera]

It would be a useful feature. Would solve so many roadblocks.

[mdeggies]

+1

[julianfrank]

+1

[sachinar]

+1

[matti]

I'm still actively developing https://github.com/matti/terraform-shell-resource module, the number of issues and github stars etc clearly shows that there is a massive demand for this one.

while my module works pretty well now after 0.12, having this in terraform would be super good.

[ska-ops]

+1

[MaxPeal]

+1

[countcain]

+1

[boredabdel]

@matti is it possible to use your module to run bash files ?

[matti]

@boredabdel why not?

[boredabdel]

I'm wondering if it's possible ! and if you could provide an example it would great !

Thanks

[matti]

just call the script as the command and you are good

[openscript]

Without this feature it makes it unnecessarily complicated to join worker nodes to a Docker swarm.

[githanium]

+1

[anandsuresh]

+1

[lejluff]

+1 Is there still traction on this? This has been open for a very long time.

I'm still actively developing https://github.com/matti/terraform-shell-resource module, the number of issues and github stars etc clearly shows that there is a massive demand for this one.

while my module works pretty well now after 0.12, having this in terraform would be super good.

This resource does a local exec and I need a remote exec to capture the outputs from bash script on remote ec2
.

[ifaizan]

+1 on this. This would make life a lot easier

[avifreege]

+1 , will be very helpful for adding self_managed_node_groups with the EKS module V18.

[matti]

my original workarounds are now archived, please see https://github.com/Invicton-Labs/terraform-external-shell-resource and https://github.com/Invicton-Labs/terraform-external-shell-data

[tanvp112]

Module https://github.com/Invicton-Labs/terraform-external-shell-resource saved the day!

[ginjo]

@matti , been using your shell-resource for several years to deploy swarm clusters, and it works like a charm. I'll migrate over to the Inviction-labs code now - so happy this functionality still exists. Thanks for all your work!

[matti]

@ginjo thank you!

[byteknacker]

I also need this feature to get the service secret token of Gitlab's Admin Service Account. I need it to be a terraform output to pass it over to the GItlab CI/CD variables and use it later for another job. Please put this on priority for the next releases as it is a much requested feature.

[jaehyeon-kim]

I need this feature to create a manage endpoint of emr container and capture the output.

[apparentlymart]

Hi all! Sorry for the long silence here.

This issue has languished because at this point provisioners are essentially a legacy system in Terraform, and we aren't planning to make any further significant changes to their design. We consider provisioners to be a last resort, and would recommend using providers instead whereever possible.

A create-time provisioner is functionally equivalent to a resource type which takes an action during its "create" step, and a destroy-time provisioner is functionally equivalent to a resource type which takes an action during its "destroy step.

Resource types already have the capability to export arbitrary data from their create actions, and so the best path forward to meet the use-case this issue is representing is to use a provider which has a resource type which takes the action you want to take during its create step and returns the data you need. Modern Terraform supports third-party providers, so there are various providers dealing with variants of this problem and anyone is empowered to write a new one.

One example community provider that can potentially take a similar role as the local-exec provisioner would be scottwinkler/shell. I've not made extensive use of it myself so I can't promise it will solve all problems, but according to its documentation it allows declaring an arbitrary shell script to run during each of the resource instance lifecycle steps, and its documentation suggests that it returns the output of the scripts being run.

I also see loafoe/ssh, which looks promising as a resource-type-based replacement for the remote-exec and file provisioners if you are using them to interact with SSH servers. I've not tried this one at all so I cannot vouch for its behavior, but its documentation describes functionality both for uploading files and for executing arbitrary commands over SSH.

I don't yet see a community provider in the registry for WinRM, so that does remain one situation where the remote-exec and file provisioners may be the best option for now. The WinRM-based provisioning process has always been a bit of a hack -- WinRM doesn't really have capabilities matching SSH, so Terraform's support for it is just a collection of workarounds. Given that, I expect that a hypothetical WinRM provider could be better tailored to the WinRM protocol's capabilities rather than Terraform's current strategy of just trying to pretend it's like SSH. Modern Windows also has an OpenSSH server available, so the need for WinRM is diminished.

With all of that said, I'm going to close this issue to represent that we're unlikely to prioritize any significant architectural changes related to provisioners. Instead, we intend to focus on the ecosystem of providers and on the resource instance change lifecycle, which is largely a superset of the capabililties of Terraform's protocol for provisioner plugins.

Thanks for this feature request, and sorry for the long silence!

[matti]

the mentioned scottwinkler/terraform-provider-shell#88

forces scripts to output json...

[lorengordon]

It would truly be nice if the terraform team would talk to the packer team, and support the same set of provisioner options.

[lorengordon]

The main feature of provisioners that I truly appreciate is that output is streamed back to the user. When I'm running a script on a remote instance, that is invaluable. Providers do not seem to do that? Which makes troubleshooting waaaaay hard. Anyone know if any of the provider-based options actually do stream at least stderr back to the user?

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.