local.Command not re-running on changing inputs

[itay]

Hello!

• Vote on this issue by adding a 👍 reaction
• To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already)

Issue details

I'm trying a relatively simple usage (similar to the GithubLabels example), but it's not really working as I expected. This is using Go and v0.0.3 of local.Command.

Here is my main.go:

package main

import (
	"github.com/pulumi/pulumi-command/sdk/go/command/local"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {

		_, err := local.NewCommand(ctx, "test", &local.CommandArgs{
			Create: pulumi.String("./create.sh hello"),
			Delete: pulumi.String("./destroy.sh goodbye"),
			Environment: pulumi.StringMap{
				"foo": pulumi.String("bar"),
			},
		})
		if err != nil {
			return err
		}

		return nil
	})
}

Here is my create.sh and destroy.sh:

#!/bin/bash

echo "hello: $1" >> /tmp/pulumi_log.txt

#!/bin/bash

echo "goodbye: $1" >> /tmp/pulumi_log.txt

When I run pulumi up the first time, the resource gets created and I see the right entry in the log file. Similarly, if I explicitly run pulumi destroy, I see the right entry added to the log file.

However, if I were to change the value (e.g. changing "bar" to "bar2"), and I run pulumi up, it tells me that environment is updated, but the script never executes again (and no corresponding entry in the log).

Not sure if I'm doing something wrong or have the wrong expectations.

Steps to reproduce

See above.

Expected: Script re-runs when the inputs change.
Actual: Script doesn't re-run.

[viveklak]

This seems by design as per https://github.com/pulumi/pulumi-command/blob/master/provider/pkg/provider/provider.go#L278. Once could trigger an update by specifying a replaceOnChanges clause for environment etc. cc @lukehoban if this seems worth doing by default or if there is a good use case to consider offering an update hook in addition.

[itay]

Per the discussion in Slack: https://pulumi-community.slack.com/archives/C84L4E3N1/p1643048823205900

It looks like local.Command will not do anything when inputs are updated by default, and only re-run when the resource needs to be replaced. Additionally, the replace will only happen by default if the name changes. To work around this behavior, it is possible to add pulumi.ReplaceOnChanges([]string{"*"}) (or more specific keys such as environment) to make it replace overall. Additionally, it's probably desirable to have pulumi.DeleteBeforeReplace(true) in these cases, to first delete the resource.

In general, this is very unintuitive that nothing we changed on the local.Command resource causes an update to it.

Finally, I will note that it's unfortunate there is no way to do an update, as it forces deleting the entire resource and re-creating. In the example of managing an external resource using local.Command (in lieu of creating a full package for it, which is a huge lift in terms of effort), it would mean that the resource gets fully deleted and then recreated.

One example of this is imagine you want to create a GitHub group (similar to the GitHub label example), and that GithubGroup resource has a name and a description. Additionally, you can manage who is in that group (but that's done as a separate activity, possibly outside of Pulumi).

Say you wanted to update the description: with the "replace" semantics (i.e. do a delete and then a create), it would delete the entire group and recreate it, which would mean all the members of that group get removed. If there was an update, it would be possible to just update the description.

I know it'll be more difficult to do so without having some kind of diff semantics, but the developer can handle it however they want.

[lukehoban]

I believe this is largely the same issue tracked in #20, about adding the ability to have actions be taken during "updates" (which also will require the ability to determine what changes constitute an update). For many use cases, there is no meaningful interpretation of an update (for example, for provisioning scenarios, and for anything that could be accomplished with Terrraform style provisioners). But it is definitely true that there are scenarios for a local.Command-style provider in Pulumi which could also understand how to do updates.

The key question related to #20 is to figure out how Diff should behave to address and enable the core use cases. Should it only trigger an update when existing properties on the resource are changed (such as Environment in the case above here). Or should it be implementable via abash script itself, to decide dynamically whether an update should be performed or not (more like the full capabilities of a provider in Pulumi). It would be helpful to capture a few key use cases for adding "update" semantics to this provider, and then use those to inform a design proposal on "diff".

[itay]

Thanks @lukehoban - happy to have this ticket closed in favor of #20 if that's easier.

How can I help to get the improvement here? Specifically, I want to use this to create a "fake package" for a set of resources, as creating a real Pulumi package is such a heavy lift at the moment.

[rbm-radius]

The same thing happens with remote commands, e.g. copy file.

If you have a copy file operation from local to remote, then change either the local file or the remote file (even if you delete the remote file), no copy happens until you destroy the whole resource.

There should be a simple, intuitive way to achieve the behavior many (most?) people would expect.

[alex-hunt-materialize]

I've found a work around for some use cases for this, by storing the create command in a Pulumi Output variable, and then adding that Output variable to the triggers list. This causes Pulumi to recreate the Command resource when the Output variable changes.

This is extremely non-intuitive. I imagine pretty much everyone would expect it to re-run the command if the command changed.

[pulumi-bot]

Cannot close issue without required labels: kind/