Importing a `BucketIAMBinding` results in a unexpected diff on the bucket input #1900

zbuchheit · 2024-04-04T19:16:35Z

What happened?

When importing a BucketIAMBinding it is able to import the resource correctly but incorrectly appends b/ to the bucket input once it is imported, resulting in a diff as the code will attempt to remove the b/.

Example

Steps to Repro

Run pulumi up with the following code

Code Repro

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

// Create a GCP resource (Storage Bucket)
const bucket = new gcp.storage.Bucket("zbuchheit-bucket", {
    name: "zbuchheit-bucket",
    location: "US"
});

const objectAdminBinding = new gcp.storage.BucketIAMBinding("objectAdminBinding", {
    bucket: bucket.name,
    role: "roles/storage.objectAdmin",
    members: [
        "user:zbuchheit@pulumi.com",
    ],
});

export const bucketName = bucket.url;

Delete the resource from state using pulumi state delete urn:pulumi:dev::gcp-bucket-binding-ts::gcp:storage/bucketIAMBinding:BucketIAMBinding::objectAdminBinding --yes
Import the resource into pulumi state using the following pulumi import --file import.json

import.json

{
    "resources": [
        {
            "type": "gcp:storage/bucketIAMBinding:BucketIAMBinding",
            "name": "objectAdminBinding",
            "id": "b/zbuchheit-bucket roles/storage.objectAdmin",
            "version": "7.16.0"
        }
    ]
}

Diff Output

This produces a diff like the following

  pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:dev::gcp-bucket-binding-ts::pulumi:pulumi:Stack::gcp-bucket-binding-ts-dev]
      gcp:storage/bucketIAMBinding:BucketIAMBinding: (same) 🔓
        [id=b/zbuchheit-bucket/roles/storage.objectAdmin]
        [urn=urn:pulumi:dev::gcp-bucket-binding-ts::gcp:storage/bucketIAMBinding:BucketIAMBinding::objectAdminBinding]
        [provider=urn:pulumi:dev::gcp-bucket-binding-ts::pulumi:providers:gcp::default_7_16_0::053f77be-1eaf-46ec-93ac-cd6fecc52536]
      ~ bucket : "b/zbuchheit-bucket" => "zbuchheit-bucket"
        members: [
            [0]: "user:zbuchheit@pulumi.com"
        ]
        role   : "roles/storage.objectAdmin"

Output of `pulumi about`

CLI          
Version      3.112.0
Go Version   go1.22.1
Go Compiler  gc

Plugins
NAME    VERSION
gcp     7.16.0
nodejs  unknown

Host     
OS       darwin
Version  14.2.1
Arch     arm64

This project is written in nodejs: executable='/Users/zbuchheit/.nvm/versions/node/v18.17.1/bin/node' version='v18.17.1'

Current Stack: zbuchheit-pulumi-corp/gcp-bucket-binding-ts/dev

TYPE                                           URN
pulumi:pulumi:Stack                            urn:pulumi:dev::gcp-bucket-binding-ts::pulumi:pulumi:Stack::gcp-bucket-binding-ts-dev
pulumi:providers:gcp                           urn:pulumi:dev::gcp-bucket-binding-ts::pulumi:providers:gcp::default_7_16_0
gcp:storage/bucket:Bucket                      urn:pulumi:dev::gcp-bucket-binding-ts::gcp:storage/bucket:Bucket::zbuchheit-bucket
gcp:storage/bucketIAMBinding:BucketIAMBinding  urn:pulumi:dev::gcp-bucket-binding-ts::gcp:storage/bucketIAMBinding:BucketIAMBinding::objectAdminBinding


Found no pending operations associated with dev

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/zbuchheit-pulumi-corp
User           zbuchheit-pulumi-corp
Organizations  zbuchheit-pulumi-corp
Token type     personal

Dependencies:
NAME            VERSION
@pulumi/gcp     7.16.0
@pulumi/pulumi  3.112.0
@types/node     18.19.29

Pulumi locates its logs in /var/folders/lh/l71cdh810xb33t0jc7qmt5_80000gn/T/ by default

Additional context

Resource state after the import

            {
                "urn": "urn:pulumi:dev::gcp-bucket-binding-ts::gcp:storage/bucketIAMBinding:BucketIAMBinding::objectAdminBinding",
                "custom": true,
                "id": "b/zbuchheit-bucket/roles/storage.objectAdmin",
                "type": "gcp:storage/bucketIAMBinding:BucketIAMBinding",
                "inputs": {
                    "__defaults": [],
                    "bucket": "b/zbuchheit-bucket",
                    "members": [
                        "user:zbuchheit@pulumi.com"
                    ],
                    "role": "roles/storage.objectAdmin"
                },
                "outputs": {
                    "bucket": "b/zbuchheit-bucket",
                    "condition": null,
                    "etag": "CAI=",
                    "id": "b/zbuchheit-bucket/roles/storage.objectAdmin",
                    "members": [
                        "user:zbuchheit@pulumi.com"
                    ],
                    "role": "roles/storage.objectAdmin"
                },
                "parent": "urn:pulumi:dev::gcp-bucket-binding-ts::pulumi:pulumi:Stack::gcp-bucket-binding-ts-dev",
                "protect": true,
                "provider": "urn:pulumi:dev::gcp-bucket-binding-ts::pulumi:providers:gcp::default_7_16_0::053f77be-1eaf-46ec-93ac-cd6fecc52536",
                "created": "2024-04-04T18:59:46.766148Z",
                "modified": "2024-04-04T18:59:46.766148Z"
            }

Resource state after the up

                       {
                "urn": "urn:pulumi:dev::gcp-bucket-binding-ts::gcp:storage/bucketIAMBinding:BucketIAMBinding::objectAdminBinding",
                "custom": true,
                "id": "b/zbuchheit-bucket/roles/storage.objectAdmin",
                "type": "gcp:storage/bucketIAMBinding:BucketIAMBinding",
                "inputs": {
                    "__defaults": [],
                    "bucket": "zbuchheit-bucket",
                    "members": [
                        "user:zbuchheit@pulumi.com"
                    ],
                    "role": "roles/storage.objectAdmin"
                },
                "outputs": {
                    "bucket": "b/zbuchheit-bucket",
                    "condition": null,
                    "etag": "CAI=",
                    "id": "b/zbuchheit-bucket/roles/storage.objectAdmin",
                    "members": [
                        "user:zbuchheit@pulumi.com"
                    ],
                    "role": "roles/storage.objectAdmin"
                },
                "parent": "urn:pulumi:dev::gcp-bucket-binding-ts::pulumi:pulumi:Stack::gcp-bucket-binding-ts-dev",
                "dependencies": [
                    "urn:pulumi:dev::gcp-bucket-binding-ts::gcp:storage/bucket:Bucket::zbuchheit-bucket"
                ],
                "provider": "urn:pulumi:dev::gcp-bucket-binding-ts::pulumi:providers:gcp::default_7_16_0::053f77be-1eaf-46ec-93ac-cd6fecc52536",
                "propertyDependencies": {
                    "bucket": [
                        "urn:pulumi:dev::gcp-bucket-binding-ts::gcp:storage/bucket:Bucket::zbuchheit-bucket"
                    ],
                    "members": [],
                    "role": []
                },
                "created": "2024-04-04T18:59:46.766148Z",
                "modified": "2024-04-04T18:59:46.766148Z",
                "sourcePosition": "project:///index.ts#10,28"
            }

Additional Note

For a reason I can't explain setting --protect to what matches in the code, will actually import the resource without the b/ on the inputs and result in no diff after the import. Using resourceOptions import also works as desired, but not a viable option given the current constraints I am facing.

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

The text was updated successfully, but these errors were encountered:

VenelinMartinov · 2024-04-15T13:41:19Z

Reproed this with the following slightly modified example:

repro.sh:

#!/bin/bas
set -euxo pipefail

pulumi stack init test1 || True
pulumi up --stack test1 --yes
eval "$(pulumi stack output --shell)"

pulumi state delete $bindingUrn --yes --stack test1

cp import_template.json import.json
sed -i '' -e 's,<BUCKETID>,'"$bucketId"',g' import.json
pulumi import --file import.json --yes
pulumi preview --expect-no-changes

import_template.json:

{
    "resources": [
        {
            "type": "gcp:storage/bucketIAMBinding:BucketIAMBinding",
            "name": "objectAdminBinding",
            "id": "<BUCKETID> roles/storage.objectAdmin",
            "version": "7.18.0"
        }
    ]
}

index.ts:

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

// Create a GCP resource (Storage Bucket)
const bucket = new gcp.storage.Bucket("my-bucket", { location: "US" });

const binding = new gcp.storage.BucketIAMBinding("objectAdminBinding", {
  bucket: bucket.name,
  role: "roles/storage.objectAdmin",
  members: ["allAuthenticatedUsers"],
});

export const bucketId = bucket.id;
export const bindingUrn = binding.urn;

Oddly, when I run pulumi up after the diff it states Resources: 3 unchanged:

pulumi up --yes
Previewing update (test1)

View in Browser (Ctrl+O): https://app.pulumi.com/pulumi/gcp_bucket_import/test1/previews/74576f19-31c3-4a00-8cab-bf3dad5251f4

Loading policy packs...

     Type                             Name                     Plan     Info
     pulumi:pulumi:Stack              gcp_bucket_import-test1           
     └─ gcp:storage:BucketIAMBinding  objectAdminBinding                [diff: ~bucket,protect]

Policies:
    ✅ pulumi-internal-policies@v0.0.6

Resources:
    3 unchanged

Updating (test1)

View in Browser (Ctrl+O): https://app.pulumi.com/pulumi/gcp_bucket_import/test1/updates/8

Loading policy packs...

     Type                             Name                     Status     Info
     pulumi:pulumi:Stack              gcp_bucket_import-test1             
     └─ gcp:storage:BucketIAMBinding  objectAdminBinding                  [diff: ~bucket,protect]

Policies:
    ✅ pulumi-internal-policies@v0.0.6

Outputs:
    bindingUrn: "urn:pulumi:test1::gcp_bucket_import::gcp:storage/bucketIAMBinding:BucketIAMBinding::objectAdminBinding"
    bucketId  : "my-bucket-a65be33"

Resources:
    3 unchanged

Duration: 3s

looks like we return DIFF_NONE in the Diff GRPC call:

{
    "method": "/pulumirpc.ResourceProvider/Diff",
    "request": {
        "id": "b/my-bucket-a9682fd/roles/storage.objectAdmin",
        "urn": "urn:pulumi:test1::gcp_bucket_import::gcp:storage/bucketIAMBinding:BucketIAMBinding::objectAdminBinding",
        "olds": {
            "bucket": "b/my-bucket-a9682fd",
            "condition": null,
            "etag": "CAI=",
            "id": "b/my-bucket-a9682fd/roles/storage.objectAdmin",
            "members": [
                "allAuthenticatedUsers"
            ],
            "role": "roles/storage.objectAdmin"
        },
        "news": {
            "__defaults": [],
            "bucket": "my-bucket-a9682fd",
            "members": [
                "allAuthenticatedUsers"
            ],
            "role": "roles/storage.objectAdmin"
        },
        "oldInputs": {
            "__defaults": [],
            "bucket": "b/my-bucket-a9682fd",
            "members": [
                "allAuthenticatedUsers"
            ],
            "role": "roles/storage.objectAdmin"
        }
    },
    "response": {
        "stables": [
            "bucket",
            "condition",
            "role"
        ],
        "changes": "DIFF_NONE",
        "hasDetailedDiff": true
    },
    "metadata": {
        "kind": "resource",
        "mode": "client",
        "name": "gcp"
    }
}

Checked this in TF, it does not reproduce:

provider "google" {
  region = "us-central1"
}

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "4.84.0"
    }
  }

}

resource "google_storage_bucket_iam_binding" "binding" {
  bucket = "b/my-bucket-a9682fd"
  role = "roles/storage.objectAdmin"
  members = [
    "allAuthenticatedUsers",
  ]
}

Tried to repro by just editing the bucket argument in the binding resource but failed to repro that way.

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

// Create a GCP resource (Storage Bucket)
const bucket = new gcp.storage.Bucket("my-bucket", { location: "US" });

const useB = new pulumi.Config().getBoolean("useB");

export const name = useB
  ? bucket.name.apply((name) => "b/" + name)
  : bucket.name;

new gcp.storage.BucketIAMBinding("objectAdminBinding", {
  bucket: name,
  role: "roles/storage.objectAdmin",
  members: ["allAuthenticatedUsers"],
});

After discussing this with @Frassle, he confirmed this is an issue with the display logic in the engine. This is likely to be difficult to fix there, so I'll raise an issue there and work on hacking something in the provider to work around the issue.

VenelinMartinov · 2024-04-17T10:06:07Z

Found a workaround here - prepending b/ to the bucket name input in the original program matches the input produced by the import which yields no diff after. @zbuchheit - can you please try this out?

I've also got a patch which fixes the behaviour in #1922 but it affects other things too so I am hesitant to commit that. Platform has mentioned they will prioritize pulumi/pulumi#15944 which is the real cause of the issue.

Workaround example:

// Create a GCP resource (Storage Bucket)
const bucket = new gcp.storage.Bucket("my-bucket", { location: "US" });

const binding = new gcp.storage.BucketIAMBinding("objectAdminBinding", {
  bucket: bucket.name.apply((name) => 'b/' + name), // notice the b/ prefix
  role: "roles/storage.objectAdmin",
  members: ["allAuthenticatedUsers"],
});

export const bucketId = bucket.id;
export const bindingUrn = binding.urn;

VenelinMartinov · 2024-05-03T18:15:03Z

This was fixed in the latest version of the pulumi CLI. I've also added a regression test in #1969

Adds a regression test for #1900

zbuchheit added kind/bug Some behavior is incorrect or out of spec needs-triage Needs attention from the triage team labels Apr 4, 2024

iwahbe added area/import An issue related to `pulumi import` or the import resource option. and removed needs-triage Needs attention from the triage team labels Apr 8, 2024

guineveresaenger added the p1 A bug severe enough to be the next item assigned to an engineer label Apr 12, 2024

VenelinMartinov self-assigned this Apr 15, 2024

This was referenced Apr 15, 2024

Diff displayed even when provider returns DIFF_NONE and empty detailed diff pulumi/pulumi#15944

Closed

Binding import fix #1922

Closed

mikhailshilkov added this to the 0.103 milestone Apr 16, 2024

zbuchheit mentioned this issue Apr 17, 2024

BucketIAMBinding unexpected diff re-order of members #1935

Closed

mjeffryes mentioned this issue Apr 17, 2024

Correlate dirty refresh issues #1936

Open

VenelinMartinov added the blocked The issue cannot be resolved without 3rd party action. label Apr 18, 2024

iwahbe removed the p1 A bug severe enough to be the next item assigned to an engineer label Apr 18, 2024

iwahbe unassigned VenelinMartinov Apr 23, 2024

VenelinMartinov mentioned this issue May 3, 2024

Bucket IAM Binding Import regression test #1969

Merged

VenelinMartinov self-assigned this May 3, 2024

VenelinMartinov added resolution/fixed This issue was fixed and removed blocked The issue cannot be resolved without 3rd party action. labels May 3, 2024

VenelinMartinov closed this as completed May 3, 2024

VenelinMartinov added a commit that referenced this issue May 3, 2024

Bucket IAM Binding Import regression test (#1969)

c817fe5

Adds a regression test for #1900

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Importing a `BucketIAMBinding` results in a unexpected diff on the bucket input #1900

Importing a `BucketIAMBinding` results in a unexpected diff on the bucket input #1900

zbuchheit commented Apr 4, 2024 •

edited

Loading

VenelinMartinov commented Apr 15, 2024 •

edited

Loading

VenelinMartinov commented Apr 17, 2024

VenelinMartinov commented May 3, 2024

Importing a BucketIAMBinding results in a unexpected diff on the bucket input #1900

Importing a BucketIAMBinding results in a unexpected diff on the bucket input #1900

Comments

zbuchheit commented Apr 4, 2024 • edited Loading

What happened?

Example

Steps to Repro

Diff Output

Output of pulumi about

Additional context

Additional Note

Contributing

VenelinMartinov commented Apr 15, 2024 • edited Loading

VenelinMartinov commented Apr 17, 2024

VenelinMartinov commented May 3, 2024

Importing a `BucketIAMBinding` results in a unexpected diff on the bucket input #1900

Importing a `BucketIAMBinding` results in a unexpected diff on the bucket input #1900

zbuchheit commented Apr 4, 2024 •

edited

Loading

Output of `pulumi about`

VenelinMartinov commented Apr 15, 2024 •

edited

Loading