Importing a gcp:storage:Bucket that has labels results in a spurious diff
#1916
Assignees
Labels
area/import
An issue related to `pulumi import` or the import resource option.
bug/diff
kind/bug related to Pulumi generating wrong diffs on preview or up.
kind/bug
Some behavior is incorrect or out of spec
resolution/fixed
This issue was fixed
Comments
zbuchheit
added
kind/bug
guineveresaenger
added
p1
VenelinMartinov
removed
the
p1
mjeffryes
added
the
bug/diff
|
Confirmed this reproduces. Repro script:
name: gcp_bucket_import
runtime: yaml
resources:
my-bucket-zbuchheit:
properties:
labels:
app: my-bucket
location: US
type: gcp:storage:Bucket
outputs:
bucketUrn: ${my-bucket-zbuchheit.urn} #Added for Convenience
bucketName: ${my-bucket-zbuchheit.name} #Added for Convenience
{
"resources": [
{
"type": "gcp:storage/bucket:Bucket",
"name": "my-bucket-zbuchheit",
"id": "<PLACEHOLDER>"
}
]
}
#!/bin/bash
set -euxo pipefail
pulumi stack init test1 || True
pulumi stack select test1
pulumi up --yes
eval "$(pulumi stack output --shell)"
pulumi state delete $bucketUrn --yes
cp import_template.json import.json
sed -i '' -e 's,<PLACEHOLDER>,'"$bucketName"',g' import.json
pulumi import --file import.json --yes
pulumi preview --expect-no-changesEDIT: the issue might actually be with the import - looks like both the EDIT2: provider "google" {
region = "us-central1"
}
terraform {
required_providers {
google = {
source = "hashicorp/google"
version = "4.84.0"
}
}
}
import {
to = google_storage_bucket.example
id = "my-bucket-zbuchheit-86f392c"
}
running # __generated__ by Terraform
# Please review these resources and move them into your main configuration files.
# __generated__ by Terraform from "my-bucket-zbuchheit-86f392c"
resource "google_storage_bucket" "example" {
default_event_based_hold = false
force_destroy = false
labels = {
app = "my-bucket"
}
location = "US"
name = "my-bucket-zbuchheit-86f392c"
project = "pulumi-development"
public_access_prevention = "inherited"
requester_pays = false
storage_class = "STANDARD"
uniform_bucket_level_access = false
timeouts {
create = null
read = null
update = null
}
}
Note the upstream provider has some different handling of effective_labels vs labels and terraform_labels: Perhaps we are passing something wrong there? Still digging. Upstream commit which added this: https://github.com/GoogleCloudPlatform/magic-modules/pull/8996/files#diff-f56092529c6414ea6c407373920900197c487c3fe513550b5f18c767b4b9818d, EDIT3: After upgrading the TF provider, I reproed this in TF. Running provider "google" {
region = "us-central1"
}
terraform {
required_providers {
google = {
source = "hashicorp/google-beta"
version = "5.26.0"
}
}
}
import {
to = google_storage_bucket.example
id = "my-bucket-zbuchheit-86f392c"
}
generates # __generated__ by Terraform
# Please review these resources and move them into your main configuration files.
# __generated__ by Terraform from "my-bucket-zbuchheit-86f392c"
resource "google_storage_bucket" "example" {
provider = google-beta
default_event_based_hold = false
enable_object_retention = false
force_destroy = false
labels = {}
location = "US"
name = "my-bucket-zbuchheit-86f392c"
project = "pulumi-development"
public_access_prevention = "inherited"
requester_pays = false
rpo = "DEFAULT"
storage_class = "STANDARD"
uniform_bucket_level_access = false
soft_delete_policy {
retention_duration_seconds = 604800
}
}
Note that the same issue with {
"version": 4,
"terraform_version": "1.7.0",
"serial": 1,
"lineage": "d7b6b1b3-501f-275e-132a-aea3c82101a3",
"outputs": {},
"resources": [
{
"mode": "managed",
"type": "google_storage_bucket",
"name": "example",
"provider": "provider[\"registry.terraform.io/hashicorp/google-beta\"]",
"instances": [
{
"schema_version": 1,
"attributes": {
"autoclass": [],
"cors": [],
"custom_placement_config": [],
"default_event_based_hold": false,
"effective_labels": {
"app": "my-bucket"
},
"enable_object_retention": false,
"encryption": [],
"force_destroy": false,
"id": "my-bucket-zbuchheit-86f392c",
"labels": {},
"lifecycle_rule": [],
"location": "US",
"logging": [],
"name": "my-bucket-zbuchheit-86f392c",
"project": "pulumi-development",
"project_number": 921927215178,
"public_access_prevention": "inherited",
"requester_pays": false,
"retention_policy": [],
"rpo": "DEFAULT",
"self_link": "https://www.googleapis.com/storage/v1/b/my-bucket-zbuchheit-86f392c",
"soft_delete_policy": [
{
"effective_time": "2024-04-23T12:09:06.112Z",
"retention_duration_seconds": 604800
}
],
"storage_class": "STANDARD",
"terraform_labels": {},
"timeouts": null,
"uniform_bucket_level_access": false,
"url": "gs://my-bucket-zbuchheit-86f392c",
"versioning": [],
"website": []
},
"sensitive_attributes": [],
"private": "eyJlMmJmYjczMC1lY2FhLTExZTYtOGY4OC0zNDM2M2JjN2M0YzAiOnsiY3JlYXRlIjo2MDAwMDAwMDAwMDAsInJlYWQiOjI0MDAwMDAwMDAwMCwidXBkYXRlIjoyNDAwMDAwMDAwMDB9LCJzY2hlbWFfdmVyc2lvbiI6IjEifQ=="
}
]
}
],
"check_results": null
}
|
Merged
VenelinMartinov
added
the
area/import
VenelinMartinov
added a commit
that referenced
this issue
Apr 26, 2024
fixes #1916 This patches the upstream handling of labels to import `labels` and `pulumiLabels` as well as `effectiveLabels`. Note we handle `defaultLabels` on the provider by removing them from the imported `labels`. This allows for storage buckets to be imported cleanly along with their labels. The upstream provider does not import these properties. [The TF GCP provider allows](https://www.hashicorp.com/blog/terraform-google-provider-adds-updates-to-default-labels) for non-managed labels on resources. this PR changes this in our provider - it will now import ALL labels and assume they are all managed in pulumi by default. Note that we still allow manually editing the inputs after importing. There is no way to fix #1916 without this though - we either have to assume the labels are managed by pulumi or none are. Note that I adopted the import machinery from pulumi/pulumi-aws#3859 I've opened #1959 as a possible follow-up if we decide to fix this behaviour for other resources. Implementation might be tricky. --------- Co-authored-by: Ian Wahbe <ian@wahbe.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What happened?
When attempting to import a bucket, there was a diff on labels and pulumiLabels following the import.
Output of pulumi preview after import
Example
Code Repro Pulumi.yaml
Pulumi import.json file for convenience
{ "resources": [ { "type": "gcp:storage/bucket:Bucket", "name": "my-bucket-zbuchheit", "id": "my-bucket-zbuchheit-<PLACEHOLDER>" } ] }Output of pulumi preview --diff after import
Repro Steps
pulumi state deleteto remove the bucket from statepulumi importpulumi previewto witness the diffState
Bucket State after Import
{ "urn": "urn:pulumi:dev::service-account::gcp:storage/bucket:Bucket::my-bucket-zbuchheit", "custom": true, "id": "my-bucket-zbuchheit-d530101", "type": "gcp:storage/bucket:Bucket", "inputs": { "__defaults": [], "location": "US", "name": "my-bucket-zbuchheit-d530101", "project": "redacted", "publicAccessPrevention": "inherited", "rpo": "DEFAULT" }, "outputs": { "__meta": "{\"afa4b0a0-ba45-4826-bb5c-4db5030322ab\":{\"create\":600000000000,\"read\":240000000000,\"update\":240000000000},\"schema_version\":\"1\"}", "autoclass": null, "cors": [], "customPlacementConfig": null, "defaultEventBasedHold": false, "effectiveLabels": { "app": "my-bucket" }, "enableObjectRetention": false, "encryption": null, "forceDestroy": false, "id": "my-bucket-zbuchheit-d530101", "labels": {}, "lifecycleRules": [], "location": "US", "logging": null, "name": "my-bucket-zbuchheit-d530101", "project": "redacted", "projectNumber": "redacted", "publicAccessPrevention": "inherited", "pulumiLabels": {}, "requesterPays": false, "retentionPolicy": null, "rpo": "DEFAULT", "selfLink": "https://www.googleapis.com/storage/v1/b/my-bucket-zbuchheit-d530101", "softDeletePolicy": { "effectiveTime": "2024-04-11T22:42:38.548Z", "retentionDurationSeconds": 604800 }, "storageClass": "STANDARD", "uniformBucketLevelAccess": false, "url": "gs://my-bucket-zbuchheit-d530101", "versioning": null, "website": null }, "parent": "urn:pulumi:dev::service-account::pulumi:pulumi:Stack::service-account-dev", "protect": true, "provider": "urn:pulumi:dev::service-account::pulumi:providers:gcp::default::b686e756-874b-4ae5-ae73-9202c1b68f88", "created": "2024-04-11T23:04:03.199433Z", "modified": "2024-04-11T23:04:03.199433Z" }Bucket State after Up
{ "urn": "urn:pulumi:dev::service-account::gcp:storage/bucket:Bucket::my-bucket-zbuchheit", "custom": true, "id": "my-bucket-zbuchheit-d530101", "type": "gcp:storage/bucket:Bucket", "inputs": { "__defaults": [ "forceDestroy", "name", "storageClass" ], "forceDestroy": false, "labels": { "app": "my-bucket" }, "location": "US", "name": "my-bucket-zbuchheit-d530101", "storageClass": "STANDARD" }, "outputs": { "__meta": "{\"e2bfb730-ecaa-11e6-8f88-34363bc7c4c0\":{\"create\":600000000000,\"read\":240000000000,\"update\":240000000000},\"schema_version\":\"1\"}", "autoclass": null, "cors": [], "customPlacementConfig": null, "defaultEventBasedHold": false, "effectiveLabels": { "4dabf18193072939515e22adb298388d": "1b47061264138c4ac30d75fd1eb44270", "ciphertext": "AAABAALnBETznY6JnfjyS/VJ0BqBWq7hvLJd5/7G2OoYEcVeFDeV//IUaP5aZS1sGblI" }, "enableObjectRetention": false, "encryption": null, "forceDestroy": false, "id": "my-bucket-zbuchheit-d530101", "labels": { "app": "my-bucket" }, "lifecycleRules": [], "location": "US", "logging": null, "name": "my-bucket-zbuchheit-d530101", "project": "redacted", "projectNumber": "redacted", "publicAccessPrevention": "inherited", "pulumiLabels": { "4dabf18193072939515e22adb298388d": "1b47061264138c4ac30d75fd1eb44270", "ciphertext": "AAABAOJM6LMHrQZqo1hv+TJ32RJObfIzkyfgupE93sHRNHpjmnXWvfSh49qnrRvsiw+k" }, "requesterPays": false, "retentionPolicy": null, "rpo": "DEFAULT", "selfLink": "https://www.googleapis.com/storage/v1/b/my-bucket-zbuchheit-d530101", "softDeletePolicy": { "effectiveTime": "2024-04-11T22:42:38.548Z", "retentionDurationSeconds": 604800 }, "storageClass": "STANDARD", "uniformBucketLevelAccess": false, "url": "gs://my-bucket-zbuchheit-d530101", "versioning": null, "website": null }, "parent": "urn:pulumi:dev::service-account::pulumi:pulumi:Stack::service-account-dev", "provider": "urn:pulumi:dev::service-account::pulumi:providers:gcp::default::b686e756-874b-4ae5-ae73-9202c1b68f88", "propertyDependencies": { "labels": [], "location": [] }, "additionalSecretOutputs": [ "effectiveLabels", "pulumiLabels" ], "created": "2024-04-11T23:04:03.199433Z", "modified": "2024-04-11T23:09:34.175844Z" }Diff between state files
Output of
pulumi aboutAdditional context
No response
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).
The text was updated successfully, but these errors were encountered: