suppress dataplex labels #6518

prauc · 2022-09-07T07:32:19Z

Suppress automatically added Dataplex Labels, when running tf plan.
Dataplex Labels added by the system led to an inconsistent tf state.

Closes #12449

If this PR is for Terraform, I acknowledge that I have:

Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
Read the Release Notes Guide before writing my release note below.

storage: fixed a bug where user specified labels get overwritten by Dataplex auto generated labels

modular-magician · 2022-09-07T07:33:20Z

Hello! I am a robot who works on Magic Modules PRs.

I've detected that you're a community contributor. @shuyama1, a repository maintainer, has been assigned to assist you and help review your changes.

❓ First time contributing? Click here for more details

Your assigned reviewer will help review your code by:

Ensuring it's backwards compatible, covers common error cases, etc.
Summarizing the change into a user-facing changelog note.
Passes tests, either our "VCR" suite, a set of presubmit tests, or with manual test runs.

You can help make sure that review is quick by running local tests and ensuring they're passing in between each push you make to your PR's branch. Also, try to leave a comment with each push you make, as pushes generally don't generate emails.

If your reviewer doesn't get back to you within a week after your most recent change, please feel free to leave a comment on the issue asking them to take a look! In the absence of a dedicated review dashboard most maintainers manage their pending reviews through email, and those will sometimes get lost in their inbox.

modular-magician · 2022-09-07T07:44:20Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 1 file changed, 22 insertions(+), 4 deletions(-))
Terraform Beta: Diff ( 1 file changed, 22 insertions(+), 4 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

modular-magician · 2022-09-07T08:34:46Z

Tests analytics

Total tests: 2165
Passed tests 1927
Skipped tests: 235
Failed tests: 3

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccComputeInstance_soleTenantNodeAffinities|TestAccCGCSnippet_eventarcWorkflowsExample|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample

modular-magician · 2022-09-07T08:40:04Z

Tests passed during RECORDING mode:
TestAccCGCSnippet_eventarcWorkflowsExample[Debug log]
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[Debug log]

Tests failed during RECORDING mode:
TestAccComputeInstance_soleTenantNodeAffinities[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

shuyama1 · 2022-09-09T08:32:33Z

mmv1/third_party/terraform/resources/resource_storage_bucket.go.erb

@@ -376,6 +378,22 @@ func resourceStorageBucket() *schema.Resource {
 	}
 }

+const resourceDataplexGoogleProvidedLabelPrefix = "labels.goog-dataplex"
+
+func resourceDataplexLabelDiffSuppress(k, old, new string, d *schema.ResourceData) bool {


The diff suppress function looks good! But are we able to add a unit test on this? It could be something like https://github.com/GoogleCloudPlatform/magic-modules/blob/main/mmv1/third_party/terraform/tests/resource_cloud_run_domain_mapping_test.go#L9

thanks, @shuyama1! :)
I've added some tests.

Here is the output of my added test:

=== RUN TestLabelDiffSuppress --- PASS: TestLabelDiffSuppress (0.00s)

Let me know if I need to change anything else (naming, .. etc.)

Thanks for adding the test!

modular-magician · 2022-09-12T09:07:18Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 126 insertions(+), 4 deletions(-))
Terraform Beta: Diff ( 2 files changed, 126 insertions(+), 4 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

modular-magician · 2022-09-12T10:12:12Z

Tests analytics

Total tests: 2170
Passed tests 1932
Skipped tests: 236
Failed tests: 2

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccComputeInstance_soleTenantNodeAffinities|TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample

modular-magician · 2022-09-12T10:17:22Z

Tests passed during RECORDING mode:
TestAccPrivatecaCertificateAuthority_privatecaCertificateAuthoritySubordinateExample[Debug log]

Tests failed during RECORDING mode:
TestAccComputeInstance_soleTenantNodeAffinities[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

shuyama1 · 2022-09-13T16:57:40Z

mmv1/third_party/terraform/tests/resource_storage_bucket_test.go.erb

+	for tn, tc := range cases {
+		if testLabelDiffSuppress(tc.K, tc.Old, tc.New) != tc.ExpectDiffSuppress {
+			t.Errorf("bad: %s, %q: %q => %q expect DiffSuppress to return %t", tn, tc.K, tc.Old, tc.New, tc.ExpectDiffSuppress)


Suggested change

for tn, tc := range cases {

if testLabelDiffSuppress(tc.K, tc.Old, tc.New) != tc.ExpectDiffSuppress {

t.Errorf("bad: %s, %q: %q => %q expect DiffSuppress to return %t", tn, tc.K, tc.Old, tc.New, tc.ExpectDiffSuppress)

for tn, tc := range cases {

if resourceDataplexLabelDiffSuppress(tc.K, tc.Old, tc.New) != tc.ExpectDiffSuppress {

t.Errorf("bad: %s, %q: %q => %q expect DiffSuppress to return %t", tn, tc.K, tc.Old, tc.New, tc.ExpectDiffSuppress)

No need to create a new function. resourceDataplexLabelDiffSuppress is the function we want to test and we can just call it directly here.

Thanks, @shuyama1 ! I think we need to adjust the function call to resourceDataplexLabelDiffSuppress(tc.K, tc.Old, tc.New, nil) instead of resourceDataplexLabelDiffSuppress(tc.K, tc.Old, tc.New) as the resourceDataplexLabelDiffSuppress expects four arguments and can't miss the last one - even though it is not used in the function.. :(

Make sense! Thanks!

shuyama1 · 2022-09-13T16:58:09Z

mmv1/third_party/terraform/resources/resource_storage_bucket.go.erb

@@ -376,6 +378,22 @@ func resourceStorageBucket() *schema.Resource {
 	}
 }

+const resourceDataplexGoogleProvidedLabelPrefix = "labels.goog-dataplex"
+
+func resourceDataplexLabelDiffSuppress(k, old, new string, d *schema.ResourceData) bool {


Thanks for adding the test!

modular-magician · 2022-09-28T08:57:50Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 102 insertions(+), 4 deletions(-))
Terraform Beta: Diff ( 2 files changed, 102 insertions(+), 4 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

modular-magician · 2022-09-28T08:58:55Z

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Failed tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

modular-magician · 2022-09-28T11:20:59Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 101 insertions(+), 4 deletions(-))
Terraform Beta: Diff ( 2 files changed, 101 insertions(+), 4 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

modular-magician · 2022-09-28T12:32:51Z

Tests analytics

Total tests: 2183
Passed tests 1941
Skipped tests: 239
Failed tests: 3

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccFirebaserulesRelease_BasicRelease|TestAccComputeInstance_soleTenantNodeAffinities|TestAccCGCSnippet_eventarcWorkflowsExample

modular-magician · 2022-09-28T12:38:13Z

Tests passed during RECORDING mode:
TestAccFirebaserulesRelease_BasicRelease[Debug log]

Tests failed during RECORDING mode:
TestAccComputeInstance_soleTenantNodeAffinities[Error message] [Debug log]
TestAccCGCSnippet_eventarcWorkflowsExample[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

shuyama1 · 2022-09-28T19:30:12Z

mmv1/third_party/terraform/tests/resource_storage_bucket_test.go.erb

+			Old:                "test-bucket",
+			New:                "test-bucket",


I don't think we need to test when old == new as I don't think the DiffSuppressFunc will be triggered if the values are the same. Maybe change New and Old values to be different and test if the diff will be suppressed (expect not to be suppress)

you are right! I've updated the cases so they make more sense! :)

modular-magician · 2022-09-29T07:22:58Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 101 insertions(+), 4 deletions(-))
Terraform Beta: Diff ( 2 files changed, 101 insertions(+), 4 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

modular-magician · 2022-09-29T08:28:05Z

Tests analytics

Total tests: 2192
Passed tests 1938
Skipped tests: 240
Failed tests: 14

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccFirebaserulesRelease_BasicRelease|TestAccComputeRouterInterface_basic|TestAccComputeInstance_soleTenantNodeAffinities|TestAccComputeForwardingRule_forwardingRuleRegionalHttpXlbExample|TestAccComputeForwardingRule_forwardingRuleExternallbExample|TestAccComputeForwardingRule_internalTcpUdpLbWithMigBackendExample|TestAccComputeVpnTunnel_vpnTunnelBetaExample|TestAccComputeGlobalForwardingRule_externalTcpProxyLbMigBackendExample|TestAccClouddeployDeliveryPipeline_DeliveryPipeline|TestAccComputeForwardingRule_networkTier|TestAccComputeForwardingRule_update|TestAccSqlDatabaseInstance_mysqlMajorVersionUpgrade|TestAccComputeFirewallPolicy_update|TestAccComputeFirewallPolicyRule_update

modular-magician · 2022-09-29T09:01:43Z

Tests passed during RECORDING mode:
TestAccFirebaserulesRelease_BasicRelease[Debug log]
TestAccComputeRouterInterface_basic[Debug log]
TestAccComputeForwardingRule_forwardingRuleRegionalHttpXlbExample[Debug log]
TestAccComputeForwardingRule_forwardingRuleExternallbExample[Debug log]
TestAccComputeForwardingRule_internalTcpUdpLbWithMigBackendExample[Debug log]
TestAccComputeVpnTunnel_vpnTunnelBetaExample[Debug log]
TestAccComputeGlobalForwardingRule_externalTcpProxyLbMigBackendExample[Debug log]
TestAccClouddeployDeliveryPipeline_DeliveryPipeline[Debug log]
TestAccComputeForwardingRule_networkTier[Debug log]
TestAccComputeForwardingRule_update[Debug log]
TestAccSqlDatabaseInstance_mysqlMajorVersionUpgrade[Debug log]
TestAccComputeFirewallPolicy_update[Debug log]
TestAccComputeFirewallPolicyRule_update[Debug log]

Tests failed during RECORDING mode:
TestAccComputeInstance_soleTenantNodeAffinities[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

shuyama1 · 2022-09-30T17:10:38Z

/gcbrun

modular-magician · 2022-09-30T17:22:33Z

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 2 files changed, 101 insertions(+), 4 deletions(-))
Terraform Beta: Diff ( 2 files changed, 101 insertions(+), 4 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

modular-magician · 2022-09-30T17:59:15Z

Tests analytics

Total tests: 2193
Passed tests 1948
Skipped tests: 240
Failed tests: 5

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccComputeInstance_soleTenantNodeAffinities|TestAccComputeForwardingRule_update|TestAccCGCSnippet_eventarcWorkflowsExample|TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample|TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample

modular-magician · 2022-09-30T18:02:52Z

Tests passed during RECORDING mode:
TestAccComputeForwardingRule_update[Debug log]
TestAccBigtableAppProfile_bigtableAppProfileMulticlusterExample[Debug log]
TestAccBigtableAppProfile_bigtableAppProfileAnyclusterExample[Debug log]

Tests failed during RECORDING mode:
TestAccComputeInstance_soleTenantNodeAffinities[Error message] [Debug log]
TestAccCGCSnippet_eventarcWorkflowsExample[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

shuyama1

LGTM! Thank you!

suppress dataplex labels

6d4333a

modular-magician requested a review from shuyama1 September 7, 2022 07:33

shuyama1 reviewed Sep 9, 2022

View reviewed changes

add test label suppress tests

9ad55e9

modular-magician requested a review from shuyama1 September 12, 2022 08:56

shuyama1 requested changes Sep 13, 2022

View reviewed changes

fix test

5916bbe

modular-magician requested a review from shuyama1 September 28, 2022 08:43

fix test

bfda4b0

remove unnecessary package from import

29b3dbc

shuyama1 reviewed Sep 28, 2022

View reviewed changes

adjust testcases

d91dc32

modular-magician requested a review from shuyama1 September 29, 2022 07:11

shuyama1 approved these changes Sep 30, 2022

View reviewed changes

shuyama1 merged commit 13f6fa6 into GoogleCloudPlatform:main Sep 30, 2022

This was referenced Sep 30, 2022

suppress dataplex labels hashicorp/terraform-provider-google-beta#4743

Merged

suppress dataplex labels hashicorp/terraform-provider-google#12694

Merged

prauc deleted the dataplex branch October 4, 2022 08:54

kunzese mentioned this pull request May 11, 2023

Fix inconsistent final plan when labels are added to resources #7923

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

suppress dataplex labels #6518

suppress dataplex labels #6518

prauc commented Sep 7, 2022 •

edited by shuyama1

Loading

modular-magician commented Sep 7, 2022

modular-magician commented Sep 7, 2022

modular-magician commented Sep 7, 2022

modular-magician commented Sep 7, 2022

shuyama1 Sep 9, 2022 •

edited

Loading

prauc Sep 12, 2022

shuyama1 Sep 13, 2022

modular-magician commented Sep 12, 2022

modular-magician commented Sep 12, 2022

modular-magician commented Sep 12, 2022

shuyama1 Sep 13, 2022

prauc Sep 28, 2022

shuyama1 Sep 28, 2022

shuyama1 Sep 13, 2022

modular-magician commented Sep 28, 2022

modular-magician commented Sep 28, 2022

modular-magician commented Sep 28, 2022

modular-magician commented Sep 28, 2022

modular-magician commented Sep 28, 2022

shuyama1 Sep 28, 2022 •

edited

Loading

prauc Sep 29, 2022

modular-magician commented Sep 29, 2022

modular-magician commented Sep 29, 2022

modular-magician commented Sep 29, 2022

shuyama1 commented Sep 30, 2022

modular-magician commented Sep 30, 2022

modular-magician commented Sep 30, 2022

modular-magician commented Sep 30, 2022

shuyama1 left a comment

suppress dataplex labels #6518

suppress dataplex labels #6518

Conversation

prauc commented Sep 7, 2022 • edited by shuyama1 Loading

modular-magician commented Sep 7, 2022

modular-magician commented Sep 7, 2022

Diff report:

modular-magician commented Sep 7, 2022

Tests analytics

Action taken

modular-magician commented Sep 7, 2022

shuyama1 Sep 9, 2022 • edited Loading

Choose a reason for hiding this comment

prauc Sep 12, 2022

Choose a reason for hiding this comment

shuyama1 Sep 13, 2022

Choose a reason for hiding this comment

modular-magician commented Sep 12, 2022

Diff report:

modular-magician commented Sep 12, 2022

Tests analytics

Action taken

modular-magician commented Sep 12, 2022

shuyama1 Sep 13, 2022

Choose a reason for hiding this comment

prauc Sep 28, 2022

Choose a reason for hiding this comment

shuyama1 Sep 28, 2022

Choose a reason for hiding this comment

shuyama1 Sep 13, 2022

Choose a reason for hiding this comment

modular-magician commented Sep 28, 2022

Diff report:

modular-magician commented Sep 28, 2022

Tests analytics

modular-magician commented Sep 28, 2022

Diff report:

modular-magician commented Sep 28, 2022

Tests analytics

Action taken

modular-magician commented Sep 28, 2022

shuyama1 Sep 28, 2022 • edited Loading

Choose a reason for hiding this comment

prauc Sep 29, 2022

Choose a reason for hiding this comment

modular-magician commented Sep 29, 2022

Diff report:

modular-magician commented Sep 29, 2022

Tests analytics

Action taken

modular-magician commented Sep 29, 2022

shuyama1 commented Sep 30, 2022

modular-magician commented Sep 30, 2022

Diff report:

modular-magician commented Sep 30, 2022

Tests analytics

Action taken

modular-magician commented Sep 30, 2022

shuyama1 left a comment

Choose a reason for hiding this comment

prauc commented Sep 7, 2022 •

edited by shuyama1

Loading

shuyama1 Sep 9, 2022 •

edited

Loading

shuyama1 Sep 28, 2022 •

edited

Loading