Add field metastore_config to google_dataproc_cluster

[shuyama1]

This resolves hashicorp/terraform-provider-google#9419

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Generated Terraform, and ran make test and make lint to ensure it passes unit and linter tests.
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Ran relevant acceptance tests (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

dataproc: added field `metastore_config` to `google_dataproc_cluster` (beta)

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 1 file changed, 21 insertions(+))
Terraform Beta: Diff ( 3 files changed, 158 insertions(+))

[shuyama1]

/gcbrun

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform GA: Diff ( 1 file changed, 21 insertions(+))
Terraform Beta: Diff ( 4 files changed, 160 insertions(+), 3 deletions(-))

[modular-magician]

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccComputeInstanceFromMachineImage_basic|TestAccComputeInstanceFromMachineImage_overrideMetadataDotStartupScript|TestAccDataprocCluster_withMetastoreConfig|TestAccPrivatecaCertificate_privatecaCertificateWithTemplateExample|TestAccPrivatecaCaPool_privatecaCapoolEmptyBaseline You can view the result here: https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=203110

[c2thorn]

Great PR, looks very solid!
One thing I noticed was that we are making the field beta-only. From my quick search, I couldn't find anything that indicated this wasn't GA. I found https://cloud.google.com/dataproc-metastore/docs/attach-dataproc which didn't indicate any beta features. Could this work in GA?

[shuyama1]

I found https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/dataproc_metastore_service, and it says the resource google_dataproc_metastore_service is still in beta, and that's why I make the field beta-only. Should I make it in GA instead? Anyways, let me try if this works in GA first.

[shuyama1]

I tried it in GA and it didn't work, as the resource google_dataproc_metastore_service is not supported in GA provider. But since the API did indicate it as a beta feature, should we also bring the resource google_dataproc_metastore_service to GA?

[c2thorn]

@shuyama1 Gotcha, didn't realize the TF resource equivalent was still in beta. I think it is fine to leave as is, and move over when we GA dataproc metastore service. We could GA it now, but I believe that is out of scope for this task. Would you mind filing an issue to GA both google_dataproc_metastore_service and this field?

[shuyama1]

Sure! Will do!

[modular-magician]

Hi! I'm the modular magician. Your PR generated some diffs in downstreams - here they are.

Diff report:

Terraform Beta: Diff ( 1 file changed, 1 insertion(+), 1 deletion(-))

[modular-magician]

I have triggered VCR tests in RECORDING mode for the following tests that failed during VCR: TestAccPrivatecaCaPool_privatecaCapoolEmptyBaseline You can view the result here: https://ci-oss.hashicorp.engineering/viewQueued.html?itemId=203197

[modular-magician]

Tests failed during RECORDING mode: TestAccPrivatecaCaPool_privatecaCapoolEmptyBaseline Please fix these to complete your PR