terraform/k8s-infra-prow-build: add dashboards

[spiffxp]

Related:

• Part of: Setup a gitops-driven workflow for google cloud monitoring dashboards #1376
• Part of: [Umbrella issue] How we monitor k8s-infra ?  #2588

I added these to the k8s-infra-prow-build terraform module since it's been handling everything for this GCP project. I'm happy to move to the k8s-infra-monitoring module if that's your preference @ameukam

Actually "deploying" this is going to require a terraform import since the dashboards already exist (these were copied out of audit and piped through jq for legibility)

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• infra/gcp/terraform/k8s-infra-prow-build/OWNERS [spiffxp]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[spiffxp]

/uncc @e-blackwelder @michelle192837
/cc @ameukam

/cc @cjwagner @chaodaiG
In case you have other suggestions based on the terraform work you've done for oss-test-infra

[spiffxp]

/hold
for comment

[chaodaiG]

@cjwagner and I were discussing about this, mainly around whether to source control dashboard(s) or not. For us it's still at exploratory state, so we might not want to source control the exported json file, at least not at this time, on the other hand we are not seeing any one manually editing this json for dashboard changes.

The main reason for exporting the json, is for serving the purpose of rolling it back and being able to track history. So alternatively we are considering automating the json export to GCS, which we'll be using a versioned blob so that we can track history. The automation can potentially be done by a periodic prow job, or even Cloud Run etc.

[ameukam]

I don't have a strong preference for the moment. I think I want some answers to the meta-questions (#2588) related to k8s-infra monitoring before we decide what's the correct way to handle monitoring dashboards.

/lgtm

[spiffxp]

/hold cancel

[spiffxp]

for f in dashboards/*.json; do 
  echo $f; 
  file=$(basename $f); 
  name=$(<$f jq -r .name); 
  terraform import google_monitoring_dashboard.dashboards[\"$file\"] $name;
done