Accept prefix as metric_types for stackdriver metricset in GCP

[kaiyan-sheng]

What does this PR do?

This PR is to add accepting prefix using metric_types config parameter into stackdriver metricset for user to specify a prefix of the metric types instead of listing out all metric types one by one in the config.

Why is it important?

This will make stackdriver configuration a lot shorter/simpler. For example, if you want to collect all instance metric types for compute service, you can use:

- module: googlecloud
  metricsets:
    - stackdriver
  period: 1m
  metrics:
    - aligner: ALIGN_NONE
      service: compute
      metric_types: 
        - "instance"

Instead of listing all metric types like below:

- module: googlecloud
  metricsets:
    - stackdriver
  period: 1m
  metrics:
    - aligner: ALIGN_NONE
      service: compute
      metric_types:
        - "instance/cpu/reserved_cores"
        - "instance/cpu/scheduler_wait_time"
        - "instance/cpu/usage_time"
        - "instance/cpu/utilization"
        - "instance/disk/max_read_bytes_count"
        - "instance/disk/max_read_ops_count"
        - "instance/disk/max_write_bytes_count"
        - "instance/disk/max_write_ops_count"
        - "instance/disk/read_bytes_count"
        - "instance/disk/read_ops_count"
        ......

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Note: I need to spend some time on figuring out how to mock GCP APIs or add integration tests for GCP overall in a separate PR.

How to test this PR locally

Enable googlecloud module and change googlecloud.yml to:

- module: googlecloud
  metricsets:
    - stackdriver
  zone: "europe-west1-c"
  project_id: elastic-observability
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 1m
  metrics:
    - aligner: ALIGN_NONE
      service: compute
      metric_types: 
        - "instance/cpu"

Start Metricbeat and you should see these cpu metrics collected:

googlecloud.stackdriver.instance.cpu.reserved_cores.value
googlecloud.stackdriver.instance.cpu.usage_time.value
googlecloud.stackdriver.instance.cpu.utilization.value

Note: googlecloud.stackdriver.instance.cpu.scheduler_wait_time might not exist because it is only available for VMs that belong to the e2 family and it's for measuring the wait time for vCPU is ready to run but unexpectedly not scheduled to run.

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request #19345 updated]

• Start Time: 2020-07-01T21:24:21.123+0000

• Duration: 36 min 58 sec

Test stats 🧪

Test	Results
Failed	0
Passed	555
Skipped	128
Total	683

[elasticmachine]

Pinging @elastic/integrations-platforms (Team:Platforms)

[kaiyan-sheng]

@sorantis Does this metric_type_prefix align with your vision of configuring stackdriver metricset? TIA!

[sorantis]

@kaiyan-sheng looks good! Is it possible to reuse the same parameter though?

- module: googlecloud
  metricsets:
    - stackdriver
  period: 1m
  metrics:
    - aligner: ALIGN_NONE
      service: compute
      metric_types:
        - "instance/cpu"
        - "guest/memory"
        ......

If not, then can the metric_type_prefix param be configured with multiple prefixes?

[kaiyan-sheng]

@sorantis Yes, when we can use the same config, why add a new one 😄 I just changed the PR to accept prefix using metric_types config parameter. Thank you for the suggestion!!