Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cherry-pick #19345 to 7.x: Accept prefix as metric_types for stackdriver metricset in GCP #19606

Merged
merged 2 commits into from
Jul 2, 2020
Merged

Cherry-pick #19345 to 7.x: Accept prefix as metric_types for stackdriver metricset in GCP #19606

merged 2 commits into from
Jul 2, 2020

Conversation

kaiyan-sheng
Copy link
Contributor

@kaiyan-sheng kaiyan-sheng commented Jul 2, 2020

Cherry-pick of PR #19345 to 7.x branch. Original message:

What does this PR do?

This PR is to add accepting prefix using metric_types config parameter into stackdriver metricset for user to specify a prefix of the metric types instead of listing out all metric types one by one in the config.

Why is it important?

This will make stackdriver configuration a lot shorter/simpler. For example, if you want to collect all instance metric types for compute service, you can use:

- module: googlecloud
  metricsets:
    - stackdriver
  period: 1m
  metrics:
    - aligner: ALIGN_NONE
      service: compute
      metric_types: 
        - "instance"

Instead of listing all metric types like below:

- module: googlecloud
  metricsets:
    - stackdriver
  period: 1m
  metrics:
    - aligner: ALIGN_NONE
      service: compute
      metric_types:
        - "instance/cpu/reserved_cores"
        - "instance/cpu/scheduler_wait_time"
        - "instance/cpu/usage_time"
        - "instance/cpu/utilization"
        - "instance/disk/max_read_bytes_count"
        - "instance/disk/max_read_ops_count"
        - "instance/disk/max_write_bytes_count"
        - "instance/disk/max_write_ops_count"
        - "instance/disk/read_bytes_count"
        - "instance/disk/read_ops_count"
        ......

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Note: I need to spend some time on figuring out how to mock GCP APIs or add integration tests for GCP overall in a separate PR.

How to test this PR locally

Enable googlecloud module and change googlecloud.yml to:

- module: googlecloud
  metricsets:
    - stackdriver
  zone: "europe-west1-c"
  project_id: elastic-observability
  credentials_file_path: "your JSON credentials file path"
  exclude_labels: false
  period: 1m
  metrics:
    - aligner: ALIGN_NONE
      service: compute
      metric_types: 
        - "instance/cpu"

Start Metricbeat and you should see these cpu metrics collected:

googlecloud.stackdriver.instance.cpu.reserved_cores.value
googlecloud.stackdriver.instance.cpu.usage_time.value
googlecloud.stackdriver.instance.cpu.utilization.value

Note: googlecloud.stackdriver.instance.cpu.scheduler_wait_time might not exist because it is only available for VMs that belong to the e2 family and it's for measuring the wait time for vCPU is ready to run but unexpectedly not scheduled to run.

* Add metric_types for stackdriver metricset in GCP

(cherry picked from commit 6bfe793)
@kaiyan-sheng kaiyan-sheng added [zube]: In Review backport Team:Platforms Label for the Integrations - Platforms team labels Jul 2, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jul 2, 2020
@kaiyan-sheng kaiyan-sheng self-assigned this Jul 2, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jul 2, 2020

💚 Build Succeeded

Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: [Pull request #19606 updated]

  • Start Time: 2020-07-02T14:12:52.043+0000

  • Duration: 45 min 44 sec

Test stats 🧪

Test Results
Failed 0
Passed 547
Skipped 128
Total 675

@kaiyan-sheng kaiyan-sheng merged commit e5db04f into elastic:7.x Jul 2, 2020
@kaiyan-sheng kaiyan-sheng deleted the backport_19345_7.x branch July 2, 2020 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport Team:Platforms Label for the Integrations - Platforms team [zube]: Done
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants