Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make elasticsearch/index_recovery metricset work for Stack Monitoring without xpack.enabled flag #20810

Conversation

sayden
Copy link
Contributor

@sayden sayden commented Aug 26, 2020

WIP

Missing steps:

  • metricset fields.yml
  • elasticsearch/fields.yml aliases
  • Manual testing in Kibana

@sayden sayden self-assigned this Aug 26, 2020
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 26, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Aug 26, 2020

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Pull request #20810 updated

  • Start Time: 2020-12-01T14:22:41.882+0000

  • Duration: 78 min 34 sec

Test stats 🧪

Test Results
Failed 0
Passed 2246
Skipped 526
Total 2772

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test Results
Failed 0
Passed 2246
Skipped 526
Total 2772

@sayden sayden added Metricbeat Metricbeat Team:Services (Deprecated) Label for the former Integrations-Services team labels Aug 27, 2020
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 27, 2020
@sayden sayden marked this pull request as ready for review October 22, 2020 10:25
@elasticmachine
Copy link
Collaborator

Pinging @elastic/stack-monitoring (Stack monitoring)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

Copy link
Contributor

@chrisronline chrisronline left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@andresrc
Copy link
Contributor

@ycombinator I think this is ready for your final review, thanks!

},
"index": {
"name": ".monitoring-es-6-2018.11.20",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are removing this field, which would be a breaking change. We should add it back.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bump

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind, I see this has been updated now.

"name": "es1_1"
},
"type": "EMPTY_STORE"
"shards": [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, this is a tricky change.

This appears to be a breaking change too. Not so much defining a new elasticsearch.index.recovery.shards field because that's an additive change. But more that we're taking away the elasticsearch.index.recovery.id, elasticsearch.index.recovery.primary, etc. fields.

I think the reason you had to do this is because previously each document produced by the index_recovery metricset represented a single shard, whereas what Stack Monitoring expects each document to contain an array of shards.

@chrisronline what do you need the data type of this elasticsearch.index.recovery.shards field to be for queries to work correctly: object or nested? Depending on that we might need to figure out how we handle this breaking change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't perform any aggregations across this data, so object works fine.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, that's good news.

I'm still concerned about the breaking change in document structure here, for any existing users of the elasticsearch/index_recovery metricset (non-xpack).

@chrisronline How much work would it be to make the UI work on the current (before this PR) structure of elasticsearch/index_recovery metricset (non-xpack) documents where one document == one shard? Keep in mind that this would be code we'd get to clean up in 8.0 as we could introduce a breaking change in document structure in this metricset at that point.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be possible. We only access this data from a single code path and adding any kind of funky logic in there is fine, knowing it can go away in 8.0.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much @chrisronline ❤️ . Essentially you'll need logic to check for the new structure (1 document == 1 shard) that will be produced by this metricset when its run without the xpack.enabled flag but fall back to the current logic of expecting multiple shard inner documents in a single type: index_recovery document. Hope that makes sense.

In that case, @sayden, we should keep the 1 document == 1 shard structure that is currently (prior to this PR) being produced by this metricset. We can add more fields to that document, if needed for Stack Monitoring, of course.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ycombinator Yes, that's my understanding. Should be fine

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah! Ok! Sounds good! I'll revert to the OSS original structure of 1 doc == 1 shard.

@ycombinator
Copy link
Contributor

@sayden I don't see a fields.yml file in this PR.

@sayden
Copy link
Contributor Author

sayden commented Nov 3, 2020

That's because there's no "type": "index_recovery" in the monitoring mapping so the original fields.yml was untouched. I can add the fields though

@sayden sayden force-pushed the feature/mb/elasticsearch/index_recovery_xpack_flag branch from cab7374 to b169e66 Compare November 4, 2020 22:35
@ycombinator
Copy link
Contributor

ycombinator commented Nov 5, 2020

Hey @sayden, could you update the PR description please? It's not clear if this PR is still WIP and there are things you are still working on or if it's ready for me to review.

Also, there are failing tests that look related, so I'm assuming you need to do more on this PR before it's ready for me to review again.


In general, for each of these PRs, can I suggest a bit of process please?

  • Please keep the PR descriptions up to date at all times. Since there are so many such PRs open at the same time, it's hard to keep track of their states in my head. Being able to just look at a PR's description to know where things stand would be nice.

  • Whenever a PR is ready for my review (code review), please request my review by adding me to the Reviewers section in the top right of the PR:
    image

  • If I'm already a reviewer and the PR is ready for a re-review, please request my review again by clicking the "re-request review" icon next to my name:
    image

Otherwise right now I'm having to periodically look at each PR and try to guess what state it's in, and whether it's awaiting my review. Thanks!

# Conflicts:
#	metricbeat/docs/fields.asciidoc
#	metricbeat/module/elasticsearch/fields.go
#	metricbeat/module/elasticsearch/node_stats/_meta/fields.yml
@sayden sayden force-pushed the feature/mb/elasticsearch/index_recovery_xpack_flag branch from b169e66 to 02be3b3 Compare November 5, 2020 17:45
@elasticmachine
Copy link
Collaborator

elasticmachine commented Nov 5, 2020

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Test stats 🧪

Test Results
Failed 2
Passed 2248
Skipped 500
Total 2750

Genuine test errors 2

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Build&Test / metricbeat-goIntegTest / TestXPackEnabled – elasticsearch
  • Name: Build&Test / metricbeat-goIntegTest / TestXPackEnabled/index_recovery – elasticsearch

@sayden sayden force-pushed the feature/mb/elasticsearch/index_recovery_xpack_flag branch from b822954 to 5f8560d Compare November 10, 2020 16:51
Copy link
Contributor

@ycombinator ycombinator left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code changes LGTM but tests are failing in CI and test failures look relevant.

@sayden sayden force-pushed the feature-stack-monitoring-mb-ecs branch from 58e2a7d to 9de5959 Compare November 12, 2020 11:44
@sayden sayden merged commit 74491e9 into elastic:feature-stack-monitoring-mb-ecs Dec 1, 2020
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Stack Monitoring Metricbeat Metricbeat Team:Services (Deprecated) Label for the former Integrations-Services team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants