[Monitoring] Metricbeat migration improvements #42600

chrisronline · 2019-08-05T13:02:42Z

This PR is step 4 of the Metricbeat setup wizard.

This PR is addressing issues that came out of testing and the other PRs, specifically the following items:

Add better UX for ES migration if the ES cluster is the one Kibana is talking to

This is working now. To test, first setup internal collection for Elasticsearch, then migrate (using the UI) to Metricbeat. After setting up monitoring through Metricbeat, there will be a callout and button at the top of the Elasticsearch nodes listing page (this isn't new in this PR, fyi). This button will automatically disable internal collection if the monitored cluster is the same one as the one Kibana is talking to. If it isn't, it will open the flyout with instructions. Please test both ways.

Share code better, but ensure it's consistent if possible with our areas of the codebase

This is a little dated, but we now have two routes instead of three. Please let me know if this is not sufficient and you think we should be using a single route.

We should fetch setup data at the same interval as the other data for the page. This will allow the setup mode UIs to automatically update periodically which is more seamless experience for the user. As a result of changing this, we may not need a Check Data button in the flyout anymore. This comment started the discussion.

This is done as well. The Check Data button has been removed and the experience is much more seamless now. Thanks @ycombinator!

Ensure this PR is applied to logstash, beats and APM panels

Done as well.

…p_wizard_step3

…uster

…p_wizard_step3

ycombinator · 2019-08-08T14:18:50Z

Looks good from the code point of view (with exception of some comments).

Going to test it functionally

@igoristic FWIW I've found that testing the PR functionally first and then reviewing the code changes second is often more efficient because any code changes from the functional review will need to be reviewed anyway.

elasticmachine · 2019-08-08T15:27:13Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

…_setup_wizard_step4

elasticmachine · 2019-08-09T14:34:41Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

ycombinator · 2019-08-12T19:20:52Z

I've started reviewing this PR. So far the only item I've been able to test is:

Add better UX for ES migration if the ES cluster is the one Kibana is talking to

This works as expected! 👍

However, while trying to test this item, I found two bugs unrelated to this PR: #43144 and #43145.

I will continue to test the rest of this PR but just wanted to report on my progress so far.

ycombinator · 2019-08-12T19:24:37Z

Share code better, but ensure it's consistent if possible with our areas of the codebase

This change LGTM as well.

elasticmachine · 2019-08-12T20:22:46Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

ycombinator · 2019-08-12T20:24:24Z

We should fetch setup data at the same interval as the other data for the page.

Tested this and it works as expected. 👍

ycombinator · 2019-08-12T21:10:55Z

Ensure this PR is applied to logstash, beats and APM panels

This didn't work as expected for Logstash. I ran Logstash with a simple pipeline (input { stdin {} } output { elasticsearch {} }) which ended up creating an index names logstash-2019.08.12-000001. When I go the Cluster Overview page in Setup Mode, I expected to see a flag icon in the Logstash box, but there was none.

chrisronline · 2019-08-13T18:43:37Z

This didn't work as expected for Logstash. I ran Logstash with a simple pipeline (input { stdin {} } output { elasticsearch {} }) which ended up creating an index names logstash-2019.08.12-000001. When I go the Cluster Overview page in Setup Mode, I expected to see a flag icon in the Logstash box, but there was none.

Nice find! I updated the way we detect these which should fix this!

…p_wizard_step4

elasticmachine · 2019-08-13T19:46:30Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

chrisronline · 2019-08-14T14:31:57Z

@igoristic @ycombinator This is ready for another round!

ycombinator · 2019-08-14T16:56:45Z

Nice find! I updated the way we detect these which should fix this!

I'm still seeing this issue. Could you double check on your end please? 😄

chrisronline · 2019-08-14T17:09:03Z

@ycombinator This should be addressed now in 7684403

…le all internal collection

ycombinator

Functionally this PR LGTM now. Over to @igoristic for code review.

elasticmachine · 2019-08-14T18:50:25Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

igoristic · 2019-08-14T22:26:24Z

Thanks for making the changes! LGTM. Great job!

* Support for logstash * Beats support * Fix cherry-pick api issue * Support for logstash * Updates for beats and logstash * APM migration working * Tweaks for beats migration * Update copy for setup new button * If on cloud, disable setup mode * Handle new beat flow better * Better phrasing for APM * Add beat type to disable step * Fix i18n issue * Fix jest tests * Fix api tests * PR feedback * Update copy * Remove unnecessary code * Support shortcut to finish ES migration if we are on the connected cluster * Undo changes that are now in a separate PR * Share code better by exposing a single route with an optional parameter * Disable more links * Fix overview link for logstash * PR feedback * Fix tests * PR feedback * Fix tests * We still need this route * Updates * Only show if there are instances too * Change how we set the newly discovered cluster uuid * Move this to support beats/apm weirdness * Fix tests * Remove out of date translations * PR feedback * Detect products a different way, to detect empty indices * Look against the production cluster instead of the monitoring one * When disabling ES internal collection, only disable ES - do not disable all internal collection

chrisronline · 2019-08-15T17:47:14Z

Backport:

7.x: 3434d05

chrisronline added 30 commits July 22, 2019 11:09

Support for logstash

4b343af

Beats support

8d85b2b

Fix cherry-pick api issue

9c0c2ee

Support for logstash

142ac48

Updates for beats and logstash

f15cd75

APM migration working

b5f928d

Tweaks for beats migration

8e0453e

Update copy for setup new button

164df37

If on cloud, disable setup mode

99d546b

Handle new beat flow better

16861c1

Better phrasing for APM

911b3b1

Add beat type to disable step

bc03b08

Fix i18n issue

94dcb34

Fix jest tests

e009eb3

Fix api tests

7ac0ad3

PR feedback

f3943a7

Update copy

35d48d4

Merge remote-tracking branch 'elastic/master' into monitoring/mb_setu…

de7792c

…p_wizard_step3

Remove unnecessary code

0d47d3a

Support shortcut to finish ES migration if we are on the connected cl…

788477b

…uster

Undo changes that are now in a separate PR

d34ccc3

Share code better by exposing a single route with an optional parameter

8103231

Disable more links

146a589

Fix overview link for logstash

5a735ec

PR feedback

7605f91

Merge remote-tracking branch 'elastic/master' into monitoring/mb_setu…

aba082c

…p_wizard_step3

Fix tests

a58514e

PR feedback

ee42eb2

Merge in part3

0cdc4ef

Merge in master

2402c74

PR feedback

1f59836

Merge branch 'master' of github.com:elastic/kibana into monitoring/mb…

7239bfe

…_setup_wizard_step4

chrisronline mentioned this pull request Aug 13, 2019

[Setup UI] Monitoring ES cluster shows other clusters' nodes #43145

Closed

Detect products a different way, to detect empty indices

f64226c

Merge remote-tracking branch 'elastic/master' into monitoring/mb_setu…

3e7d379

…p_wizard_step4

Look against the production cluster instead of the monitoring one

7684403

When disabling ES internal collection, only disable ES - do not disab…

4d13ccb

…le all internal collection

ycombinator approved these changes Aug 14, 2019

View reviewed changes

igoristic approved these changes Aug 14, 2019

View reviewed changes

chrisronline merged commit cb2dd9f into elastic:master Aug 15, 2019

chrisronline deleted the monitoring/mb_setup_wizard_step4 branch August 15, 2019 16:14

chrisronline mentioned this pull request Aug 15, 2019

[7.x] [Monitoring] Metricbeat migration improvements (#42600) #43364

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Monitoring] Metricbeat migration improvements #42600

[Monitoring] Metricbeat migration improvements #42600

chrisronline commented Aug 5, 2019 •

edited

Loading

ycombinator commented Aug 8, 2019

elasticmachine commented Aug 8, 2019

elasticmachine commented Aug 9, 2019

ycombinator commented Aug 12, 2019 •

edited

Loading

ycombinator commented Aug 12, 2019

elasticmachine commented Aug 12, 2019

ycombinator commented Aug 12, 2019

ycombinator commented Aug 12, 2019

chrisronline commented Aug 13, 2019

elasticmachine commented Aug 13, 2019

chrisronline commented Aug 14, 2019

ycombinator commented Aug 14, 2019

chrisronline commented Aug 14, 2019

ycombinator left a comment

elasticmachine commented Aug 14, 2019

igoristic commented Aug 14, 2019

chrisronline commented Aug 15, 2019

[Monitoring] Metricbeat migration improvements #42600

[Monitoring] Metricbeat migration improvements #42600

Conversation

chrisronline commented Aug 5, 2019 • edited Loading

ycombinator commented Aug 8, 2019

elasticmachine commented Aug 8, 2019

💚 Build Succeeded

elasticmachine commented Aug 9, 2019

💚 Build Succeeded

ycombinator commented Aug 12, 2019 • edited Loading

ycombinator commented Aug 12, 2019

elasticmachine commented Aug 12, 2019

💔 Build Failed

ycombinator commented Aug 12, 2019

ycombinator commented Aug 12, 2019

chrisronline commented Aug 13, 2019

elasticmachine commented Aug 13, 2019

💚 Build Succeeded

chrisronline commented Aug 14, 2019

ycombinator commented Aug 14, 2019

chrisronline commented Aug 14, 2019

ycombinator left a comment

Choose a reason for hiding this comment

elasticmachine commented Aug 14, 2019

💚 Build Succeeded

igoristic commented Aug 14, 2019

chrisronline commented Aug 15, 2019

chrisronline commented Aug 5, 2019 •

edited

Loading

ycombinator commented Aug 12, 2019 •

edited

Loading