[Monitoring] Metricbeat migration net new user experience #39832

chrisronline · 2019-06-27T19:05:16Z

This PR is step 2, part 2 of the Metricbeat setup wizard and is also known as the "net new user" experience.

The main purpose of this PR is to provide a UX for new users to enable monitoring for their stack products. Currently, this PR is only concerned with Elasticsearch and Kibana, but a future PR will add support for other stack products (This is because there is/was outstanding work to enable metricbeat collection for these other products).

The current UX for enabling monitoring is extremely simple - it's just a button click in the UI. However, because of the need to install and start a beat, the process is slightly more complicated but the work done in this PR should help the user understand how to enable monitoring, and also guide the user through the experience too.

Testing

Elasticsearch monitoring

Start up ES
Start up Kibana
Load Kibana in the browser
Navigate to the stack monitoring application
You should land on the no-data page which is the default experience until step 4 is merged
To overwrite this behavior to test this PR, manually navigate to http://localhost:5601/app/monitoring#/loading?_g=(inSetupMode:!t) which will route you through the new user setup flow created through this PR
Follow instructions on the page to enable Elasticsearch monitoring with Metricbeat

Kibana monitoring

Follow up steps above to enable Elasticsearch monitoring with Metricbeat
Navigate to the stack monitoring application, ensuring setupMode is enabled.
- The url should look like: http://localhost:5601/app/monitoring#/overview?_g=(cluster_uuid:yhZt4d92TLKQd8YtH-JO-Q,inSetupMode:!t)
Scroll down to see the Kibana section on the overview page
Note that you'll see Instances: 0 (0/1)
Click Instances: 0
Follow instructions on the page to enable Kibana monitoring through Metricbeat

Edge cases

Please test with multiple instances of ES
I don't really see a need to test multiple instances of Kibana, as there is no way for us to know ahead of time that there is another kibana instance (so we can suggest they add monitoring to it).

…p_wizard_step2_part2

elasticmachine · 2019-07-01T18:43:38Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2019-07-01T19:47:36Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

…enabled on a stack product

elasticmachine · 2019-07-02T16:28:31Z

Pinging @elastic/stack-monitoring

elasticmachine · 2019-07-02T18:10:30Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

ycombinator · 2019-07-03T07:46:32Z

Start up ES - ensure any monitoring settings are disabled
Start up Kibana, ensuring the xpack.monitoring.kibana.collection.enabled flag is set to false

I'm not sure these steps should be a prerequisite for testing the UX introduced by this PR. A net new user will start with default settings. So we should make sure the UX introduced by this PR works with these default settings, right?

ycombinator · 2019-07-03T07:51:39Z

Load Kibana in the browser
Navigate to the stack monitoring application
Note that you will no longer see the "No data screen", instead you will be taken to the Elasticsearch nodes listing page

Does this mean that when this PR is merged the user will start seeing the new UX already? If so, I thought we don't want this to be the case until the Step 4 PR is merged, right?

chrisronline · 2019-07-10T15:23:52Z

@ycombinator @igoristic I've addressed your feedback and this is ready for another round of review.

@ycombinator In terms of this comment, I've decided to go with this visual, using EUI colors to indicate status. LMK what you think

elasticmachine · 2019-07-10T16:28:23Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

ycombinator · 2019-07-10T17:32:00Z

I took another pass at this PR. All my previous feedback has been addressed.

I've decided to go with this visual, using EUI colors to indicate status.

I like this change. I also noticed the tooltips and found them to be helpful. I think the wording in them could be improved but that's for later as part of overall language cleanup anyway.

I noticed #39832 (comment) hasn't been addressed yet but chatting off-PR, I know you were thinking of addressing this in a follow up PR also across the board. Can you add this to the meta issue so we don't lose track of it?

Finally, one use case that I hadn't previously thought of:

What if a net new user has Beats or Logstash or APM Server also running and wants to know how to setup Metricbeat monitoring for those. Should we show sections for these on the cluster overview page with buttons linking to setup instructions (or something else to the same effect)?

chrisronline · 2019-07-10T17:55:22Z

Can you add this to the meta issue so we don't lose track of it?

Added.

What if a net new user has Beats or Logstash or APM Server also running and wants to know how to setup Metricbeat monitoring for those. Should we show sections for these on the cluster overview page with buttons linking to setup instructions (or something else to the same effect)?

Agreed. This will be added/covered in Step 3 from the meta

ycombinator

Functionally LGTM!

chrisronline · 2019-07-15T14:24:31Z

@igoristic I think I addressed all your points, but let me know if I missed something and I'd be happy to fix it up!

igoristic · 2019-07-18T16:46:26Z

@igoristic I think I addressed all your points, but let me know if I missed something and I'd be happy to fix it up!

@chrisronline Everything looks good for the most part 👍 Also, looks great functionally 😄

Still a little concerned about:
#39832 (comment)
#39832 (comment)

But, might not be a big deal here, and probably just a matter of preference

…p_wizard_step2_part2

elasticmachine · 2019-07-18T20:00:42Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

… products

elasticmachine · 2019-07-18T21:31:25Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

chrisronline · 2019-07-19T13:24:39Z

@ycombinator @igoristic Some changes came in to support our existing api tests that you might want to review here: c63327c

ycombinator

LGTM.

elasticmachine · 2019-07-19T17:42:10Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

spalger · 2019-07-19T17:48:08Z

retest

elasticmachine · 2019-07-19T18:49:33Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

…_setup_wizard_step2_part2

elasticmachine · 2019-07-22T13:41:40Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

…b-panel-for-stopping-jobs * 'master' of github.com:elastic/kibana: (58 commits) [DOCS] Timelion cleanup (elastic#41381) [Docs] Add simple phrase highlighting to Logs UI (elastic#41610) [Maps] Rename modules for clarity (elastic#41608) [Monitoring] Metricbeat migration net new user experience (elastic#39832) [Maps] Only color legend icon with dynamic color when dynamic config is complete (elastic#41607) [TSVB] [Markdown] markdown section do not render after change data parameter (elastic#41576) [Vega] (Step 2) Shim new platform - renaming vega -> vis_type_vega (elastic#41565) update dark mode tsvb test (elastic#41618) [i18n] .i18nrc file as the source of truth and enhance tooling (elastic#39774) Reactify Top Nav Menu (kbn_top_nav) (elastic#40262) fix(code/frontend): should update search results if search options change (elastic#41232) Use kibana-ci-proxy-cache for chrome and gecko drivers (elastic#41581) [SIEM] Fix draggables to work with escapeId for the ML severity column (elastic#41621) [Canvas] Updates esdocs default count to 1000 (elastic#41604) [Uptime] Fix duration chart for Safari (elastic#41619) [Canvas] Restores "Today" as a quick time range in time filter (elastic#41528) docs: lowercase app (elastic#41612) [Code] Update git repository update frequency (elastic#41541) Remove language=json on code blocks due to performance hit (elastic#41540) [DOCS] Update anchors and links for Elasticserach API relocation. (elastic#41372) ...

…41666) * ES net new user experience * Adding sorting support * WIP * Progress on the ES nodes listing page * Fix the loading page * Kibana net new user experience * Remove unnecessary files * Remove this one too * Add this file back in * Add a way to add the cluster uuid to global state once monitoring is enabled on a stack product * Ensure this PR can be merged to master by hiding the functionality behind direct links * Add messaging for net new user experience on Kibana instances page * Ensure we poll for more data as soon as they are done in the flyout * PR feedback * Fix api tests * Add `skipLiveData` flag to avoid fetching data from the running stack products * Add comment

chrisronline · 2019-08-01T16:19:59Z

Backport:

7.x: 01d4b69

chrisronline added 8 commits June 14, 2019 16:33

ES net new user experience

4de8f56

Merge remote-tracking branch 'elastic/master' into monitoring/mb_setu…

1b34d68

…p_wizard_step2_part2

Adding sorting support

0f276fe

Merge in master

85c00c1

WIP

32ed8a7

Progress on the ES nodes listing page

7fbb375

Fix the loading page

b034a59

Kibana net new user experience

cc415ec

chrisronline added 3 commits July 1, 2019 14:51

Remove unnecessary files

b442366

Remove this one too

931bcee

Add this file back in

d18b41d

ycombinator mentioned this pull request Jul 2, 2019

Implement a UI within Stack Monitoring to help users migrate to MB shipping/collecting #39010

Closed

5 tasks

Add a way to add the cluster uuid to global state once monitoring is …

7307db0

…enabled on a stack product

chrisronline marked this pull request as ready for review July 2, 2019 16:28

chrisronline requested review from igoristic and ycombinator July 2, 2019 16:28

chrisronline self-assigned this Jul 2, 2019

chrisronline added release_note:skip Skip the PR/issue when compiling release notes review Team:Monitoring Stack Monitoring team v7.4.0 v8.0.0 labels Jul 2, 2019

chrisronline removed the review label Jul 3, 2019

chrisronline removed the request for review from ycombinator July 3, 2019 10:22

PR feedback

cfaaed1

ycombinator approved these changes Jul 10, 2019

View reviewed changes

igoristic approved these changes Jul 18, 2019

View reviewed changes

chrisronline added 2 commits July 18, 2019 14:58

Merge remote-tracking branch 'elastic/master' into monitoring/mb_setu…

3c48f84

…p_wizard_step2_part2

Fix api tests

fe58db8

Add skipLiveData flag to avoid fetching data from the running stack…

c63327c

… products

Add comment

e508199

ycombinator approved these changes Jul 19, 2019

View reviewed changes

Merge branch 'master' of github.com:elastic/kibana into monitoring/mb…

89dd59c

…_setup_wizard_step2_part2

chrisronline merged commit 6356088 into elastic:master Jul 22, 2019

chrisronline deleted the monitoring/mb_setup_wizard_step2_part2 branch July 22, 2019 14:12

This was referenced Jul 22, 2019

[7.x] [Monitoring] Metricbeat migration net new user experience (#39832) #41666

Merged

[Monitoring] Metricbeat migration for logstash, beats and apm #40442

Merged

chrisronline mentioned this pull request Aug 6, 2019

[Monitoring] Metricbeat migration improvements #42600

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Monitoring] Metricbeat migration net new user experience #39832

[Monitoring] Metricbeat migration net new user experience #39832

chrisronline commented Jun 27, 2019 •

edited

Loading

elasticmachine commented Jul 1, 2019

elasticmachine commented Jul 1, 2019

elasticmachine commented Jul 2, 2019

elasticmachine commented Jul 2, 2019

ycombinator commented Jul 3, 2019

ycombinator commented Jul 3, 2019 •

edited

Loading

chrisronline commented Jul 10, 2019

elasticmachine commented Jul 10, 2019

ycombinator commented Jul 10, 2019

chrisronline commented Jul 10, 2019

ycombinator left a comment

chrisronline commented Jul 15, 2019

igoristic commented Jul 18, 2019

elasticmachine commented Jul 18, 2019

elasticmachine commented Jul 18, 2019

chrisronline commented Jul 19, 2019

ycombinator left a comment

elasticmachine commented Jul 19, 2019

spalger commented Jul 19, 2019

elasticmachine commented Jul 19, 2019

elasticmachine commented Jul 22, 2019

chrisronline commented Aug 1, 2019

[Monitoring] Metricbeat migration net new user experience #39832

[Monitoring] Metricbeat migration net new user experience #39832

Conversation

chrisronline commented Jun 27, 2019 • edited Loading

Testing

Elasticsearch monitoring

Kibana monitoring

Edge cases

elasticmachine commented Jul 1, 2019

💔 Build Failed

elasticmachine commented Jul 1, 2019

💔 Build Failed

elasticmachine commented Jul 2, 2019

elasticmachine commented Jul 2, 2019

💔 Build Failed

ycombinator commented Jul 3, 2019

ycombinator commented Jul 3, 2019 • edited Loading

chrisronline commented Jul 10, 2019

elasticmachine commented Jul 10, 2019

💔 Build Failed

ycombinator commented Jul 10, 2019

chrisronline commented Jul 10, 2019

ycombinator left a comment

Choose a reason for hiding this comment

chrisronline commented Jul 15, 2019

igoristic commented Jul 18, 2019

elasticmachine commented Jul 18, 2019

💔 Build Failed

elasticmachine commented Jul 18, 2019

💚 Build Succeeded

chrisronline commented Jul 19, 2019

ycombinator left a comment

Choose a reason for hiding this comment

elasticmachine commented Jul 19, 2019

💔 Build Failed

spalger commented Jul 19, 2019

elasticmachine commented Jul 19, 2019

💔 Build Failed

elasticmachine commented Jul 22, 2019

💚 Build Succeeded

chrisronline commented Aug 1, 2019

chrisronline commented Jun 27, 2019 •

edited

Loading

ycombinator commented Jul 3, 2019 •

edited

Loading