[Monitoring/Reporting Stats] Move hardcoded Reporting stats type collection out of monitoring plugin #18894

tsullivan · 2018-05-08T00:02:13Z

Registers the reporting collector in plugins/reporting/index.js instead of plugins/monitoring/server/kibana_monitoring/collectors/get_reporting_collector.js
KIBANA_REPORTING_TYPE moved from the Monitoring plugin to the Reporting plugin

Depends on:

tsullivan · 2018-05-08T00:02:57Z

x-pack/plugins/monitoring/server/kibana_monitoring/init_kibana_monitoring.js

@@ -21,7 +21,8 @@ export async function initKibanaMonitoring(kbnServer, server) {
      plugins: [monitoringBulk]
    });

-    startCollector(kbnServer, server, client);
+    const collector = startCollector(kbnServer, server, client);
+    server.expose('typeCollector', collector); // NOTE: consumers need to wait on Monitoring plugin status go green, and that this object was successfully exposed


The note is relevant to line 163 in x-pack/plugins/reporting/index.js

elasticmachine · 2018-05-08T01:07:02Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-05-08T17:56:02Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-05-08T18:43:56Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

tsullivan · 2018-05-08T19:16:15Z

I've removed review, added WIP label.

Reporting is not registering its collector in this code yet. It looks like the monitoring plugin needs to wait until the registration method is exposed before the plugin can go green. Since exposing the collector object that has the registration method becomes a first-class concern of Monitoring, I want to make Monitoring's green status held up by that, to resolve the race condition.

tsullivan · 2018-05-08T21:57:14Z

Added depends on #18939

elasticmachine · 2018-05-08T23:14:19Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-05-09T21:43:07Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

elasticmachine · 2018-05-11T00:18:00Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

tsullivan · 2018-05-15T16:29:10Z

x-pack/plugins/reporting/index.js

@@ -146,7 +147,8 @@ export const reporting = (kibana) => {
      validateConfig(config, message => server.log(['reporting', 'warning'], message));
      logConfiguration(config, message => server.log(['reporting', 'debug'], message));

-      const xpackMainPlugin = server.plugins.xpack_main;
+      const { xpack_main: xpackMainPlugin, monitoring: monitoringPlugin } = server.plugins;


This needs testing on what happens when Monitoring is disabled but Reporting is enabled. cc @pickypg

elasticmachine · 2018-05-15T23:27:51Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request

… Reporting

tsullivan · 2018-05-22T00:22:09Z

...ns/monitoring/server/kibana_monitoring/collectors/__tests__/get_reporting_usage_collector.js

@@ -1,44 +0,0 @@
-/*


This test wasn't really doing very much, and hasn't been replaced with a similar mocha test in the Reporting plugin.

tsullivan · 2018-05-22T00:23:08Z

x-pack/plugins/monitoring/server/kibana_monitoring/collectors/get_reporting_usage_collector.js

@@ -1,18 +0,0 @@
-/*


This has been moved to Reporting and replaces reporting/server/usage/get_usage.js

elasticmachine · 2018-05-22T01:18:43Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

stacey-gammon · 2018-05-22T14:53:35Z

x-pack/plugins/monitoring/server/kibana_monitoring/lib/get_collector_types_combiner.js

 } from '../../../common/constants';
+import { KIBANA_REPORTING_TYPE } from '../../../../reporting/common/constants';


Is there a step two planned to get rid of this and the abstraction leak below? Or maybe you can get rid of it now?

This module that combines collection types together into a bulk payload unfortunately does have to stay, but just for the short term. It's to support the behavior of Kibana Monitoring that sends kibana stats to the monitoring/_bulk endpoint in ES. Unfortunately, this module needs to have a lot of awareness of the usage collectors because in the current schema, usage is a field of kibana_stats.

We will take the responsibility of sending stats to ES out of Kibana and have Metricbeats used for monitoring all the systems. Once we do that, we can remove get_collector_types_combiner

@stacey-gammon https://github.com/elastic/kibana/blob/6.x/x-pack/plugins/xpack_main/server/routes/api/v1/kibana_stats.js has the same story as this module

…nitoring

tsullivan · 2018-05-23T18:24:35Z

I merged in master to get #18986 into this branch. CI is running right now and I will also run some manual tests before merging.

elasticmachine · 2018-05-23T19:11:28Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request

…ection out of monitoring plugin (elastic#18894) * [Monitoring/Telemetry collection] use `typeCollector.registerType` in Reporting * a few cleanup changes

…ection out of monitoring plugin (#18894) (#19409) * [Monitoring/Telemetry collection] use `typeCollector.registerType` in Reporting * a few cleanup changes

tsullivan added review v7.0.0 v6.4.0 Team:Monitoring Stack Monitoring team (Deprecated) Feature:Reporting Use Reporting:Screenshot, Reporting:CSV, or Reporting:Framework instead Feature:Telemetry labels May 8, 2018

tsullivan requested review from kobelb, pickypg and stacey-gammon May 8, 2018 00:02

tsullivan commented May 8, 2018

View reviewed changes

tsullivan added WIP Work in progress and removed review labels May 8, 2018

tsullivan mentioned this pull request May 8, 2018

[Monitoring] Add a yellow status phase in plugin init #18939

Merged

tsullivan changed the title ~~[Monitoring/Kibana Stats] Expose typeCollector.register method for external plugins to register their stats collector~~ [Monitoring/Reporting Stats] Move hardcoded Reporting stats type collection out of monitoring plugin May 8, 2018

tsullivan force-pushed the reporting/register-usage-collector-with-monitoring branch 2 times, most recently from c8b0454 to 21f95d5 Compare May 9, 2018 20:10

tsullivan force-pushed the reporting/register-usage-collector-with-monitoring branch 2 times, most recently from 6f4906a to bba238c Compare May 10, 2018 23:02

tsullivan mentioned this pull request May 10, 2018

Allow plugins to register their own stats for an _xpack/usage API #12504

Closed

tsullivan commented May 15, 2018

View reviewed changes

tsullivan force-pushed the reporting/register-usage-collector-with-monitoring branch from 86f872d to 11e0f44 Compare May 15, 2018 21:49

tsullivan mentioned this pull request May 15, 2018

Add some collector classes for objects that get registered in a CollectorSet #19098

Merged

tsullivan mentioned this pull request May 18, 2018

[XpackMain] Add _xpack/usage API #19232

Merged

[Monitoring/Telemetry collection] use typeCollector.registerType in…

6e6e6ae

… Reporting

tsullivan force-pushed the reporting/register-usage-collector-with-monitoring branch from 11e0f44 to 6e6e6ae Compare May 22, 2018 00:07

a few cleanup changes

cf44a32

tsullivan commented May 22, 2018

View reviewed changes

stacey-gammon reviewed May 22, 2018

View reviewed changes

tsullivan added review and removed WIP Work in progress labels May 22, 2018

stacey-gammon approved these changes May 22, 2018

View reviewed changes

Merge branch 'master' into reporting/register-usage-collector-with-mo…

9ae9415

…nitoring

tsullivan merged commit ce61ae4 into elastic:master May 24, 2018

tsullivan deleted the reporting/register-usage-collector-with-monitoring branch May 24, 2018 18:14

tsullivan mentioned this pull request May 24, 2018

[6.x] [Monitoring/Reporting Stats] Move hardcoded Reporting stats type collection out of monitoring plugin (#18894) #19409

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Monitoring/Reporting Stats] Move hardcoded Reporting stats type collection out of monitoring plugin #18894

[Monitoring/Reporting Stats] Move hardcoded Reporting stats type collection out of monitoring plugin #18894

tsullivan commented May 8, 2018 •

edited

Loading

tsullivan May 8, 2018 •

edited

Loading

elasticmachine commented May 8, 2018

elasticmachine commented May 8, 2018

elasticmachine commented May 8, 2018

tsullivan commented May 8, 2018

tsullivan commented May 8, 2018

elasticmachine commented May 8, 2018

elasticmachine commented May 9, 2018

elasticmachine commented May 11, 2018

tsullivan May 15, 2018

elasticmachine commented May 15, 2018

tsullivan May 22, 2018

tsullivan May 22, 2018

elasticmachine commented May 22, 2018

stacey-gammon May 22, 2018

tsullivan May 22, 2018

tsullivan May 22, 2018

tsullivan commented May 23, 2018

elasticmachine commented May 23, 2018

		} from '../../../common/constants';
		import { KIBANA_REPORTING_TYPE } from '../../../../reporting/common/constants';

[Monitoring/Reporting Stats] Move hardcoded Reporting stats type collection out of monitoring plugin #18894

[Monitoring/Reporting Stats] Move hardcoded Reporting stats type collection out of monitoring plugin #18894

Conversation

tsullivan commented May 8, 2018 • edited Loading

tsullivan May 8, 2018 • edited Loading

Choose a reason for hiding this comment

elasticmachine commented May 8, 2018

💔 Build Failed

elasticmachine commented May 8, 2018

💔 Build Failed

elasticmachine commented May 8, 2018

💚 Build Succeeded

tsullivan commented May 8, 2018

tsullivan commented May 8, 2018

elasticmachine commented May 8, 2018

💔 Build Failed

elasticmachine commented May 9, 2018

💚 Build Succeeded

elasticmachine commented May 11, 2018

💚 Build Succeeded

tsullivan May 15, 2018

Choose a reason for hiding this comment

elasticmachine commented May 15, 2018

💔 Build Failed

tsullivan May 22, 2018

Choose a reason for hiding this comment

tsullivan May 22, 2018

Choose a reason for hiding this comment

elasticmachine commented May 22, 2018

💚 Build Succeeded

stacey-gammon May 22, 2018

Choose a reason for hiding this comment

tsullivan May 22, 2018

Choose a reason for hiding this comment

tsullivan May 22, 2018

Choose a reason for hiding this comment

tsullivan commented May 23, 2018

elasticmachine commented May 23, 2018

💚 Build Succeeded

tsullivan commented May 8, 2018 •

edited

Loading

tsullivan May 8, 2018 •

edited

Loading