[Feature Request] Support push metrics to Prometheus Pushgateway

[xianjingfeng]

The push mode will be more convenient and more real-time. We have implemented it. If necessary, I will create a PR

[roryqi]

Why will the push mode be more convenient and more real-time? Could you give some examples?

[xianjingfeng]

1.We need install node exporter in every host, and if not, real-time performance cannot be ensured.
2.We need change the configration of node exporter after we add an instance.

[roryqi]

In our production environment, we use crontab to fetch the servers' metrics every minute and push the metrics to our prometheus server. I don't get the difference if we use exporter.

[xianjingfeng]

This depends on operation and maintenance. If automation tools is well, it may not be inconvenient.

[xianjingfeng]

@jerqi Can you share how do you collect metrics in your k8s cluster?

[roryqi]

@jerqi Can you share how do you collect metrics in your k8s cluster?

We plan to use a k8s sidecar to collect the metrics.

[xianjingfeng]

I think we should support export metrics, like hadoop,hbase etc. Users can implement custom export plugin to export metrics to their monitoring system. WDYT? @jerqi

[roryqi]

I think we should support export metrics, like hadoop,hbase etc. Users can implement custom export plugin to export metrics to their monitoring system. WDYT? @jerqi

I'm not familiar with this aspect. If other mature systems have this , I think it will be ok for me. @zuston Do you have some suggestion?

[zuston]

The similar mechanism is supported by other systems, like Flink: https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/

I think this is a good improvement of observability

[zuston]

Let's reopen this?

[roryqi]

Let's reopen this?

OK.

[roryqi]

Should we have a concept of MetricsReporter refer to Flink first?