Prometheus Input Package

[ishleenk17]

• Enhancement

What does this PR do?

The Prometheus Input Package is developed in this PR.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

Bring up any service pushing metrics to Prometheus Endpoint. Add the Prometheus Input Integration from Kibana. Add the appropriate input configurations. The metrics should be received in Kibana.

Related issues

• Closes [Input Package] Create new Prometheus Input package #4615

Screenshots

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2023-05-16T11:57:00.267+0000

• Duration: 13 min 41 sec

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[shmsr]

Left some comments.

[shmsr]

Left some minor comments. Otherwise, everything else LGTM.

[elasticmachine]

Package prometheus_input - 0.0.1 containing this change is available at https://epr.elastic.co/search?package=prometheus_input

[ruflin]

Glad to see this landed. Few follow up questions / thoughts:

• Mappings: The mappings here, what dataset are these loaded for?
• Do we use TSDB data streams by default? What are the default dimensions?
• Do we have routing permissions enabled on this input package. @gsantoro As it is an input package, will it have it by default?

Document based routing has landed in 8.8 and I think prometheus metrics is a prime use case for it. Even though we might not have default routing in place for now, we eventually should have, for example based on a service name field or similar. (@felixbarny ).

Prometheus also have some default mappings that are needed. If we do routing, the target data stream should have the same mappings (@weltenwort ). This implies that potentially we need reusable component templates loaded by input packages that then can be used also by the target data streams. @eyalkoren thinks about the naming of these.

Most folks I pinged above are thinking more about logs then metrics at the moment, but I expect things to work exactly the same in the metrics use case and prometheus is a good example for many discussions we had recently.

[ruflin]

@ishleenk17 Too late but I realised the PR is likely not the right place for all of the discussions above. Where should we take it?

[ishleenk17]

@ishleenk17 Too late but I realised the PR is likely not the right place for all of the discussions above. Where should we take it?

Lets track and discuss it as part of the enhancements for Prometheus Package ticket

[ruflin]

SGTM. Copied it over to #6320 (comment) Lets continue the conversation there.