Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add prometheus support #314

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

Guilospanck
Copy link
Contributor

@Guilospanck Guilospanck commented Dec 20, 2024

Closes #166 .

This PR adds support for Prometheus, while giving an example of its integration with Grafana as well.

Summary

  • Adds new (optional) cargo project named metrics that holds information about the Prometheus metrics and that can be used in a singleton way;
  • Adds docker-compose.yml file to examplify how to deploy floresta+prometheus+grafana infra;
  • Adds two example metrics: system memory used (GB) and current block height.

Example of added metrics in Grafana:

image

Testing

Be sure to have Docker Compose

Run docker-compose up -d --build, wait for it to start all services and then go to http://localhost:3000/, where you can log into Grafana using the given credentials found in the docker-compose.yml file.

There, you can go to Dashboards on the left, add New, select Prometheus as the data source (should be pre-selected) and then choose one of the metrics, the data visualization type and run the queries to see it in action.

Improvements

I'm not too familiar with the codebase, therefore adding metrics to the best places would be better done by someone who has.

During the tests with the deployment of the infra with docker-compose.yml noticed that there is something off going on with the system, probably related to this, that it was driving the CPU usage over 100% and the memory at around 8GB, maintaining this last one during the initial block downloading at least.

The high CPU and memory usage would go off the roofs around this part.

@Davidson-Souza
Copy link
Collaborator

This is so cool! I'm using a grafana dashboard with this, and it looks sick! Thanks for working on this.

During the tests with the deployment of the infra with docker-compose.yml noticed that there is something off going on with the system, probably related to #294, that it was driving the CPU usage over 100% and the memory at around 8GB, maintaining this last one during the initial block downloading at least (that's why the docker compose has this configuration

I had the same thing happening here, it seems worse on docker than native. I've narrowed that down to sled, because the spikes happen exactly when sled flushes. Right after downloading all headers, we do one flush. On my machine, florestad inside docker was using 12GB of RAM 🤯 . This is why I've started #251. After this PR, we use less than 1GB of RAM, even for IBD (The [app](https://github.com/jvsena42/floresta_app] uses #251, otherwise android would kill it).

For this PR, could we get prometheus as an (optional) feature for forestad?

@Davidson-Souza Davidson-Souza added the enhancement New feature or request label Dec 23, 2024
@Guilospanck
Copy link
Contributor Author

@Davidson-Souza Done ✅ Lemme know if this is what you had in mind.
And the improvement you have for the #251 would be awesome for the project 💯

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for Prometheus
2 participants