GSoC 2021 Summary: Metrics and monitoring

[An-DJ]

Project

pgagroal: Metrics and monitoring

Summary

This project aims to enhance the monitoring part which characterizes the current runtime status of pgagroal's pool, connections, clients, servers and internal resource usage. The dashboard as project outcomes can illustrate the key metrics and show clearly what DBA desires to know with pgagroal.

The main 2 parts are as follows:

• Organize more significant metrics which characterizes the current runtime status
• Create dashboards to better map trends in key metrics

What is done

• 1443ddb Add grafana dashboard to pgagroal
  • PR Add grafana dashboard to pgagroal #152
• 58907bc Split struct prometheus from struct configuration
  • Issue Split the struct Prometheus from struct Configuration #153
  • PR Split struct prometheus from struct configuration #155
• 44b7eb9 Add metrics of client pgagroal_client_active, pgagroal_client_wait and corresponding panels
  • Issue Add more client metrics #158
  • PR Add metrics of client #157
• 0b394f7 Add new metric pgagroal_wait_time and corresponding panel
  • Issue Add more pool metrics #159
  • PR [#159] Add new metric pgagroal_wait_time #160
• f1bbe8a Add new pool metric pgagroal_query_count and corresponding panel
  • Issue Add more pool metrics #159
  • PR [#159] Add new pool metric pgagroal_query_count and Query rate panel #161
• b11de2b Add new pool metric pgagroal_tx_count and corresponding panel
  • Issue Add more pool metrics #159
  • PR [#159] Add new pool metric pgagroal_tx_count and Transaction rate panel #162
• 816d412 Add internal network traffic metrics pgagroal_network_sent, pgagroal_network_received and corresponding panels
  • Issue Add more internal metrics #163
  • PR [#163] Add internal network traffic metrics #164
• be95328 Add internal socket metrics pgagroal_client_sockets, pgagroal_self_sockets and corresponding panels
  • Issue Add more internal metrics #163
  • PR [#163] Add internal socket metrics #165
• 523e7bd Change Query rate and Transaction rate panels' unit % to rps
  • PR Change Query rate and Transaction rate panels' unit #166
• 436741e Add connections per database panel
  • PR Add connections per database panel #167
• d51e3e4 Add new metric pgagroal_pipeline_mode and corresponding panel
  • PR Add new metric pgagroal_pipeline_mode #168
• d463bf3 Add new metric pgagroal_connection_query_count and new panel "Query rate per database"
  • PR Add new metric pgagroal_connection_query_count and new panel "Query rate per database" #169
• 3326583 Add additional description to metrics page and grafana dashboard
  • PR Add additional description to metrics page and grafana dashboard #171

TODO

• Add metrics to describe the query time and transaction time
• Change appropriate total metrics to per-db or per-user statistics metrics

References

• Project home page
• GSoC 2021 ideas: (pgagroal: Metrics and monitoring)
• GSoC 2021 PostgreSQL Project Proposal (pgagroal: Metrics and monitoring)
• Related discussion

[jesperpedersen]

Completed with a pass 🥳