Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

p2p: add telemetry and DHT/libp2p metrics #5941

Merged
merged 11 commits into from
Apr 2, 2024

Conversation

algorandskiy
Copy link
Contributor

@algorandskiy algorandskiy commented Feb 23, 2024

Summary

Add missing p2p telemetry similarly to wsNetwork:

  • meshThread peers connection stats - generalize and reuse sendPeerConnectionsTelemetryStatus from wsNet
  • ConnectedIn / ConnectedOut
  • Disconnect

In order to propagate telemetryID and instanceName provided by logging module, p2p peers expose a fake /algorand-telemetry/1.0.0/{TID}/{INSTANCE} protocol so that remote side can discover it and treat as some extra meta information. This is a workaround for HTTP transport with its custom HTTP headers.

Add DHT stats collected with OpenCensus (so that a new dependency):

  1. Enabled DHT stats when metrics collection is enabled
  2. Converted to Metric-compatible type when gathered
  3. Added to heartbeat telementry event

Add libp2p prometheus metrics gathering. Supporting only counters and gauges (missing histograms, similarly to DHT).

Test Plan

Added unit tests for telemetryID/instanceName functionality.
Added a unit test for stats similar to what DHT uses.
Added a unit test for metrics similar to what libp2p uses.

@algorandskiy algorandskiy added Enhancement p2p Work related to the p2p project labels Feb 23, 2024
@algorandskiy algorandskiy self-assigned this Feb 23, 2024
Copy link

codecov bot commented Feb 23, 2024

Codecov Report

Attention: Patch coverage is 79.51220% with 42 lines in your changes are missing coverage. Please review.

Project coverage is 56.00%. Comparing base (ac58d8e) to head (d144fc7).

Files Patch % Lines
util/metrics/prometheus.go 75.00% 10 Missing and 2 partials ⚠️
util/metrics/opencensus.go 84.84% 9 Missing and 1 partial ⚠️
network/p2p/p2p.go 75.86% 5 Missing and 2 partials ⚠️
network/wsNetwork.go 57.14% 6 Missing ⚠️
network/p2p/dht/dht.go 0.00% 3 Missing and 1 partial ⚠️
network/p2pNetwork.go 91.66% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@               Coverage Diff               @@
##           feature/p2p    #5941      +/-   ##
===============================================
+ Coverage        55.91%   56.00%   +0.09%     
===============================================
  Files              493      495       +2     
  Lines            68699    68890     +191     
===============================================
+ Hits             38413    38582     +169     
- Misses           27672    27689      +17     
- Partials          2614     2619       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@algorandskiy algorandskiy marked this pull request as ready for review February 26, 2024 23:32
@algorandskiy algorandskiy changed the title generalize sendPeerConnectionsTelemetryStatus and use for p2p code p2p: add telemetry Feb 26, 2024
network/p2p/p2p.go Outdated Show resolved Hide resolved
network/p2p/p2p.go Show resolved Hide resolved
@algorandskiy algorandskiy force-pushed the pavel/p2p-telemetry branch 2 times, most recently from f0a62ee to fc7dcff Compare March 7, 2024 10:15
@algorandskiy algorandskiy changed the title p2p: add telemetry p2p: add telemetry and DHT metrics Mar 7, 2024
@algorandskiy algorandskiy changed the title p2p: add telemetry and DHT metrics p2p: add telemetry and DHT/libp2p metrics Mar 14, 2024
@algorandskiy
Copy link
Contributor Author

Pulled master into feature/p2p and rebased. Good for a final review now.

@algorandskiy algorandskiy merged commit 46b5e99 into algorand:feature/p2p Apr 2, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement p2p Work related to the p2p project
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants