-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[feat][misc] PIP-264: Implement topic lookup metrics using OpenTelemetry #22058
[feat][misc] PIP-264: Implement topic lookup metrics using OpenTelemetry #22058
Conversation
@dragosvictor Please add the following content to your PR description and select a checkbox:
|
pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java
Outdated
Show resolved
Hide resolved
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #22058 +/- ##
============================================
- Coverage 73.71% 73.59% -0.13%
- Complexity 32116 32571 +455
============================================
Files 1874 1875 +1
Lines 139220 139276 +56
Branches 15260 15260
============================================
- Hits 102628 102500 -128
- Misses 28695 28858 +163
- Partials 7897 7918 +21
Flags with carried forward coverage won't be shown. Click here to find out more.
|
pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, I liked your idea you explain in the comment you made on
pulsar.broker.topic.lookup.operation.pending.[usage,limit]
- having its name align since you said this code is also called outside of AdminAPI.
If this is the case, it makes total sense to align it based on your suggestion.
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/test/java/org/apache/pulsar/client/api/BrokerServiceLookupTest.java
Outdated
Show resolved
Hide resolved
pulsar-broker-common/src/main/java/org/apache/pulsar/common/stats/package-info.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
💯
@lhotari You're up next on the review wrap up :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Good work @dragosvictor
PIP-264
Motivation
Part of PIP-264 related work to convert metrics to OpenTelemetry format. This PR handles topic lookup metrics, as currently defined in the documentation.
Note that metric
pulsar_broker_load_manager_bundle_assignment
fits better into the Load Balancing metrics section, so is not included in this work.Modifications
Added metrics as described below:
pulsar_broker_lookup
pulsar.broker.lookup.request.duration
pulsar.lookup.response=failure
replacespulsar_broker_lookup_failures
pulsar.lookup.response=broker
replacespulsar_broker_lookup_answers
pulsar.lookup.response=redirect
replacespulsar_broker_lookup_redirects
pulsar_broker_lookup_pending_requests
pulsar.broker.topic.lookup.operation.pending.usage
BrokerService#getPendingLookupRequest
pulsar.broker.topic.lookup.operation.pending.limit
ServiceConfiguration#getMaxConcurrentLookupRequest
pulsar_broker_topic_load_pending_requests
pulsar.broker.topic.load.operation.pending.usage
BrokerService#getPendingTopicLoadRequests
pulsar.broker.topic.load.operation.pending.limit
ServiceConfiguration#getMaxConcurrentTopicLoadRequest
Moved instantiation of
PulsarBrokerOpenTelemetry
object toPulsarService
constructor, as it is needed by theNamespaceService
andBrokerService
. The OpenTelemetry standard recommends setting up the SDK as early as possible, so this suits us well.Changed
PulsarTestContext
to allow OpenTelemetry metrics validation as part of the tests.Verifying this change
This change added tests and can be verified as follows:
(example:)
BrokerServiceLookupTest#testMultipleBrokerLookup
andBrokerServiceLookupTest#testLookupConnectionNotCloseIfGetUnloadingExOrMetadataEx
to cover the metric validationDoes this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
doc
doc-required
doc-not-needed
doc-complete
Matching PR in forked repository
PR in forked repository: dragosvictor#11