Different monitoring systems have variations in data formats and reporting methods. In order to allow users to utilize different monitoring systems in a unified manner and abstract away the underlying differences, tRPC-Cpp provides a layer of abstraction for Metrics plugins. It defines the organization format for metrics data and the basic capabilities that need to be provided.
This document introduces the interface definition, usage, and development process of Metrics plugins in tRPC-Cpp.
Metrics plugins must follow the interface definition of trpc::Metrics and implement the logic for inter-module reporting (ModuleReport), single-dimensional attribute reporting (SingleAttrReport), and multi-dimensional attributes reporting (MultiAttrReport).
ModuleReport refers to the reporting of metrics data for RPC invocations between modules. It includes information about the caller module, the called module, and the invocation result. It can be further categorized into two types: caller reporting (for tracking client-side invocation information) and callee reporting (for tracking server-side invocation information).
The interface definition for ModuleReport is as follows:
constexpr int kMetricsCallerSource = 0;
constexpr int kMetricsCalleeSource = 1;
/// @brief Metrics data for inter-module calls
struct ModuleMetricsInfo {
/// Reporting source, where kMetricsCallerSource indicates reporting from the caller and kMetricsCalleeSource
/// indicates reporting from the callee
int source;
/// Statistical information
std::map<std::string, std::string> infos;
/// Call's status, where 0 indicates success, 1 indicates exception, and 2 indicates timeout
int state;
/// Call's status code of framework
int frame_ret_code;
/// Call's status code of user interface
int interface_ret_code;
/// Call's duration
uint32_t cost_time;
/// Call's protocol
std::string protocol;
/// Extend information, can be used to provide more information when need
std::any extend_info;
};
class Metrics : public Plugin {
public:
/// @brief Reports the metrics data for inter-module calls
/// @param info metrics data
/// @return the reported result, where 0 indicates success and non-zero indicates failure
virtual int ModuleReport(const ModuleMetricsInfo& info) = 0;
virtual int ModuleReport(ModuleMetricsInfo&& info) { return ModuleReport(info); }
};
ModuleMetricsInfo
encapsulates the unified format of inter-module metrics data, where different Metrics plugins can set different fields as needed. Metrics::ModuleReport
is used to execute the actual reporting logic of the plugin.
Notes:
- The
source
field ofModuleMetricsInfo
is used to differentiate between client-side reporting and server-side reporting, where 0 indicates client-side reporting and 1 indicates server-side reporting. - The
infos
field ofModuleMetricsInfo
can record a series of key-value pairs, allowing the recording of information such as the environment, service, and address of the caller/callee. - The tRPC-Cpp defines the most common fields as much as possible. However, due to significant differences in metrics data among different monitoring systems, there may be some data that cannot be recorded using the fields defined in
ModuleMetricsInfo
. Such information can be recorded using theextend_info
field.
AttributeReport refers to the monitoring and reporting of observable data by the framework and users, in addition to the inter-module metrics data reported during RPC calls.
It includes two types of attribute reporting: single-dimensional attribute reporting(SingleAttrReport) and multi-dimensional attributes reporting(MultiAttrReport). The main differences between them are as follows:
Type | Number of labels | Number of statistical values |
---|---|---|
SingleAttrReport | Only one | Only one |
MultiAttrReport | One or more | One or more |
If we only consider the definition, single-dimensional attribute reporting can be seen as a special case of multi-dimensional attributes reporting. However, when implementing the reporting logic, it is possible to simplify the handling of single-dimensional attribute data. Therefore, we provides interfaces for both types of reporting.
The reporting of single-dimensional and multi-dimensional attributes involves the use of special data types defined in the framework. Here, we provide a unified explanation of these data types:
-
MetricsPolicy
/// @brief Statistical strategy enum MetricsPolicy { /// Overwrite the statistical value SET = 0, /// Calculate the sum value SUM = 1, /// Calculate the mean value AVG = 2, /// Calculate the maximum value MAX = 3, /// Calculate the minimum value MIN = 4, /// Calculate the median value MID = 5, /// Calculate the quartiles QUANTILES = 6, /// Calculate the histogram HISTOGRAM = 7, };
MetricsPolicy refers to the statistical strategy for metrics data, and different Metrics plugins can implement their supported strategies based on the capabilities of the backend system.
-
SummaryQuantiles
using SummaryQuantiles = std::vector<std::vector<double>>;
This type of parameter is used to specify the quantiles to be calculated. For example, in Prometheus, the Summary data type requires the specification of quantiles.
-
HistogramBucket
using HistogramBucket = std::vector<double>;
This type of parameter is used to specify the intervals for data statistics. For example, in Prometheus, the Histogram data type requires the specification of distribution buckets.
The interface definition for SingleAttrReport is as follows:
/// @brief Metrics data with single-dimensional attribute
struct SingleAttrMetricsInfo {
/// Metrics name
std::string name;
/// Metrics dimension
std::string dimension;
/// Metrics policy
MetricsPolicy policy;
/// Statistical value
double value;
/// The bucket used to gather histogram statistics
HistogramBucket bucket;
/// The quantiles used to gather summary statistics
SummaryQuantiles quantiles;
};
class Metrics : public Plugin {
public:
/// @brief Reports the metrics data with single-dimensional attribute
/// @param info metrics data
/// @return the reported result, where 0 indicates success and non-zero indicates failure
virtual int SingleAttrReport(const SingleAttrMetricsInfo& info) = 0;
virtual int SingleAttrReport(SingleAttrMetricsInfo&& info) { return SingleAttrReport(info); }
};
SingleAttrMetricsInfo
encapsulates the unified format for single-dimensional attribute data, and different Metrics plugins can set different fields as needed. Metrics::SingleAttrReport
is used to execute the actual reporting logic of the plugin.
Notes:
- Different monitoring systems have different formats for attribute labels. If a label has only one
key
, you can use either thename
ordimension
ofSingleAttrMetricsInfo
. If the label is akey-value pair
, you can use bothname
anddimension
together to form a key-value pair.
The interface definition for MultiAttrReport is as follows:
/// @brief Metrics data with multi-dimensional attributes
struct MultiAttrMetricsInfo {
/// Metrics name
std::string name;
/// Metrics tags
std::map<std::string, std::string> tags;
/// Metrics dimensions
std::vector<std::string> dimensions;
/// Statistical values
std::vector<std::pair<MetricsPolicy, double>> values;
/// The bucket used to gather histogram statistics
HistogramBucket bucket;
/// The quantiles used to gather summary statistics
SummaryQuantiles quantiles;
};
class Metrics : public Plugin {
public:
/// @brief Reports the metrics data with multi-dimensional attributes
/// @param info metrics data
/// @return the reported result, where 0 indicates success and non-zero indicates failure
virtual int MultiAttrReport(const MultiAttrMetricsInfo& info) = 0;
virtual int MultiAttrReport(MultiAttrMetricsInfo&& info) { return MultiAttrReport(info); }
};
MultiAttrMetricsInfo
encapsulates the unified format for multi-dimensional attributes data, and different Metrics plugins can set different fields as needed. Metrics::MultiAttrReport
is used to execute the actual reporting logic of the plugin.
Notes:
- Different monitoring systems have different formats for attribute labels. If the labels consist of a set of
keys
, you can usedimensions
inMultiAttrMetricsInfo
. If the labels consist of a set ofkey-value pairs
, you can usetags
. - The values in
MultiAttrMetricsInfo
allow specifying different statistical strategies for different value.Metrics::MultiAttrReport
will aggregate and report each value according to its specified statistical strategy.
The tRPC-Cpp encapsulates the Metrics plugins, allowing users to conveniently and uniformly utilize different monitoring systems. The main usage patterns are as follows:
Users simply need to add the filters for the corresponding Metrics plugin in the framework configuration file. The framework will automatically collect and report inter-module metrics data without the need for users to write code to invoke the reporting interface themselves. Therefore, Metrics plugins also need to implement corresponding ServerFilter and ClientFilter. (Please refer to the filter documentation for the principles of filter.)
Let's take Prometheus Metrics plugin as an example:
-
Enable caller reporting:
client: filter: - prometheus
-
Enable callee reporting:
server: filter: - prometheus
The framework provides a set of unified reporting interfaces. Users only need to specify the name of the Metrics plugin to report the metrics data to the corresponding monitoring system:
/// @brief Trpc metrics data for inter-module calls
struct TrpcModuleMetricsInfo {
/// Metrics plugin name. Specifies which plugin to report the data to.
std::string plugin_name;
/// Metrics data
ModuleMetricsInfo module_info;
};
/// @brief Trpc metrics data with single-dimensional attribute
struct TrpcSingleAttrMetricsInfo {
/// Metrics plugin name. Specifies which plugin to report the data to.
std::string plugin_name;
/// Metrics data
SingleAttrMetricsInfo single_attr_info;
};
/// @brief Trpc metrics data with multi-dimensional attributes
struct TrpcMultiAttrMetricsInfo {
/// Metrics plugin name. Specifies which plugin to report the data to.
std::string plugin_name;
/// Metrics data
MultiAttrMetricsInfo multi_attr_info;
};
/// @brief Metrics interfaces for user programing
namespace metrics {
/// @brief Report the trpc metrics data for inter-module calls
/// @param info metrics data
/// @return the reported result, where 0 indicates success and non-zero indicates failure
int ModuleReport(const TrpcModuleMetricsInfo& info);
int ModuleReport(TrpcModuleMetricsInfo&& info);
/// @brief Report the trpc metrics data with single-dimensional attribute
/// @param info metrics data
/// @return the reported result, where 0 indicates success and non-zero indicates failure
int SingleAttrReport(const TrpcSingleAttrMetricsInfo& info);
int SingleAttrReport(TrpcSingleAttrMetricsInfo&& info);
/// @brief Report the metrics data with multi-dimensional attributes
/// @param info metrics data
/// @return the reported result, where 0 indicates success and non-zero indicates failure
int MultiAttrReport(const TrpcMultiAttrMetricsInfo& info);
int MultiAttrReport(TrpcMultiAttrMetricsInfo&& info);
} // namespace metrics
Users only need to set plugin_name
to the desired Metrics plugin name and then call the reporting interface provided by trpc::metrics
to complete the metrics data reporting. When using it, please note that:
- The Metrics plugin corresponding to
plugin_name
must be registered during framework startup, otherwise the reporting will fail. - Under normal circumstances, users will not directly call
ModuleReport
, but instead rely on the framework to automatically report through filters.
According to the specifications of the Metrics plugin, developers need to:
- Implementing a Metrics plugin: used to report data to the monitoring system.
- Implementing two filters: collect metrics data for both the caller and callee during RPC calls, and invoke the Metrics plugin for reporting.
- Registering the Metrics plugin and its corresponding filters.
The Metrics plugin needs to inherit trpc::Metrics. And the interfaces that must be overridden are as follows:
Interface | Description | Note |
---|---|---|
Name | Return the plugin name | It must be unique and cannot be the same as other plugin names. |
ModuleReport | Reporting the inter-module metrics data | Refer to the section on ModuleReport for details, and ensure that the interface supports safe multi-threaded calls. |
SingleAttrReport | Reporting the single-dimensional attribute metrics data | Refer to the section on SingleAttrReport for details, and ensure that the interface supports safe multi-threaded calls. |
MultiAttrReport | Reporting the multi-dimensional attributes metrics data | Refer to the section on MultiAttrReport for details, and ensure that the interface supports safe multi-threaded calls. |
How to implement filters: please refer to the filters documentation for details. It is important to note that the name returned by the filters' Name
interface must match the name of the plugin.
How to retrieve inter-module metrics data: the information such as caller module, callee module, and invocation results can be obtained from the Context
and framework configuration. For specific examples, please refer to the implementation of the Prometheus monitoring plugin.
How to choose the filter points:
-
The client filter can choose
CLIENT_PRE_RPC_INVOKE
andCLIENT_POST_RPC_INVOKE
as filter points.- They can obtain all the information about the current invocation and calculate the duration from the initiation of the RPC call to its completion.
- It is necessary to invoke the Metrics plugin's
ModuleReport
interface to report the caller data when reaching theCLIENT_POST_RPC_INVOKE
hook. - The
CLIENT_POST_RPC_INVOKE
hook will also be executed during one-way calls, and no special handling is required.
-
The server filter can choose
SERVER_POST_RECV_MSG
andSERVER_PRE_SEND_MSG
as filter points.- They can obtain all the information about the current invocation and calculate the duration from the moment the request packet is received to the moment the response packet is about to be sent by the framework.
- It is necessary to invoke the Metrics plugin's
ModuleReport
interface to report the caller data when reaching theSERVER_PRE_SEND_MSG
hook. - The
SERVER_PRE_SEND_MSG
hook will also be executed during one-way calls, and no special handling is required.
The interface for registering Metrics plugin:
using MetricsPtr = RefPtr<Metrics>;
class TrpcPlugin {
public:
/// @brief Register custom metrics plugin
bool RegisterMetrics(const MetricsPtr& metrics);
};
The interfaces for registering filters:
using MessageServerFilterPtr = std::shared_ptr<MessageServerFilter>;
using MessageClientFilterPtr = std::shared_ptr<MessageClientFilter>;
class TrpcPlugin {
public:
/// @brief Register custom server filter
bool RegisterServerFilter(const MessageServerFilterPtr& filter);
/// @brief Register custom client filter
bool RegisterClientFilter(const MessageClientFilterPtr& filter);
};
Let's illustrate with an example. Assuming that we implement a TestMetrics plugin, along with TestServerFilter and TestClientFilter filters.
-
For server scenarios, users need to register in the
TrpcApp::RegisterPlugins
function during service startup:class HelloworldServer : public ::trpc::TrpcApp { public: ... int RegisterPlugins() override { TrpcPlugin::GetInstance()->RegisterMetrics(MakeRefCounted<TestMetrics>()); TrpcPlugin::GetInstance()->RegisterServerFilter(std::make_shared<TestServerFilter>()); TrpcPlugin::GetInstance()->RegisterClientFilter(std::make_shared<TestClientFilter>()); return 0; } };
-
For pure client scenarios, registration should be done after initializing the framework configuration but before starting other framework modules:
int main(int argc, char* argv[]) { ParseClientConfig(argc, argv); TrpcPlugin::GetInstance()->RegisterMetrics(MakeRefCounted<TestMetrics>()); TrpcPlugin::GetInstance()->RegisterClientFilter(std::make_shared<TestClientFilter>()); return ::trpc::RunInTrpcRuntime([]() { return Run(); }); }
Note that the registered plugins and filters only need to be constructed. The framework will automatically invoke their Init
function during startup for initialization.