From 463c1306a149f988393ba25891473b9f476b6987 Mon Sep 17 00:00:00 2001 From: Alexander Wert Date: Sat, 1 Jul 2023 16:45:48 +0200 Subject: [PATCH 1/4] Restructured messaging semantic conventions Signed-off-by: Alexander Wert --- specification/README.md | 1 + specification/faas/aws-lambda.md | 12 +- specification/faas/faas-spans.md | 3 +- specification/general/trace-general.md | 5 +- specification/messaging/README.md | 17 ++ .../instrumentation => messaging}/kafka.md | 95 +++++++++-- .../messaging-spans.md} | 151 +++--------------- specification/messaging/rabbitmq.md | 22 +++ specification/messaging/rocketmq.md | 51 ++++++ .../instrumentation/README.md | 5 - 10 files changed, 212 insertions(+), 150 deletions(-) create mode 100644 specification/messaging/README.md rename specification/{metrics/semantic_conventions/instrumentation => messaging}/kafka.md (68%) rename specification/{trace/semantic_conventions/messaging.md => messaging/messaging-spans.md} (72%) create mode 100644 specification/messaging/rabbitmq.md create mode 100644 specification/messaging/rocketmq.md delete mode 100644 specification/metrics/semantic_conventions/instrumentation/README.md diff --git a/specification/README.md b/specification/README.md index b5278dd9a1..575ce86f3c 100644 --- a/specification/README.md +++ b/specification/README.md @@ -14,6 +14,7 @@ Semantic Conventions are defined for the following areas: * [HTTP](http/README.md): Semantic Conventions for HTTP client and server operations. * [Database](database/README.md): Semantic Conventions for database operations. * [RPC](rpc/README.md): Semantic Conventions for RPC client and server operations. +* [Messaging](messaging/README.md): Semantic Conventions for Messaging operations and systems. * [System](system/README.md): System Semantic Conventions. * *Other areas can be found in the signal specific Semantic Conventions below* diff --git a/specification/faas/aws-lambda.md b/specification/faas/aws-lambda.md index 08285e3e50..34e6a2b0e6 100644 --- a/specification/faas/aws-lambda.md +++ b/specification/faas/aws-lambda.md @@ -114,9 +114,9 @@ added as a link to the span. This means the span may have as many links as messa See [compatibility](../../supplementary-guidelines/compatibility/aws.md#context-propagation) for more info. - [`faas.trigger`][faas] MUST be set to `pubsub`. -- [`messaging.operation`](/specification/trace/semantic_conventions/messaging.md) MUST be set to `process`. -- [`messaging.system`](/specification/trace/semantic_conventions/messaging.md) MUST be set to `AmazonSQS`. -- [`messaging.destination.kind` or `messaging.source.kind`](/specification/trace/semantic_conventions/messaging.md#messaging-attributes) MUST be set to `queue`. +- [`messaging.operation`](/specification/messaging/messaging-spans.md) MUST be set to `process`. +- [`messaging.system`](/specification/messaging/messaging-spans.md) MUST be set to `AmazonSQS`. +- [`messaging.destination.kind` or `messaging.source.kind`](/specification/messaging/messaging-spans.md#messaging-attributes) MUST be set to `queue`. ### SQS Message @@ -128,10 +128,10 @@ added as a link to the span. See [compatibility](../../supplementary-guidelines/compatibility/aws.md#context-propagation) for more info. - [`faas.trigger`][faas] MUST be set to `pubsub`. -- [`messaging.operation`](/specification/trace/semantic_conventions/messaging.md#messaging-attributes) MUST be set to `process`. -- [`messaging.system`](/specification/trace/semantic_conventions/messaging.md#messaging-attributes) MUST be set to `AmazonSQS`. +- [`messaging.operation`](/specification/messaging/messaging-spans.md#messaging-attributes) MUST be set to `process`. +- [`messaging.system`](/specification/messaging/messaging-spans.md#messaging-attributes) MUST be set to `AmazonSQS`. -Other [Messaging attributes](/specification/trace/semantic_conventions/messaging.md#messaging-attributes) SHOULD be set based on the available information in the SQS message +Other [Messaging attributes](/specification/messaging/messaging-spans.md#messaging-attributes) SHOULD be set based on the available information in the SQS message event. Note that `AWSTraceHeader` is the only supported mechanism for propagating `Context` in instrumentation for SQS diff --git a/specification/faas/faas-spans.md b/specification/faas/faas-spans.md index a75837d02e..f561665e2b 100644 --- a/specification/faas/faas-spans.md +++ b/specification/faas/faas-spans.md @@ -217,7 +217,8 @@ The function responsibility is to provide an answer to an inbound HTTP request. A function is set to be executed when messages are sent to a messaging system. In this case, multiple messages could be batch and forwarded at once to the same function invocation. -Therefore, a different root span of type `faas` MUST be created for each message processed by the function, following the [Messaging systems semantic conventions](/specification/trace/semantic_conventions/messaging.md). +Therefore, a different root span of type `faas` MUST be created for each message processed by the function, +following the [Messaging systems semantic conventions](/specification/messaging/messaging-spans.md). This way, it is possible to correlate each individual message with its invocation sender. ### Timer diff --git a/specification/general/trace-general.md b/specification/general/trace-general.md index 763654f872..371cccd327 100644 --- a/specification/general/trace-general.md +++ b/specification/general/trace-general.md @@ -19,9 +19,12 @@ The following semantic conventions for spans are defined: * [HTTP](/specification/http/http-spans.md): For HTTP client and server spans. * [Database](/specification/database/database-spans.md): For SQL and NoSQL client call spans. * [RPC/RMI](/specification/rpc/rpc-spans.md): For remote procedure call (e.g., gRPC) spans. -* [Messaging](/specification/trace/semantic_conventions/messaging.md): For messaging systems (queues, publish/subscribe, etc.) spans. +* [Messaging](/specification/messaging/messaging-spans.md): For messaging systems (queues, publish/subscribe, etc.) spans. * [FaaS](/specification/faas/faas-spans.md): For [Function as a Service](https://en.wikipedia.org/wiki/Function_as_a_service) (e.g., AWS Lambda) spans. * [Exceptions](/specification/exceptions/exceptions-spans.md): For recording exceptions associated with a span. +* [Messaging](/specification/messaging/messaging-spans.md): For messaging systems (queues, publish/subscribe, etc.) spans. +* [FaaS](/specification/trace/semantic_conventions/faas.md): For [Function as a Service](https://en.wikipedia.org/wiki/Function_as_a_service) (e.g., AWS Lambda) spans. +* [Exceptions](/specification/trace/semantic_conventions/exceptions.md): For recording exceptions associated with a span. * [Compatibility](trace-compatibility.md): For spans generated by compatibility components, e.g. OpenTracing Shim layer. * [Feature Flags](/specification/feature-flags/feature-flags-spans.md): For recording feature flag evaluations associated with a span. diff --git a/specification/messaging/README.md b/specification/messaging/README.md new file mode 100644 index 0000000000..f92de2501d --- /dev/null +++ b/specification/messaging/README.md @@ -0,0 +1,17 @@ +# Semantic conventions for Messaging systems + +**Status**: [Experimental][DocumentStatus] + +This document defines semantic conventions for messaging systems spans, metrics and logs. + +Semantic conventions for messaging systems are defined for the following signals: + +* [Messaging Spans](messaging-spans.md): Semantic Conventions for messaging *spans*. + +Technology specific semantic conventions are defined for the following messaging systems: + +* [Kafka](kafka.md): Semantic Conventions for *Apache Kafka*. +* [RabbitMQ](rabbitmq.md): Semantic Conventions for *RabbitMQ*. +* [RocketMQ](rocketmq.md): Semantic Conventions for *Apache RocketMQ*. + +[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/blob/v1.21.0/specification/document-status.md diff --git a/specification/metrics/semantic_conventions/instrumentation/kafka.md b/specification/messaging/kafka.md similarity index 68% rename from specification/metrics/semantic_conventions/instrumentation/kafka.md rename to specification/messaging/kafka.md index 6d9a91c732..cabb5c41ef 100644 --- a/specification/metrics/semantic_conventions/instrumentation/kafka.md +++ b/specification/messaging/kafka.md @@ -2,21 +2,98 @@ linkTitle: Kafka ---> -# Instrumenting Kafka +# Semantic Conventions for Kafka **Status**: [Experimental][DocumentStatus] -This document defines how to apply semantic conventions when instrumenting Kafka. - -- [Kafka Metrics](#kafka-metrics) -- [Kafka Producer Metrics](#kafka-producer-metrics) -- [Kafka Consumer Metrics](#kafka-consumer-metrics) +- [Span attributes](#span-attributes) +- [Examples](#examples) + * [Apache Kafka with Quarkus or Spring Boot Example](#apache-kafka-with-quarkus-or-spring-boot-example) +- [Metrics](#metrics) + * [General Metrics](#general-metrics) + * [Producer Metrics](#producer-metrics) + * [Consumer Metrics](#consumer-metrics) -## Kafka Metrics +The Semantic Conventions for [Apache Kafka](https://kafka.apache.org/) extend and override the [Messaging Semantic Conventions](README.md) +that describe common messaging operations attributes in addition to the Semantic Conventions +described on this page. + +`messaging.system` MUST be set to `"kafka"`. + +## Span attributes + +For Apache Kafka, the following additional attributes are defined: + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `messaging.kafka.message.key` | string | Message keys in Kafka are used for grouping alike messages to ensure they're processed on the same partition. They differ from `messaging.message.id` in that they're not unique. If the key is `null`, the attribute MUST NOT be set. [1] | `myKey` | Recommended | +| `messaging.kafka.consumer.group` | string | Name of the Kafka Consumer Group that is handling the message. Only applies to consumers, not producers. | `my-group` | Recommended | +| `messaging.kafka.destination.partition` | int | Partition the message is sent to. | `2` | Recommended | +| `messaging.kafka.message.offset` | int | The offset of a record in the corresponding Kafka partition. | `42` | Recommended | +| `messaging.kafka.message.tombstone` | boolean | A boolean that is true if the message is a tombstone. | | Conditionally Required: [2] | + +**[1]:** If the key type is not string, it's string representation has to be supplied for the attribute. If the key has no unambiguous, canonical string form, don't include its value. + +**[2]:** If value is `true`. When missing, the value is assumed to be `false`. + + +For Apache Kafka producers, [`peer.service`](/specification/general/general-attributes.md#general-remote-service-attributes) SHOULD be set to the name of the broker or service the message will be sent to. +The `service.name` of a Consumer's Resource SHOULD match the `peer.service` of the Producer, when the message is directly passed to another service. +If an intermediary broker is present, `service.name` and `peer.service` will not be the same. + +`messaging.client_id` SHOULD be set to the `client-id` of consumers, or to the `client.id` property of producers. + +## Examples + +### Apache Kafka with Quarkus or Spring Boot Example + +Given is a process P, that publishes a message to a topic T1 on Apache Kafka. +One process, CA, receives the message and publishes a new message to a topic T2 that is then received and processed by CB. + +Frameworks such as Quarkus and Spring Boot separate processing of a received message from producing subsequent messages out. +For this reason, receiving (Span Rcv1) is the parent of both processing (Span Proc1) and producing a new message (Span Prod2). +The span representing message receiving (Span Rcv1) should not set `messaging.operation` to `receive`, +as it does not only receive the message but also converts the input message to something suitable for the processing operation to consume and creates the output message from the result of processing. + +``` +Process P: | Span Prod1 | +-- +Process CA: | Span Rcv1 | + | Span Proc1 | + | Span Prod2 | +-- +Process CB: | Span Rcv2 | +``` + +| Field or Attribute | Span Prod1 | Span Rcv1 | Span Proc1 | Span Prod2 | Span Rcv2 +|-|-|-|-|-|-| +| Span name | `"T1 publish"` | `"T1 receive"` | `"T1 process"` | `"T2 publish"` | `"T2 receive`" | +| Parent | | Span Prod1 | Span Rcv1 | Span Rcv1 | Span Prod2 | +| Links | | | | | | +| SpanKind | `PRODUCER` | `CONSUMER` | `CONSUMER` | `PRODUCER` | `CONSUMER` | +| Status | `Ok` | `Ok` | `Ok` | `Ok` | `Ok` | +| `peer.service` | `"myKafka"` | | | `"myKafka"` | | +| `service.name` | | `"myConsumer1"` | `"myConsumer1"` | | `"myConsumer2"` | +| `messaging.system` | `"kafka"` | `"kafka"` | `"kafka"` | `"kafka"` | `"kafka"` | +| `messaging.destination.name` | `"T1"` | | | | | +| `messaging.source.name` | | `"T1"` | `"T1"` | `"T2"` | `"T2"` | +| `messaging.operation` | | | `"process"` | | `"receive"` | +| `messaging.client_id` | | `"5"` | `"5"` | `"5"` | `"8"` | +| `messaging.kafka.message.key` | `"myKey"` | `"myKey"` | `"myKey"` | `"anotherKey"` | `"anotherKey"` | +| `messaging.kafka.consumer.group` | | `"my-group"` | `"my-group"` | | `"another-group"` | +| `messaging.kafka.partition` | `"1"` | `"1"` | `"1"` | `"3"` | `"3"` | +| `messaging.kafka.message.offset` | `"12"` | `"12"` | `"12"` | `"32"` | `"32"` | + +## Metrics + +This section defines how to apply semantic conventions when collecting Kafka metrics. + +### General Metrics **Description:** General Kafka metrics. @@ -46,7 +123,7 @@ This document defines how to apply semantic conventions when instrumenting Kafka | messaging.kafka.partition.replicas.in_sync | UpDownCounter | Int64 | replicas | `{replica}` | Number of synchronized replicas of partition | `topic` | The ID (integer) of a topic | | | | | | | | `partition` | The number (integer) of the partition| -## Kafka Producer Metrics +### Producer Metrics **Description:** Kafka Producer level metrics. @@ -65,7 +142,7 @@ This document defines how to apply semantic conventions when instrumenting Kafka | messaging.kafka.producer.record-sent.rate | Gauge | Double | records sent rate | `{record_sent}/s` | The average number of records sent per second for a specific topic. | `client-id` | `client-id` value | | | | | | | | `topic` | topic name | -## Kafka Consumer Metrics +### Consumer Metrics **Description:** Kafka Consumer level metrics. diff --git a/specification/trace/semantic_conventions/messaging.md b/specification/messaging/messaging-spans.md similarity index 72% rename from specification/trace/semantic_conventions/messaging.md rename to specification/messaging/messaging-spans.md index 09e3e08236..2552b16cb5 100644 --- a/specification/trace/semantic_conventions/messaging.md +++ b/specification/messaging/messaging-spans.md @@ -25,14 +25,11 @@ * [Consumer attributes](#consumer-attributes) * [Per-message attributes](#per-message-attributes) * [Attributes specific to certain messaging systems](#attributes-specific-to-certain-messaging-systems) - + [RabbitMQ](#rabbitmq) - + [Apache Kafka](#apache-kafka) - + [Apache RocketMQ](#apache-rocketmq) - [Examples](#examples) * [Topic with multiple consumers](#topic-with-multiple-consumers) - * [Apache Kafka with Quarkus or Spring Boot Example](#apache-kafka-with-quarkus-or-spring-boot-example) * [Batch receiving](#batch-receiving) * [Batch processing](#batch-processing) +- [Semantic Conventions for specific messaging technologies](#semantic-conventions-for-specific-messaging-technologies) @@ -224,14 +221,14 @@ The following operations related to messages are defined for these semantic conv | `messaging.message.id` | string | A value used by the messaging system as an identifier for the message, represented as a string. | `452a7c7c7c7048c2f887f61572b18fc2` | Recommended: [11] | | `messaging.message.payload_compressed_size_bytes` | int | The compressed size of the message payload in bytes. | `2048` | Recommended: [12] | | `messaging.message.payload_size_bytes` | int | The (uncompressed) size of the message payload in bytes. Also use this attribute if it is unknown whether the compressed or uncompressed payload size is reported. | `2738` | Recommended: [13] | -| [`network.protocol.name`](../../general/general-attributes.md) | string | [OSI Application Layer](https://osi-model.com/application-layer/) or non-OSI equivalent. The value SHOULD be normalized to lowercase. | `amqp`; `mqtt` | Recommended | -| [`network.protocol.version`](../../general/general-attributes.md) | string | Version of the application layer protocol used. See note below. [14] | `3.1.1` | Recommended | -| [`network.transport`](../../general/general-attributes.md) | string | [OSI Transport Layer](https://osi-model.com/transport-layer/) or [Inter-process Communication method](https://en.wikipedia.org/wiki/Inter-process_communication). The value SHOULD be normalized to lowercase. | `tcp`; `udp` | Recommended | -| [`network.type`](../../general/general-attributes.md) | string | [OSI Network Layer](https://osi-model.com/network-layer/) or non-OSI equivalent. The value SHOULD be normalized to lowercase. | `ipv4`; `ipv6` | Recommended | -| [`server.address`](../../general/general-attributes.md) | string | Logical server hostname, matches server FQDN if available, and IP or socket address if FQDN is not known. [15] | `example.com` | Conditionally Required: If available. | -| [`server.socket.address`](../../general/general-attributes.md) | string | Physical server IP address or Unix socket address. If set from the client, should simply use the socket's peer address, and not attempt to find any actual server IP (i.e., if set from client, this may represent some proxy server instead of the logical server). | `10.5.3.2` | Recommended: If different than `server.address`. | -| [`server.socket.domain`](../../general/general-attributes.md) | string | The domain name of an immediate peer. [16] | `proxy.example.com` | Recommended: [17] | -| [`server.socket.port`](../../general/general-attributes.md) | int | Physical server port. | `16456` | Recommended: If different than `server.port`. | +| [`network.protocol.name`](../general/general-attributes.md) | string | [OSI Application Layer](https://osi-model.com/application-layer/) or non-OSI equivalent. The value SHOULD be normalized to lowercase. | `amqp`; `mqtt` | Recommended | +| [`network.protocol.version`](../general/general-attributes.md) | string | Version of the application layer protocol used. See note below. [14] | `3.1.1` | Recommended | +| [`network.transport`](../general/general-attributes.md) | string | [OSI Transport Layer](https://osi-model.com/transport-layer/) or [Inter-process Communication method](https://en.wikipedia.org/wiki/Inter-process_communication). The value SHOULD be normalized to lowercase. | `tcp`; `udp` | Recommended | +| [`network.type`](../general/general-attributes.md) | string | [OSI Network Layer](https://osi-model.com/network-layer/) or non-OSI equivalent. The value SHOULD be normalized to lowercase. | `ipv4`; `ipv6` | Recommended | +| [`server.address`](../general/general-attributes.md) | string | Logical server hostname, matches server FQDN if available, and IP or socket address if FQDN is not known. [15] | `example.com` | Conditionally Required: If available. | +| [`server.socket.address`](../general/general-attributes.md) | string | Physical server IP address or Unix socket address. If set from the client, should simply use the socket's peer address, and not attempt to find any actual server IP (i.e., if set from client, this may represent some proxy server instead of the logical server). | `10.5.3.2` | Recommended: If different than `server.address`. | +| [`server.socket.domain`](../general/general-attributes.md) | string | The domain name of an immediate peer. [16] | `proxy.example.com` | Recommended: [17] | +| [`server.socket.port`](../general/general-attributes.md) | int | Physical server port. | `16456` | Recommended: If different than `server.port`. | **[1]:** If a custom value is used, it MUST be of low cardinality. @@ -327,81 +324,6 @@ All attributes that are specific for a messaging system SHOULD be populated in ` * `messaging.{system}.consumer.*`: Describes message consumer properties * `messaging.{system}.batch.*`: Describes message batch properties -#### RabbitMQ - -In RabbitMQ, the destination is defined by an *exchange* and a *routing key*. -`messaging.destination.name` MUST be set to the name of the exchange. This will be an empty string if the default exchange is used. - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `messaging.rabbitmq.destination.routing_key` | string | RabbitMQ message routing key. | `myKey` | Conditionally Required: If not empty. | - - -#### Apache Kafka - -For Apache Kafka, the following additional attributes are defined: - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `messaging.kafka.message.key` | string | Message keys in Kafka are used for grouping alike messages to ensure they're processed on the same partition. They differ from `messaging.message.id` in that they're not unique. If the key is `null`, the attribute MUST NOT be set. [1] | `myKey` | Recommended | -| `messaging.kafka.consumer.group` | string | Name of the Kafka Consumer Group that is handling the message. Only applies to consumers, not producers. | `my-group` | Recommended | -| `messaging.kafka.destination.partition` | int | Partition the message is sent to. | `2` | Recommended | -| `messaging.kafka.message.offset` | int | The offset of a record in the corresponding Kafka partition. | `42` | Recommended | -| `messaging.kafka.message.tombstone` | boolean | A boolean that is true if the message is a tombstone. | | Conditionally Required: [2] | - -**[1]:** If the key type is not string, it's string representation has to be supplied for the attribute. If the key has no unambiguous, canonical string form, don't include its value. - -**[2]:** If value is `true`. When missing, the value is assumed to be `false`. - - -For Apache Kafka producers, [`peer.service`](/specification/general/general-attributes.md#general-remote-service-attributes) SHOULD be set to the name of the broker or service the message will be sent to. -The `service.name` of a Consumer's Resource SHOULD match the `peer.service` of the Producer, when the message is directly passed to another service. -If an intermediary broker is present, `service.name` and `peer.service` will not be the same. - -`messaging.client_id` SHOULD be set to the `client-id` of consumers, or to the `client.id` property of producers. - -#### Apache RocketMQ - -Specific attributes for Apache RocketMQ are defined below. - - -| Attribute | Type | Description | Examples | Requirement Level | -|---|---|---|---|---| -| `messaging.rocketmq.namespace` | string | Namespace of RocketMQ resources, resources in different namespaces are individual. | `myNamespace` | Required | -| `messaging.rocketmq.client_group` | string | Name of the RocketMQ producer/consumer group that is handling the message. The client type is identified by the SpanKind. | `myConsumerGroup` | Required | -| `messaging.rocketmq.message.delivery_timestamp` | int | The timestamp in milliseconds that the delay message is expected to be delivered to consumer. | `1665987217045` | Conditionally Required: [1] | -| `messaging.rocketmq.message.delay_time_level` | int | The delay time level for delay message, which determines the message delay time. | `3` | Conditionally Required: [2] | -| `messaging.rocketmq.message.group` | string | It is essential for FIFO message. Messages that belong to the same message group are always processed one by one within the same consumer group. | `myMessageGroup` | Conditionally Required: If the message type is FIFO. | -| `messaging.rocketmq.message.type` | string | Type of message. | `normal` | Recommended | -| `messaging.rocketmq.message.tag` | string | The secondary classifier of message besides topic. | `tagA` | Recommended | -| `messaging.rocketmq.message.keys` | string[] | Key(s) of message, another way to mark message besides message id. | `[keyA, keyB]` | Recommended | -| `messaging.rocketmq.consumption_model` | string | Model of message consumption. This only applies to consumer spans. | `clustering` | Recommended | - -**[1]:** If the message type is delay and delay time level is not specified. - -**[2]:** If the message type is delay and delivery timestamp is not specified. - -`messaging.rocketmq.message.type` MUST be one of the following: - -| Value | Description | -|---|---| -| `normal` | Normal message | -| `fifo` | FIFO message | -| `delay` | Delay message | -| `transaction` | Transaction message | - -`messaging.rocketmq.consumption_model` MUST be one of the following: - -| Value | Description | -|---|---| -| `clustering` | Clustering consumption model | -| `broadcasting` | Broadcasting consumption model | - - -`messaging.client_id` SHOULD be set to the client ID that is automatically generated by the Apache RocketMQ SDK. - ## Examples ### Topic with multiple consumers @@ -426,48 +348,11 @@ Process CB: | Span CB1 | | `server.address` | `"ms"` | `"ms"` | `"ms"` | | `server.port` | `1234` | `1234` | `1234` | | `messaging.system` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | -| `messaging.destination.name` | `"T"` | `"T"` | `"T"` | +| `messaging.destination.name` | `"T"` | | | +| `messaging.source.name` | | `"T"` | `"T"` | | `messaging.operation` | | `"process"` | `"process"` | | `messaging.message.id` | `"a1"` | `"a1"`| `"a1"` | -### Apache Kafka with Quarkus or Spring Boot Example - -Given is a process P, that publishes a message to a topic T1 on Apache Kafka. -One process, CA, receives the message and publishes a new message to a topic T2 that is then received and processed by CB. - -Frameworks such as Quarkus and Spring Boot separate processing of a received message from producing subsequent messages out. -For this reason, receiving (Span Rcv1) is the parent of both processing (Span Proc1) and producing a new message (Span Prod2). -The span representing message receiving (Span Rcv1) should not set `messaging.operation` to `receive`, -as it does not only receive the message but also converts the input message to something suitable for the processing operation to consume and creates the output message from the result of processing. - -``` -Process P: | Span Prod1 | --- -Process CA: | Span Rcv1 | - | Span Proc1 | - | Span Prod2 | --- -Process CB: | Span Rcv2 | -``` - -| Field or Attribute | Span Prod1 | Span Rcv1 | Span Proc1 | Span Prod2 | Span Rcv2 -|-|-|-|-|-|-| -| Span name | `"T1 publish"` | `"T1 receive"` | `"T1 process"` | `"T2 publish"` | `"T2 receive`" | -| Parent | | Span Prod1 | Span Rcv1 | Span Rcv1 | Span Prod2 | -| Links | | | | | | -| SpanKind | `PRODUCER` | `CONSUMER` | `CONSUMER` | `PRODUCER` | `CONSUMER` | -| Status | `Ok` | `Ok` | `Ok` | `Ok` | `Ok` | -| `peer.service` | `"myKafka"` | | | `"myKafka"` | | -| `service.name` | | `"myConsumer1"` | `"myConsumer1"` | | `"myConsumer2"` | -| `messaging.system` | `"kafka"` | `"kafka"` | `"kafka"` | `"kafka"` | `"kafka"` | -| `messaging.destination.name` | `"T1"` | `"T1"` | `"T1"` | `"T2"` | `"T2"` | -| `messaging.operation` | | | `"process"` | | `"receive"` | -| `messaging.client_id` | | `"5"` | `"5"` | `"5"` | `"8"` | -| `messaging.kafka.message.key` | `"myKey"` | `"myKey"` | `"myKey"` | `"anotherKey"` | `"anotherKey"` | -| `messaging.kafka.consumer.group` | | `"my-group"` | `"my-group"` | | `"another-group"` | -| `messaging.kafka.partition` | `"1"` | `"1"` | `"1"` | `"3"` | `"3"` | -| `messaging.kafka.message.offset` | `"12"` | `"12"` | `"12"` | `"32"` | `"32"` | - ### Batch receiving Given is a process P, that publishes two messages to a queue Q on messaging system MS, and a process C, which receives both of them in one batch (Span Recv1) and processes each message separately (Spans Proc1 and Proc2). @@ -492,7 +377,8 @@ Process C: | Span Recv1 | | `server.address` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | | `server.port` | `1234` | `1234` | `1234` | `1234` | `1234` | | `messaging.system` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | -| `messaging.destination.name` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | +| `messaging.destination.name` | `"Q"` | `"Q"` | | | | +| `messaging.source.name` | | | `"Q"` | `"Q"` | `"Q"` | | `messaging.operation` | | | `"receive"` | `"process"` | `"process"` | | `messaging.message.id` | `"a1"` | `"a2"` | | `"a1"` | `"a2"` | | `messaging.batch.message_count` | | | 2 | | | @@ -526,9 +412,18 @@ Process C: | Span Recv1 | Span Recv2 | | `server.address` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | | `server.port` | `1234` | `1234` | `1234` | `1234` | `1234` | | `messaging.system` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | -| `messaging.destination.name` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | +| `messaging.destination.name` | `"Q"` | `"Q"` | | | | +| `messaging.source.name` | | | `"Q"` | `"Q"` | `"Q"` | | `messaging.operation` | | | `"receive"` | `"receive"` | `"process"` | | `messaging.message.id` | `"a1"` | `"a2"` | `"a1"` | `"a2"` | | | `messaging.batch.message_count` | | | 1 | 1 | 2 | +## Semantic Conventions for specific messaging technologies + +More specific Semantic Conventions are defined for the following messaging technologies: + +* [Kafka](kafka.md): Semantic Conventions for *Apache Kafka*. +* [RabbitMQ](rabbitmq.md): Semantic Conventions for *RabbitMQ*. +* [RocketMQ](rocketmq.md): Semantic Conventions for *Apache RocketMQ*. + [DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/blob/v1.21.0/specification/document-status.md diff --git a/specification/messaging/rabbitmq.md b/specification/messaging/rabbitmq.md new file mode 100644 index 0000000000..171d64d0f8 --- /dev/null +++ b/specification/messaging/rabbitmq.md @@ -0,0 +1,22 @@ +# Semantic Conventions for RibbitMQ + +**Status**: [Experimental][DocumentStatus] + +The Semantic Conventions for [RibbitMQ](https://www.rabbitmq.com/) extend and override the [Messaging Semantic Conventions](README.md) +that describe common messaging operations attributes in addition to the Semantic Conventions +described on this page. + +`messaging.system` MUST be set to `"rabbitmq"`. + +## RabbitMQ attributes + +In RabbitMQ, the destination is defined by an *exchange* and a *routing key*. +`messaging.destination.name` MUST be set to the name of the exchange. This will be an empty string if the default exchange is used. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `messaging.rabbitmq.destination.routing_key` | string | RabbitMQ message routing key. | `myKey` | Conditionally Required: If not empty. | + + +[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/blob/v1.21.0/specification/document-status.md diff --git a/specification/messaging/rocketmq.md b/specification/messaging/rocketmq.md new file mode 100644 index 0000000000..a758fb0c9b --- /dev/null +++ b/specification/messaging/rocketmq.md @@ -0,0 +1,51 @@ +# Semantic Conventions for RocketMQ + +**Status**: [Experimental][DocumentStatus] + +The Semantic Conventions for [Apache RocketMQ](https://rocketmq.apache.org/) extend and override the [Messaging Semantic Conventions](README.md) +that describe common messaging operations attributes in addition to the Semantic Conventions +described on this page. + +`messaging.system` MUST be set to `"rocketmq"`. + +## Apache RocketMQ attributes + +Specific attributes for Apache RocketMQ are defined below. + + +| Attribute | Type | Description | Examples | Requirement Level | +|---|---|---|---|---| +| `messaging.rocketmq.namespace` | string | Namespace of RocketMQ resources, resources in different namespaces are individual. | `myNamespace` | Required | +| `messaging.rocketmq.client_group` | string | Name of the RocketMQ producer/consumer group that is handling the message. The client type is identified by the SpanKind. | `myConsumerGroup` | Required | +| `messaging.rocketmq.message.delivery_timestamp` | int | The timestamp in milliseconds that the delay message is expected to be delivered to consumer. | `1665987217045` | Conditionally Required: [1] | +| `messaging.rocketmq.message.delay_time_level` | int | The delay time level for delay message, which determines the message delay time. | `3` | Conditionally Required: [2] | +| `messaging.rocketmq.message.group` | string | It is essential for FIFO message. Messages that belong to the same message group are always processed one by one within the same consumer group. | `myMessageGroup` | Conditionally Required: If the message type is FIFO. | +| `messaging.rocketmq.message.type` | string | Type of message. | `normal` | Recommended | +| `messaging.rocketmq.message.tag` | string | The secondary classifier of message besides topic. | `tagA` | Recommended | +| `messaging.rocketmq.message.keys` | string[] | Key(s) of message, another way to mark message besides message id. | `[keyA, keyB]` | Recommended | +| `messaging.rocketmq.consumption_model` | string | Model of message consumption. This only applies to consumer spans. | `clustering` | Recommended | + +**[1]:** If the message type is delay and delay time level is not specified. + +**[2]:** If the message type is delay and delivery timestamp is not specified. + +`messaging.rocketmq.message.type` MUST be one of the following: + +| Value | Description | +|---|---| +| `normal` | Normal message | +| `fifo` | FIFO message | +| `delay` | Delay message | +| `transaction` | Transaction message | + +`messaging.rocketmq.consumption_model` MUST be one of the following: + +| Value | Description | +|---|---| +| `clustering` | Clustering consumption model | +| `broadcasting` | Broadcasting consumption model | + + +`messaging.client_id` SHOULD be set to the client ID that is automatically generated by the Apache RocketMQ SDK. + +[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/blob/v1.21.0/specification/document-status.md diff --git a/specification/metrics/semantic_conventions/instrumentation/README.md b/specification/metrics/semantic_conventions/instrumentation/README.md deleted file mode 100644 index 7c916ab142..0000000000 --- a/specification/metrics/semantic_conventions/instrumentation/README.md +++ /dev/null @@ -1,5 +0,0 @@ -# Instrumentation - -**Status**: [Experimental][DocumentStatus] - -[DocumentStatus]: https://github.com/open-telemetry/opentelemetry-specification/blob/v1.21.0/specification/document-status.md From aec38bdc5f404a13bd7898285a3d38d417551b5a Mon Sep 17 00:00:00 2001 From: Alexander Wert Date: Mon, 3 Jul 2023 17:35:05 +0200 Subject: [PATCH 2/4] applied changes from removing `messaging.source.*` Signed-off-by: Alexander Wert --- specification/messaging/kafka.md | 3 +-- specification/messaging/messaging-spans.md | 9 +++------ 2 files changed, 4 insertions(+), 8 deletions(-) diff --git a/specification/messaging/kafka.md b/specification/messaging/kafka.md index cabb5c41ef..d51677b02c 100644 --- a/specification/messaging/kafka.md +++ b/specification/messaging/kafka.md @@ -80,8 +80,7 @@ Process CB: | Span Rcv2 | | `peer.service` | `"myKafka"` | | | `"myKafka"` | | | `service.name` | | `"myConsumer1"` | `"myConsumer1"` | | `"myConsumer2"` | | `messaging.system` | `"kafka"` | `"kafka"` | `"kafka"` | `"kafka"` | `"kafka"` | -| `messaging.destination.name` | `"T1"` | | | | | -| `messaging.source.name` | | `"T1"` | `"T1"` | `"T2"` | `"T2"` | +| `messaging.destination.name` | `"T1"` | `"T1"` | `"T1"` | `"T2"` | `"T2"` | | `messaging.operation` | | | `"process"` | | `"receive"` | | `messaging.client_id` | | `"5"` | `"5"` | `"5"` | `"8"` | | `messaging.kafka.message.key` | `"myKey"` | `"myKey"` | `"myKey"` | `"anotherKey"` | `"anotherKey"` | diff --git a/specification/messaging/messaging-spans.md b/specification/messaging/messaging-spans.md index 2552b16cb5..dbabf55627 100644 --- a/specification/messaging/messaging-spans.md +++ b/specification/messaging/messaging-spans.md @@ -348,8 +348,7 @@ Process CB: | Span CB1 | | `server.address` | `"ms"` | `"ms"` | `"ms"` | | `server.port` | `1234` | `1234` | `1234` | | `messaging.system` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | -| `messaging.destination.name` | `"T"` | | | -| `messaging.source.name` | | `"T"` | `"T"` | +| `messaging.destination.name` | `"T"` | `"T"` | `"T"` | | `messaging.operation` | | `"process"` | `"process"` | | `messaging.message.id` | `"a1"` | `"a1"`| `"a1"` | @@ -377,8 +376,7 @@ Process C: | Span Recv1 | | `server.address` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | | `server.port` | `1234` | `1234` | `1234` | `1234` | `1234` | | `messaging.system` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | -| `messaging.destination.name` | `"Q"` | `"Q"` | | | | -| `messaging.source.name` | | | `"Q"` | `"Q"` | `"Q"` | +| `messaging.destination.name` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | | `messaging.operation` | | | `"receive"` | `"process"` | `"process"` | | `messaging.message.id` | `"a1"` | `"a2"` | | `"a1"` | `"a2"` | | `messaging.batch.message_count` | | | 2 | | | @@ -412,8 +410,7 @@ Process C: | Span Recv1 | Span Recv2 | | `server.address` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | `"ms"` | | `server.port` | `1234` | `1234` | `1234` | `1234` | `1234` | | `messaging.system` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | `"rabbitmq"` | -| `messaging.destination.name` | `"Q"` | `"Q"` | | | | -| `messaging.source.name` | | | `"Q"` | `"Q"` | `"Q"` | +| `messaging.destination.name` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | `"Q"` | | `messaging.operation` | | | `"receive"` | `"receive"` | `"process"` | | `messaging.message.id` | `"a1"` | `"a2"` | `"a1"` | `"a2"` | | | `messaging.batch.message_count` | | | 1 | 1 | 2 | From c25ab0c5c76309e164970f86a16d1e949c3f6ea8 Mon Sep 17 00:00:00 2001 From: Alexander Wert Date: Mon, 3 Jul 2023 17:36:09 +0200 Subject: [PATCH 3/4] Fix typo in RabbitMQ heading Signed-off-by: Alexander Wert --- specification/messaging/rabbitmq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specification/messaging/rabbitmq.md b/specification/messaging/rabbitmq.md index 171d64d0f8..d3b0b3dec3 100644 --- a/specification/messaging/rabbitmq.md +++ b/specification/messaging/rabbitmq.md @@ -1,4 +1,4 @@ -# Semantic Conventions for RibbitMQ +# Semantic Conventions for RabbitMQ **Status**: [Experimental][DocumentStatus] From 8f8dd4b4412ed36e3ba2e99fd4201226902f222a Mon Sep 17 00:00:00 2001 From: Alexander Wert Date: Mon, 3 Jul 2023 18:23:00 +0200 Subject: [PATCH 4/4] fixed links after rebase Signed-off-by: Alexander Wert --- specification/general/trace-general.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/specification/general/trace-general.md b/specification/general/trace-general.md index 371cccd327..00b79f20dc 100644 --- a/specification/general/trace-general.md +++ b/specification/general/trace-general.md @@ -22,9 +22,6 @@ The following semantic conventions for spans are defined: * [Messaging](/specification/messaging/messaging-spans.md): For messaging systems (queues, publish/subscribe, etc.) spans. * [FaaS](/specification/faas/faas-spans.md): For [Function as a Service](https://en.wikipedia.org/wiki/Function_as_a_service) (e.g., AWS Lambda) spans. * [Exceptions](/specification/exceptions/exceptions-spans.md): For recording exceptions associated with a span. -* [Messaging](/specification/messaging/messaging-spans.md): For messaging systems (queues, publish/subscribe, etc.) spans. -* [FaaS](/specification/trace/semantic_conventions/faas.md): For [Function as a Service](https://en.wikipedia.org/wiki/Function_as_a_service) (e.g., AWS Lambda) spans. -* [Exceptions](/specification/trace/semantic_conventions/exceptions.md): For recording exceptions associated with a span. * [Compatibility](trace-compatibility.md): For spans generated by compatibility components, e.g. OpenTracing Shim layer. * [Feature Flags](/specification/feature-flags/feature-flags-spans.md): For recording feature flag evaluations associated with a span.