From f2b9c32f2644172035b7aac36b5a179984eae118 Mon Sep 17 00:00:00 2001 From: Cijo Thomas Date: Thu, 25 Jul 2024 08:14:10 -0700 Subject: [PATCH 1/4] Add SimpleConcurrentProcessor, clarify existing export concurrency --- CHANGELOG.md | 4 ++++ specification/logs/sdk.md | 41 ++++++++++++++++++++++++++++++--------- 2 files changed, 36 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ea401b9dbc4..865c2716159 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,10 @@ release. ### Logs +- Add the in-development Simple Concurrent log record processor. + ([#4163](https://github.com/open-telemetry/opentelemetry-specification/pull/4163)) +- Clarify that `Export` should not called by built-in processors concurrently. + ### Events ### Resource diff --git a/specification/logs/sdk.md b/specification/logs/sdk.md index be06ddc56b6..ef4b8ed0e09 100644 --- a/specification/logs/sdk.md +++ b/specification/logs/sdk.md @@ -386,9 +386,10 @@ in [OpenTelemetry Collector](../overview.md#collector). #### Simple processor This is an implementation of `LogRecordProcessor` which passes finished logs and -passes the export-friendly `ReadableLogRecord` representation to the -configured [LogRecordExporter](#logrecordexporter), as soon as they are -finished. +passes the export-friendly `ReadableLogRecord` representation to the configured +[LogRecordExporter](#logrecordexporter), as soon as they are finished. The +export() method of the exporter MUST NOT be called concurrently for the same +exporter instance. **Configurable parameters:** @@ -427,6 +428,21 @@ the configured `processor`. * `processor` - processor to be isolated. +#### Simple Concurrent processor + +**Status**: [Development](../document-status.md) + +This is an implementation of `LogRecordProcessor` which passes finished logs and +passes the export-friendly `ReadableLogRecord` representation to the configured +[LogRecordExporter](#logrecordexporter), as soon as they are finished. Unlike +[Simple Processor](#simple-processor), this can invoke export() concurrently, +and hence MUST be documented clearly to be used only with exporters that support +concurrent export() calls. + +**Configurable parameters:** + +* `exporter` - the exporter where the `LogRecord`s are pushed. + ## LogRecordExporter `LogRecordExporter` defines the interface that protocol-specific exporters must @@ -447,14 +463,21 @@ Exports a batch of [ReadableLogRecords](#readablelogrecord). Protocol exporters that will implement this function are typically expected to serialize and transmit the data to the destination. -`Export` will never be called concurrently for the same exporter instance. -Depending on the implementation the result of the export may be returned to the +`Export` MAY be called concurrently for the same exporter instance if paired +with [Simple Concurrent Processor](#simple-concurrent-processor). Each exporter +implementation MUST document whether it supports concurrent `Export` calls, and +hence can be paired with the [Simple Concurrent +Processor](#simple-concurrent-processor). + +Depending on the implementation, the result of the export may be returned to the Processor not in the return value of the call to `Export` but in a language specific way for signaling completion of an asynchronous task. This means that -while an instance of an exporter will never have it `Export` called concurrently -it does not mean that the task of exporting can not be done concurrently. How -this is done is outside the scope of this specification. Each implementation -MUST document the concurrency characteristics the SDK requires of the exporter. +while an instance of an exporter that does not support concurrent calls will +never have `Export` called concurrently, it does not mean that the task of +exporting cannot be done concurrently. How this is done is outside the scope of +this specification. How this is done is outside the scope of this specification. +Each implementation MUST document the concurrency characteristics the SDK +requires of the exporter. `Export` MUST NOT block indefinitely, there MUST be a reasonable upper limit after which the call must time out with an error result (`Failure`). From 8ecc57b0cf7cf8a24060275e160e459b7577ba47 Mon Sep 17 00:00:00 2001 From: Cijo Thomas Date: Thu, 25 Jul 2024 08:30:09 -0700 Subject: [PATCH 2/4] toc --- specification/logs/sdk.md | 1 + 1 file changed, 1 insertion(+) diff --git a/specification/logs/sdk.md b/specification/logs/sdk.md index ef4b8ed0e09..c358d0a6447 100644 --- a/specification/logs/sdk.md +++ b/specification/logs/sdk.md @@ -29,6 +29,7 @@ + [Simple processor](#simple-processor) + [Batching processor](#batching-processor) + [Isolating processor](#isolating-processor) + + [Simple Concurrent processor](#simple-concurrent-processor) - [LogRecordExporter](#logrecordexporter) * [LogRecordExporter operations](#logrecordexporter-operations) + [Export](#export) From de530e8112c1c73aa8a7343c7857c0fdbf210895 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Robert=20Paj=C4=85k?= Date: Fri, 26 Jul 2024 08:47:27 +0200 Subject: [PATCH 3/4] Update specification/logs/sdk.md --- specification/logs/sdk.md | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/specification/logs/sdk.md b/specification/logs/sdk.md index c358d0a6447..f0e7d43b661 100644 --- a/specification/logs/sdk.md +++ b/specification/logs/sdk.md @@ -476,9 +476,8 @@ specific way for signaling completion of an asynchronous task. This means that while an instance of an exporter that does not support concurrent calls will never have `Export` called concurrently, it does not mean that the task of exporting cannot be done concurrently. How this is done is outside the scope of -this specification. How this is done is outside the scope of this specification. -Each implementation MUST document the concurrency characteristics the SDK -requires of the exporter. +this specification. Each implementation MUST document the concurrency +characteristics the SDK requires of the exporter. `Export` MUST NOT block indefinitely, there MUST be a reasonable upper limit after which the call must time out with an error result (`Failure`). From 8d70036fe722b4d351b21739f40ade8d0e40cbfe Mon Sep 17 00:00:00 2001 From: Cijo Thomas Date: Fri, 26 Jul 2024 09:37:23 -0700 Subject: [PATCH 4/4] add review suggestions --- specification/logs/sdk.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/specification/logs/sdk.md b/specification/logs/sdk.md index 32574a1b6bc..86e50da90f4 100644 --- a/specification/logs/sdk.md +++ b/specification/logs/sdk.md @@ -395,7 +395,7 @@ in [OpenTelemetry Collector](../overview.md#collector). This is an implementation of `LogRecordProcessor` which passes finished logs and passes the export-friendly `ReadableLogRecord` representation to the configured [LogRecordExporter](#logrecordexporter), as soon as they are finished. The -export() method of the exporter MUST NOT be called concurrently for the same +exporter methods of the exporter MUST NOT be called concurrently for the same exporter instance. **Configurable parameters:** @@ -442,13 +442,13 @@ the configured `processor`. This is an implementation of `LogRecordProcessor` which passes finished logs and passes the export-friendly `ReadableLogRecord` representation to the configured [LogRecordExporter](#logrecordexporter), as soon as they are finished. Unlike -[Simple Processor](#simple-processor), this can invoke export() concurrently, -and hence MUST be documented clearly to be used only with exporters that support -concurrent export() calls. +[Simple processor](#simple-processor), this can invoke exporter methods +concurrently, and hence MUST be documented clearly to be used only with +concurrent safe exporters. **Configurable parameters:** -* `exporter` - the exporter where the `LogRecord`s are pushed. +* `exporter` - the concurrent safe exporter where the `LogRecord`s are pushed. ## LogRecordExporter @@ -472,9 +472,8 @@ transmit the data to the destination. `Export` MAY be called concurrently for the same exporter instance if paired with [Simple Concurrent Processor](#simple-concurrent-processor). Each exporter -implementation MUST document whether it supports concurrent `Export` calls, and -hence can be paired with the [Simple Concurrent -Processor](#simple-concurrent-processor). +implementation MUST document whether it is concurrent safe, and hence can be +paired with the [Simple Concurrent Processor](#simple-concurrent-processor). Depending on the implementation, the result of the export may be returned to the Processor not in the return value of the call to `Export` but in a language @@ -482,8 +481,7 @@ specific way for signaling completion of an asynchronous task. This means that while an instance of an exporter that does not support concurrent calls will never have `Export` called concurrently, it does not mean that the task of exporting cannot be done concurrently. How this is done is outside the scope of -this specification. Each implementation MUST document the concurrency -characteristics the SDK requires of the exporter. +this specification. `Export` MUST NOT block indefinitely, there MUST be a reasonable upper limit after which the call must time out with an error result (`Failure`).