Is the Opentelemetry python SDK logger compatible with Thread pools

[daidokoro]

Issue

When using the Python SDK with thread pools, only a fraction of the log messages are sent.

Sample

import logging
from multiprocessing import Pool
from opentelemetry._logs import set_logger_provider
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import (
    OTLPLogExporter,
)
from opentelemetry.sdk._logs import LoggerProvider, LoggingHandler
from opentelemetry.sdk._logs.export import BatchLogRecordProcessor
from opentelemetry.sdk._logs.export import SimpleLogRecordProcessor
from opentelemetry.sdk.resources import Resource

import time
import random

# setup logger

logger_provider = LoggerProvider(
    resource=Resource.create(
        {
            "service.name": "dev",
            "service.instance.id": "instance",
        }
    ),
)

ENDPOINT="localhost:4317"

set_logger_provider(logger_provider)

exporter = OTLPLogExporter(
    endpoint=ENDPOINT,
    insecure=True
)

logger_provider.add_log_record_processor(BatchLogRecordProcessor(exporter))
handler = LoggingHandler(level=logging.INFO, logger_provider=logger_provider)

# Attach OTLP handler to root logger
logging.getLogger().addHandler(handler)
logging.getLogger().setLevel(logging.INFO)


logger  = logging.getLogger(__name__)

def some_func(i):
    # logger = logging.getLogger(f"{__name__}{i}")
    logger.info(f"i is {i}")
    # sleep random time
    time.sleep(random.random())
    logger_provider.force_flush()



if __name__ == '__main__':
    with Pool() as pool:
        pool.map(some_func, range(10))
    print('pool done')
    logger_provider.shutdown()

Expected Outcome

10 Log messages are sent to open telemetry.

Actual Outcome

Only a fraction of the messages are received. This number varies.

2023-10-06T13:40:59.143+0200    info    service/telemetry.go:90 Setting up own telemetry...
2023-10-06T13:40:59.144+0200    info    service/telemetry.go:116        Serving Prometheus metrics      {"address": ":8888", "level": "Basic"}
2023-10-06T13:40:59.144+0200    info    exporter/exporter.go:286        Development component. May change in the future.        {"kind": "exporter", "data_type": "logs", "name": "logging"}
2023-10-06T13:40:59.145+0200    info    service/service.go:140  Starting otelcontribcol...      {"Version": "0.71.0-dev", "NumCPU": 10}
2023-10-06T13:40:59.145+0200    info    extensions/extensions.go:41     Starting extensions...
2023-10-06T13:40:59.145+0200    warn    internal/warning.go:51  Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks        {"kind": "receiver", "name": "otlp", "data_type": "logs", "documentation": "https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
2023-10-06T13:40:59.145+0200    info    otlpreceiver@v0.71.0/otlp.go:94 Starting GRPC server    {"kind": "receiver", "name": "otlp", "data_type": "logs", "endpoint": "0.0.0.0:4317"}
2023-10-06T13:40:59.145+0200    info    service/service.go:157  Everything is ready. Begin running and processing data.
2023-10-06T13:41:02.596+0200    info    LogsExporter    {"kind": "exporter", "data_type": "logs", "name": "logging", "#logs": 1}
2023-10-06T13:41:02.599+0200    info    LogsExporter    {"kind": "exporter", "data_type": "logs", "name": "logging", "#logs": 1}
2023-10-06T13:41:04.581+0200    info    LogsExporter    {"kind": "exporter", "data_type": "logs", "name": "logging", "#logs": 1}
2023-10-06T13:41:05.584+0200    info    LogsExporter    {"kind": "exporter", "data_type": "logs", "name": "logging", "#logs": 1}
2023-10-06T13:41:05.603+0200    info    LogsExporter    {"kind": "exporter", "data_type": "logs", "name": "logging", "#logs": 1}

Question

Can the Opentelemetry Python SDK for logging be using with Thread pools?

[kuza55]

It looks like you need to wrap the threadpool to attach the context: https://stackoverflow.com/questions/70772036/is-it-possible-to-get-open-telemetry-tracing-in-python-to-record-spans-that-happ

This feels like something that should be done in the python auto-instrumentation libraries to me.

[srikanthccv]

This feels like something that should be done in the python auto-instrumentation libraries to me.

There was a past attempt to make that happen here open-telemetry/opentelemetry-python-contrib#1582. Anyone interested in seeing that happen are encouraged to continue the effort.

[lzchen]

Should be address by open-telemetry/opentelemetry-python-contrib#2253