all request not traced in a serverless environment

[okaisti]

Hi, so what am i doing wrong? We are running the apollo router on cloud run, which works great, with 1 second cold starts which is good for our use case.

Now, im running the router with this tracing config:

version: v1.32.0

telemetry:
  experimental_logging:
    format: "${env.LOGGING_FORMAT}"
  tracing:
    experimental_response_trace_id:
      enabled: true
      header_name: "router-trace-id"
    trace_config:
      service_name: 'apollo-router'
      sampler: 1.0
    otlp:
      endpoint: "${env.NEW_RELIC_ENDPOINT}"
      grpc:
        metadata:
          'api-key': ['${env.NEW_RELIC_LICENSE_KEY}']
  metrics:
    otlp:
      endpoint: "${env.NEW_RELIC_ENDPOINT}"
      grpc:
        metadata:
          'api-key': ['${env.NEW_RELIC_LICENSE_KEY}']

and my router logs at startup:

Configuring Otlp tracing: BatchConfig { scheduled_delay=5s, max_queue_size=2048, max_export_batch_size=512, max_export_timeout=30s, max_concurrent_exports=1 }

this setup works locally (docker compose), and adds the traceid to all requests, as well as traces every request and sends them to newrelic.

In cloud run, however, only a small subset of requests are traced, and header is only present in that case. Shouldn't all requests still have the trace id in the response header? And about the batch config, how does that work in a serverless enviroment? i guess a batch cant be sent to newrelic if the serverless execution has ended, or how does it behave?

[bnjjj]

Might be related to #4035