Support attributes with circular references in OTLP exporters

[Arattian]

What version of OpenTelemetry are you using?

    "@opentelemetry/api": "1.0.3",
    "@opentelemetry/core": "1.0.1",
    "@opentelemetry/exporter-trace-otlp-http": "0.27.0",
    "@opentelemetry/exporter-prometheus": "0.27.0",
    "@opentelemetry/resources": "1.0.1",
    "@opentelemetry/sdk-node": "0.27.0",
    "@opentelemetry/semantic-conventions": "1.0.1",
    "@opentelemetry/shim-opentracing": "1.0.1",

What version of Node are you using?

v16.13.0

Please provide the code you used to setup the OpenTelemetry SDK

async function initOtel(parentLogger, config) {
  const log = parentLogger.child({ module: 'opentelemetry' });
  log.verbose = log.trace;
  opentelemetry.api.diag.setLogger(log, getDiagLogLevel(config.logger));

  const serviceName = config.metaInfo.channelPackageJson.name;
  const hostname = os.hostname();

  const textMapPropagator = new CompositePropagator({
    propagators: [new W3CTraceContextPropagator(), new W3CBaggagePropagator()],
  });

  const traceExporter = new OTLPTraceExporter();
  const metricExporter = new PrometheusExporter();

  const sdk = new opentelemetry.NodeSDK({
    traceExporter,
    metricExporter,
    textMapPropagator,
    sampler: new ParentBasedSampler({ root: new AlwaysOnSampler() }),
    metricInterval: 5000,
    resource: new Resource({
      [SemanticResourceAttributes.SERVICE_NAME]: serviceName,
      [SemanticResourceAttributes.SERVICE_INSTANCE_ID]: hostname,
    }),
    // NOTE: do not enable any instrumentations because they are incompatible with
    // OpenTracing shim (see open-telemetry/opentelemetry-js#2016)
    instrumentations: [],
  });

  try {
    await sdk.start();
    log.info('OpenTelemetry SDK started');
  } catch (err) {
    log.error(err, "OpenTelemetry SDK couldn't start");
  }

  const shutdown = async () => {
    try {
      log.info('Shutting down OpenTelemetry SDK');
      await sdk.shutdown();
      log.info('OpenTelemetry SDK shut down successfully');
    } catch (err) {
      log.error(err, 'Error shutting down OpenTelemetry SDK');
    }
  };

  process.on('SIGTERM', shutdown);
  process.on('SIGINT', shutdown);
  process.on('SIGQUIT', shutdown);
  process.on('SIGABRT', shutdown);
};

What did you do?

Sometimes nested libraries or us may generate objects with circular object references. It would be awesome if @opentelemetry will be able to successfully transform and serialize those objects instead of failing.
Pass circular object to function toCollectorAttributes in @opentelemetry/exporter-trace-otlp-http/build/src/transform.js

What did you expect to see?

Spans are written to opentelemetry-collector successfully.

What did you see instead?

{"stack":"RangeError: Maximum call stack size exceeded\n at toCollectorKeyValueList (/Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:47:33)\n at toCollectorAnyValue (/Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:90:32)\n at toCollectorAttributeKeyValue (/Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:58:22)\n at /Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:29:16\n at Array.map (<anonymous>)\n at toCollectorAttributes (/Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:28:36)\n at toCollectorKeyValueList (/Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:49:17)\n at toCollectorAnyValue (/Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:90:32)\n at /Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:39:37\n at Array.map (<anonymous>)\n at toCollectorArrayValue (/Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:39:24)\n at toCollectorAnyValue (/Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:87:31)\n at toCollectorAttributeKeyValue (/Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:58:22)\n at /Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:29:16\n at Array.map (<anonymous>)\n at toCollectorAttributes (/Users/misak/lerua/channels/ap--otelemetry-channel/node_modules/@opentelemetry/exporter-trace-otlp-http/build/src/transform.js:28:36)","message":"Maximum call stack size exceeded","name":"RangeError"}

[dyladan]

It looks like an attribute is being created with a circular reference. If that's the case, I would say this is a feature request and not really a bug. Can you provide an example of a circular reference that you would want to be exported and how you might expect that to look? Circular objects are not easily serializable in e.g. JSON.

[morigs]

In our case: we use opentracing shim. According to OT conventions, we create error log with error.object field. Sometimes error comes from http client, so it includes res object, which reference req object. And req object has a reference to res, so we have a cycle.
It's a border case, yes, but it is quite dangerous in production, because it can consume a lot of CPU and cause trace losses.
IMO, there are few possible solutions:

1. Make some validation for cycles and throw error
2. Use something like safe-json-stringify for serialization
3. Serialize only 1st level fields (returned by Object.keys)

At least it should be mentioned in the docs or in methods comments that you shouldn't pass complex objects as attribute values

[dyladan]

ok cool thanks for the additional info

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions]

This issue was closed because it has been stale for 14 days with no activity.