Working in non-nodejs environments

[ekosz]

I was wondering if the DD team is thinking of how dd-trace-js may be used in non-nodejs environments. Things like Deno, Cloudflare Workers, etc. It looks like dd-trace-js heavily uses async_hooks which isn't available on other JS platforms.

Is there a way to use the library to just emit traces + spans, but with manual propagation (i.e. Always explicitly needing to set the parent span)?

Also, I see there is an API for sending manually created traces to DD, but it requires having an agent to collect the traces. For environments like Cloudflare Workers is it possible to send the traces directly to DD servers? Or would the suggestion be first send the traces to a central DC, which forwards them to a local agent, which then forwards them to DD?

Thanks!

[rochdev]

I was wondering if the DD team is thinking of how dd-trace-js may be used in non-nodejs environments.

Right now only Node is supported. Some environments like Deno would be potentially easier to support assuming it's compatible with Node modules, but for environments like Cloudflare Workers it would require explicit support. It's something we're looking into but I don't have a lot of details about that (happening outside of my team).

It looks like dd-trace-js heavily uses async_hooks which isn't available on other JS platforms.

Unfortunately this is needed to do automatic instrumentation. In environments without it (or something equivalent) only manual instrumentation would be possible. Since context propagation wouldn't be available in these environments, it would be up to the code to pass the spans around which would be extremely invasive to the code.

Is there a way to use the library to just emit traces + spans, but with manual propagation (i.e. Always explicitly needing to set the parent span)?

We're in the process of splitting dd-trace in multiple components and the low-level ones should work even without async_hooks but we're not there yet.

For environments like Cloudflare Workers is it possible to send the traces directly to DD servers?

Technically it's possible without the agent but you would have to use the endpoint that the agent uses which is not meant to be used externally. I would not recommend going that route as the agent is doing a lot of things that are not documented publicly, some of which might impact functionality or cause over-billing if not implemented properly.

Or would the suggestion be first send the traces to a central DC, which forwards them to a local agent, which then forwards them to DD?

That would definitely be the recommended approach, at least for now. It requires more setup but would be in line with how Datadog is meant to be used. We have plans to reduce (or remove) reliance on the agent in dd-trace which will definitely help later on as well.

[ekosz]

@rochdev Thank you so much for the context! If I end up writing my own library (probably something similar to the Cloudflare Workers Honeycomb lib), I'll post if here if it's useful for the DD team / wider DD community.

[ekosz]

@rochdev Heya! Checking back in on this issue. I know a lot of changes have happened to dd-trace-js since March and now there are experimental exporters like to logs and such. Do you know if one can use dd-trace-js without NodeJS yet and if so, how to report those traces/spans to Datadog? Do you still need to get the payload to an agent running somewhere still? Or can we someone pickup the spans from our logs?

Thanks!

[rochdev]

@ekosz dd-trace itself cannot be used outside of Node, but I know there is an effort internally to support other environments using some more lightweight code specific to those environments. I don't have much more information than this unfortunately, but I would recommend reaching out to support so that they can forward the feature request to the right team and they would have more information about a timeline.

[ekosz]

@rochdev Sounds good! Will do

[jfsiii]

@ekosz perhaps off-topic but you might find poking around https://github.com/RichiCoder1/opentelemetry-sdk-workers helpful.

There's code written by the repo owner but much of it is built on @opentelemetry/* packages and https://github.com/open-telemetry/opentelemetry-proto/tree/e757e740ed7f63b2f969b45adf7abaea4a042fad

You'll need to deploy an otel collector somewhere, it doesn't have to be on Workers, then that lib can generate and send traces to the collector which can send them to Datadog

[yoshixmk]

@rochdev I was unable to use dd-trace using Deno. Any progress?
Deno has a way to use the npm package, but I get an error like below.

error: Uncaught TypeError: executionAsyncResource is not a function
    at AsyncResourceStorage._executionAsyncResource (file:///<user_path>/node_modules/.deno/dd-trace@4.0.0/node_modules/dd-trace/packages/datadog-core/src/storage/async_resource.js:100:12)
...

[rochdev]

@yoshixmk Deno is not currently supported, and most of the library relies on features that are specific to Node, its built-in fork of v8, and libuv. I recommend reaching out to our support team with a feature request.

[Qard]

Other JS runtimes will get this for free in probably a year or so when AsyncContext reaches stage 3 and V8 implements it. Doesn't help us in the immediate term, but we should keep in mind there will be a sudden explosion in support capability when that happens.

[tian-sheng-low]

@Qard
Thanks for the words.

Other JS runtimes will get this for free in probably a year or so when AsyncContext reaches stage 3 and V8 implements it

Could you let me know where is the source? I would like to look into the detail and progress of AsyncContext and V8 implementation.

[Qard]

You can follow the proposal here: https://github.com/tc39/proposal-async-context