Deploy Jaeger

[IceKhan13]

What is the expected behavior?

Deploy Jaeger to the system.
Expose jaeger to grafana dashboard if possible (maybe avoid Jaeger and emit directly prometheus from opentelemetry spans).

Epic #139

[psschwei]

@IceKhan13 @Tansito

For help with clarifying this task, what was the reason for deploying Jaeger? Was it just for generating trace data (functionality which has been sunset in favor of OpenTelemetry)? Or did we want to use Jaeger for its collector / query tooling? Both?

[Tansito]

I don't know if @pacomf has more context maybe. From my conversations with Iskandar I know that we would like to use Jaeger as data source but I don't know If we would have something more in mind, I don't think so. In this case my suggestion would be to start with the simplest configuration and iterate later on if we need advanced things.

[psschwei]

we would like to use Jaeger as data source

this is the part I'm not 100% clear on: Jaeger uses the OpenTelemetry collector as its source for metrics. Assuming I'm understanding everything correctly, I think it'd be simplest to set up a flow like this:

Pod -> OTel collector -> Prometheus

versus trying to add Jaeger into the stack (either between the collector / prometheus or as its own endpoint).

May make sense to wait on this one until everyone's back and we can discuss...

[Tansito]

May make sense to wait on this one until everyone's back and we can discuss...

Sure! Let's do that 😄

[psschwei]

@IceKhan13 now that everyone's back, wanted to follow up on this one to get your thoughts

[IceKhan13]

Pod -> OTel collector -> Prometheus

Agree with you @psschwei. We will only need to switch JaegerExporter to OTLPSpanExporter in client.

Only thing matter is an ability to see traces / runtimes of spans :)
I think grafana/promethus duo should have that covered, right? 🤔

[Tansito]

I think grafana/promethus duo should have that covered, right?

I would say yes? I was taking a look to the configuration that Ray has for Metrics using Grafana + prometheus and it seems more than enough:

https://docs.ray.io/en/latest/ray-observability/ray-metrics.html

[akihikokuroda]

Only thing matter is an ability to see traces / runtimes of spans :)
I think grafana/promethus duo should have that covered, right? 🤔

It doesn't cover the trace span. It covers only metric span.

[akihikokuroda]

I think we need Jaeger for the distributed trace span in qs-observability for local install

[akihikokuroda]

I added instructions to install Jaeger in the local install in the README. I close this.