Proposal: New Getting Started Documentation And Reference Application

[svrnm]

This project aims to deliver a new learning experience on opentelemetry.io for OpenTelemetry end users, combined with
a "reference application" that serves as both the foundation of this experience and a demonstration tool for SIGs to showcase feature implementations.

While members of SIG Comms can create the guide, the reference application will require collaboration across SIGs. This need for cross-SIG collaboration is a key reason behind this project proposal.

This proposal is a continuation of the following existing discussions and documents:

• Restructure Language SDK & API documentation information architecture
• Nice to have: a tutorials section on opentelemetry.io where folks can easily access tutorials
• Visualizations we support in the documentation, following up on a conversation with .NET SIG in
  this issue.
• Learn OpenTelemetry: New Getting Started & Best Practices

[avillela]

Thanks for putting this together, @svrnm! I think that this captures things really well and will address a lot of the feedback that we've been getting from the community. 😁

[avillela]

@svrnm we were chatting at the End User SIG meeting today, and were wondering if it also makes sense to have a focus on infrastructure examples, and also mobile app examples?

[svrnm]

@svrnm we were chatting at the End User SIG meeting today, and were wondering if it also makes sense to have a focus on infrastructure examples, and also mobile app examples?

Eventually, yes, but at the beginning I think we need to focus to get the core right (how to add otel to an application and then get traces, metrics, logs, etc out of it). Setting up a collector should be part of that, so having a part on how infra (resources, entities, etc.) plays a roll in that, is definitely something that would come naturally. For the mobile app I see the issue that this is still very experimental, so yes we should have that (adding a "frontend" to the sample app?), but also later.

[jack-berg]

Let's promote this in the next maintainer's SIG

[julianocosta89]

Hello all 👋🏽

Wouldn't make more sense to use the OTel Demo for that?
We could have an uninstrumented fork of it, and throughout the "Learning OTel guide" we instrument each service.

We have almost all OTel supported programming languages covered, missing Erlang/Elixir that will be added back (hopefully soon).

The Demo can be run with Docker and would make life easier.

I also know that @avillela has some work on devcontainer for the Demo.
Each service could have its own devcontainer with all required tools installed, so the newcomer would just have to care about having Docker installed.

Having the SIG members to update the demo would be ideal, as of today we are mostly the Demo maintainers taking care of it. (except @pellared that keeps .NET always up-to-date 🌟 ).

[julianocosta89]

Adding another "reference application" would just be a re-work, no?
Tagging @open-telemetry/demo-maintainers to raise awareness about this issue.

[cijothomas]

Wouldn't make more sense to use the OTel Demo for that?

It maybe too complex for a helloworld/getting started doc. The main goal (I think) here is - to have the exact same app for all languages. OTel Demo app has all languages, but they all do diff. component...

[julianocosta89]

It maybe too complex for a helloworld/getting started doc. The main goal (I think) here is - to have the exact same app for all languages. OTel Demo app has all languages, but they all do diff. component...

Got it, so that would replace the rolldice app.

[svrnm]

It maybe too complex for a helloworld/getting started doc. The main goal (I think) here is - to have the exact same app for all languages. OTel Demo app has all languages, but they all do diff. component...

Yes, that was my underlying assumption. I tried to express this in the proposal as well, but I am happy to discuss this, since we should either turn the "maybe too complex" into a "too complex" or an alternative option that can reduce the potential work.

If we go down the road to use the Demo application for the getting started guide we indeed de-duplicate some work, that's a win, but here are some of my concerns, that maybe can be removed:

• each service within the demo app then needs to work standalone on a local machine, such that a developer can take the uninstrumented version of it and then follow the instructions of the specific language to get it work, i.e. a ruby developer will be presented with the "Email Service" and then work through the same steps as a JavaScript developer with the "Frontend" service.
• calling each service should be trivial. Right now I do curl localhost:8080/dice and all works, but I assume that some of the services of the demo need a payload. I don't think this is insurmountable but requires some clear considerations. I also see that most(?) services are called via gRPC, so we would need people to use grpcurl or similar.
• the base code of the service should be trivial, so finding the right spot to add code should be trivial. In the examples we have so far there is an "app file" (e.g. index.js or index.py or DiceApplication.java) and a "library file" (e.g. dice.js, dice.py, Dice.java), so if we use the demo we would need to trim that down to a very basic code and everything else are "extensions" to that. Doable but hard to manage.
• if a new feature is introduced into the getting started guide, we need to apply it across all applications, e.g. if we say we want to add exemplars, we need to figure out for each and every demo app where to put it, while with a reference app it's a single place

[svrnm]

It maybe too complex for a helloworld/getting started doc. The main goal (I think) here is - to have the exact same app for all languages. OTel Demo app has all languages, but they all do diff. component...

Got it, so that would replace the rolldice app.

yes, exactly.

[julianocosta89]

Yes, that was my underlying assumption. I tried to express this in the proposal as well, but I am happy to discuss this, since we should either turn the "maybe too complex" into a "too complex" or an alternative option that can reduce the potential work.

If we go down the road to use the Demo application for the getting started guide we indeed de-duplicate some work, that's a win, but here are some of my concerns, that maybe can be removed:

• each service within the demo app then needs to work standalone on a local machine, such that a developer can take the uninstrumented version of it and then follow the instructions of the specific language to get it work, i.e. a ruby developer will be presented with the "Email Service" and then work through the same steps as a JavaScript developer with the "Frontend" service.
• calling each service should be trivial. Right now I do curl localhost:8080/dice and all works, but I assume that some of the services of the demo need a payload. I don't think this is insurmountable but requires some clear considerations. I also see that most(?) services are called via gRPC, so we would need people to use grpcurl or similar.
• the base code of the service should be trivial, so finding the right spot to add code should be trivial. In the examples we have so far there is an "app file" (e.g. index.js or index.py or DiceApplication.java) and a "library file" (e.g. dice.js, dice.py, Dice.java), so if we use the demo we would need to trim that down to a very basic code and everything else are "extensions" to that. Doable but hard to manage.
• if a new feature is introduced into the getting started guide, we need to apply it across all applications, e.g. if we say we want to add exemplars, we need to figure out for each and every demo app where to put it, while with a reference app it's a single place

The demo would be an overkill for this.
Also, people looking for a getting started guide are not looking for a getting started in 13 different programming languages, so that would be a pain.

From the Demo, the quoteservice is the simplest service on it, but as it is the latest service called in a chain some concepts would be hard to be explained with it, like context propagation for outgoing calls or linked traces.

Either way, if we want to explain more than the basic concepts, we need at least a 2 services app, ideally 3. And if we add the Collector, that would be 4.
Then, we need to send the data somewhere, that would be Jaeger (5), Prometheus (6), and so on.
Eventually we will have a big environment with a lot of moving pieces and a lot to maintain.

Sorry for not bringing a solution, but I wanted to shed some light into the challenges.

[puckpuck]

At Honeycomb, we deliver OpenTelemetry instrumentation workshops in several languages. These workshops use a relatively simple app but cover all (nearly all?) aspects of OpenTelemetry tracing. Everything runs in a dev container, and so far, we have covered the following languages: Go, Java, JS, and Python. The workshop also has a set of public Google slides describing tracing concepts as we go through each workshop module. The slides are linked in the workshop's readme, with workshop content starting at slide 19.

Could this be used as a starting point?

[svrnm]

The demo would be an overkill for this. Also, people looking for a getting started guide are not looking for a getting started in 13 different programming languages, so that would be a pain.

Agreed, they want to start in the language that is close to them and explore concepts from there.

Either way, if we want to explain more than the basic concepts, we need at least a 2 services app, ideally 3. And if we add the Collector, that would be 4. Then, we need to send the data somewhere, that would be Jaeger (5), Prometheus (6), and so on. Eventually we will have a big environment with a lot of moving pieces and a lot to maintain.

I think that 2 services are just fine for the basic concepts, and depending on the application that we choose, we can have 1 service that can either call itself or a copied instance, think about a fibonacci service, that either does the compute itself or distributes it to peer nodes.

We need a collector for sure, but jaeger,prometheus,etc are optional/something that we will only provide via "here is how you run them in docker" in the prose, the will not be part of the reference application.

Sorry for not bringing a solution, but I wanted to shed some light into the challenges.

No worries, challenges is what we need to discuss!

At Honeycomb, we deliver OpenTelemetry instrumentation workshops in several languages. These workshops use a relatively simple app but cover all (nearly all?) aspects of OpenTelemetry tracing. Everything runs in a dev container, and so far, we have covered the following languages: Go, Java, JS, and Python. The workshop also has a set of public Google slides describing tracing concepts as we go through each workshop module. The slides are linked in the workshop's readme, with workshop content starting at slide 19.

Could this be used as a starting point?

I am more than happy to build open things that we have already, that's why I'd like to have the discussion first which reference application we choose, one point is that we should see what exists already and can be used to start from. We have the existing getting started, the new relic examples and now one more. I'll update the proposal accordingly.

[noMoreCLI]

I think that 2 services are just fine for the basic concepts, and depending on the application that we choose, we can have 1 service that can either call itself or a copied instance, think about a fibonacci service, that either does the compute itself or distributes it to peer nodes.

should these 2 services also be the foundation to demonstrate later maybe more advanced concepts such as custom context propagation or baggage? This might add complexity to the service (and maybe even confuse people just starting). However might help people on the journey to continue their learning and exposure with OTel

[svrnm]

should these 2 services also be the foundation to demonstrate later maybe more advanced concepts such as custom context propagation or baggage? This might add complexity to the service (and maybe even confuse people just starting). However might help people on the journey to continue their learning and exposure with OTel

If possible yes, but not necessarily. We should start with the basic concepts and then evolve towards more advanced topics and see how the reference application fits into that. An option I see is that there are alternate versions of the (uninstrumented) base service, e.g. if you'd like to show the concept of custom context propagation there is a variant where the 2 services communicate using a different protocol that is not easy to instrument with what is available. Same can be done for the async/message passing case.

[svrnm]

I did some prototyping on a few ideas that I had with building that out, PTAL:

https://github.com/svrnm/otel-getting-started-prototype

• It contains a sample reference specification based on the fibonacci app
• It contains a sample JS application
• It contains a sample Go application
• The application code is generated from a single source file and "evolution" of that code (e.g. add instrumentation, add OTLP) is accomplished with C macros (!). There might be better solutions, but I found it interesting to see how well this works:-)

[jpkrohling]

Comment from today's GC triage session: the reference application will likely include features that are not supported by all languages. In which case, it should be clear to the user what's going on: instead of omitting the steps, it should clearly state that the language doesn't support that use-case.

[lof000]

The demo app is great but I agree it is an overkill for that. In fact, I think trying to have a multi-language app may also frighten people.

People who are starting to learn OTel concepts just want a small boilerplate app that they can play with and learn. Why don't we try something like having a path for each language.

I can help with the java app part.

[joaopgrassi]

Just throwing this here, since I didn't see it mentioned. I tried to get something similar to a very minimal, to the point but still "real world ish" with OTel recipes.

For example a API with an endpoint that returns hello world. The API is auto-instrumented and records a custom span. The same is done for multiple languages, using their most common web frameworks:

• C# with ASP.NET https://www.otelrecipes.com/recipes?recipe=csharp.aspnetapi.logs
• Java with Springboot: https://www.otelrecipes.com/recipes?recipe=java.springbootapi.traces
• Go with Gin: https://www.otelrecipes.com/recipes?recipe=go.ginapi.traces

Wouldn't something like this be enough? I like the fact that it is small, easy to clone and run. The spans are exported to a collector which can then be configured to be sent anywhere or just to a local "backend" like Aspire for example. The only thing is I split by signal, to keep it really small. But in OTel we could have an app that have all signals together.

Not to oversell (:D) but I put a lot of effort and making sure the apps are fully end-to-end tested. The test starts the app, execute a request to it, then assert if the spans are there. The same can be done for metrics and logs. Also renovate keeps the dependencies updated and the tests make sure no breaking change is left behind = frustration free for users.

[svrnm]

People who are starting to learn OTel concepts just want a small boilerplate app that they can play with and learn. Why don't we try something like having a path for each language.

There will be one path for each language, but it will be the same sample app and the same flow (add the SDK, add OTLP exporter, etc.)

I can help with the java app part.

That would be great, let me add you to the document.

[svrnm]

Wouldn't something like this be enough? I like the fact that it is small, easy to clone and run. The spans are exported to a collector which can then be configured to be sent anywhere or just to a local "backend" like Aspire for example. The only thing is I split by signal, to keep it really small. But in OTel we could have an app that have all signals together.

Depends on what you mean with "enough", I think a "hello world" is for the getting started guide not enough, since we want to explain a lot of concepts throughout it as well, so we need something which we can use for all signals, but also for showing propagation, exemplars, using the collector and some other things people might be interested to learn about. If you mean an "as simple as possible app", then this is what this proposal is about: a simple app at the level of a fibonacci application, or a "roll the dice application" with a most basic business logic you can come up with to guide folks through the journey of instrumenting an application

Not to oversell (:D) but I put a lot of effort and making sure the apps are fully end-to-end tested. The test starts the app, execute a request to it, then assert if the spans are there. The same can be done for metrics and logs. Also renovate keeps the dependencies updated and the tests make sure no breaking change is left behind = frustration free for users.

You are not overselling, otel recipes is great, not only the simplicity of it, but also the "end-to-end tested" part. This is what we should be aiming for as well. That's also why I think we need a reference application specification, it makes testing much easier ("make sure a webserver is running at port 8080", "make sure there is an /whatever endpoint that returns X", ...)

See my prototype above (https://github.com/svrnm/otel-getting-started-prototype/), I played around with some "end to end testing" as well, and also tried to keep the example as simple as possible, but it is driven by a specification, as suggested by @jack-berg

Revisiting it and thinking about it, I think what otel recipes does is much closer to what I would like to see for the registry eventually, because it does not only look at the language, but also to a specific framework, so instead of having what we have today I would love to see it being presented in the way otel recipes is doing it.

So, in my head it goes like that:

• OTel Getting Started Guide + Reference Application has the purpose to guide folks from zero to one with an end-to-end tutorial that's engaging, understandable and with a minimal business logic to tell a story.
• OTel Recipes (or the Registry) has the purpose to kick start folks from what they have (language + frameworks). After people went through the getting started, this is the next building block what they need for implementing otel on their own
• OTel Demo is the show case app, which gives people a much broader perspective into how all of that works together. It especially comes with real performanace issues and ways on how to solve them

[svrnm]

I say, let's not make it more complicated, merge it here and then have a blog post (maybe early next year): open-telemetry/opentelemetry.io#5783

[trask]

merging, there are enough GC approvals (5 including the PR author) for a new project proposal

[trask]

@svrnm can you fix the cspell failure? thanks

[svrnm]

🎉 -- we will get started after the break :-)