Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Example and docs for .NET span profiles #3224

Merged
merged 8 commits into from
Apr 19, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,11 @@ Key benefits and features:
Get started:

- Configure Pyroscope: Begin sending profiling data to unlock the full potential of Span Profiles
- Client-Side Packages: Easily link traces and profiles using available packages for Go, Ruby, and Java
- Client-Side Packages: Easily link traces and profiles using available packages for Go, Java, Ruby and .NET
- Go: [Span profiles with Traces to profiles (Go)]({{< relref "./go-span-profiles" >}})
- Java: [Span profiles with Traces to profiles (Java)]({{< relref "./java-span-profiles" >}})
- Ruby: [Span profiles with Traces to profiles (Ruby)]({{< relref "./ruby-span-profiles" >}})
- .NET: [Span profiles with Traces to profiles (.NET)]({{< relref "./dotnet-span-profiles" >}})
aleks-p marked this conversation as resolved.
Show resolved Hide resolved
- Grafana Tempo: Visualize and analyze Span Profiles within the Grafana using a Tempo data source.

To learn more, check out our product announcement blog: [Introducing Span Profiles](/blog/2024/02/06/combining-tracing-and-profiling-for-enhanced-observability-introducing-span-profiles/).
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
---
title: Span profiles with Traces to profiles for .NET
menuTitle: Span profiles with Traces to profiles (.NET)
description: Learn about and configure Span profiles with Traces to profiles in Grafana for .NET applications.
weight: 103
---

# Span profiles with Traces to profiles for .NET

knylander-grafana marked this conversation as resolved.
Show resolved Hide resolved
![span-profiles screenshot](https://grafana.com/static/img/docs/tempo/profiles/tempo-profiles-Span-link-profile-data-source.png)

## Before you begin

Your applications must be instrumented for profiling and tracing before you can use span profiles.

* Profiling: Your application must be instrumented with Pyroscope's .NET instrumentation library. If you haven't done this yet, please refer to the [.NET]({{< relref "../language-sdks/dotnet" >}}) guide.
* Tracing: Your application must be instrumented with OpenTelemetry traces. If you haven't done this yet, please refer to the [OpenTelemetry](https://opentelemetry.io/docs/net/getting-started/) guide.
knylander-grafana marked this conversation as resolved.
Show resolved Hide resolved

{{< admonition type="note" >}}
Currently only [OpenTelemetry manual instrumentation](https://opentelemetry.io/docs/languages/net/instrumentation/) is supported for span profiles in .NET.
This limitation is there because Pyroscope's .NET profiler and OpenTelemetry's auto instrumentation are based on separate .NET CLR profilers.
knylander-grafana marked this conversation as resolved.
Show resolved Hide resolved
{{< /admonition >}}

## OpenTelemetry support
aleks-p marked this conversation as resolved.
Show resolved Hide resolved

Pyroscope can integrate with distributed tracing systems supporting the [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/net/getting-started/) standard which allows you to
link traces with the profiling data, and find resource usage for specific lines of code for your trace spans
knylander-grafana marked this conversation as resolved.
Show resolved Hide resolved

{{< admonition type="note" >}}
* Only CPU profiling is supported at the moment.
* Because of how sampling profilers work, spans shorter than the sample interval may not be captured.
{{< /admonition >}}


knylander-grafana marked this conversation as resolved.
Show resolved Hide resolved
## Configure the `Pyroscope.Tracing.OpenTelemetry` package

To start collecting Span Profiles for your .NET application, you need to include [Pyroscope.Tracing.OpenTelemetry](https://github.com/grafana/pyroscope-dotnet/tree/main/Pyroscope/Pyroscope.Tracing/Pyroscope.Tracing.OpenTelemetry) in your code.

This package provides a [`SpanProcessor`](https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/src/OpenTelemetry/BaseProcessor.cs) implementation, which connects the two telemetry signals (traces and profiles) together.

```shell
dotnet add package Pyroscope.Tracing.OpenTelemetry
```

Next, we need to create and register the `PyroscopeSpanProcessor`:
aleks-p marked this conversation as resolved.
Show resolved Hide resolved
```csharp
builder.Services.AddOpenTelemetry()
.WithTracing(b =>
{
b
.AddAspNetCoreInstrumentation()
.AddConsoleExporter()
.AddOtlpExporter()
.AddProcessor(new PyroscopeSpanProcessor.Builder()
.WithRootSpanOnly(true)
.Build());
});
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about removing the WithRootSpanOnly option? It should be true by default, and, honestly, I'm not very sure if we do want to make it configurable. We left this in other integrations to avoid breaking changes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is true by default, I added it here so that it is more visible. I don't have a strong opinion about it, I will remove the configuration option for now and add it back in the future if needed.

```

With the span processor registered, spans created automatically (e.g., for our HTTP handlers) and manually (`ActivitySource.StartActivity()`) will have profiling data associated with them.
aleks-p marked this conversation as resolved.
Show resolved Hide resolved

```

## View the span profiles in Grafana Tempo

To view the span profiles in Grafana Tempo, you need to have a Grafana instance running and a data source configured to link traces and profiles.

Refer to the [data source configuration documentation](/docs/grafana/datasources/tempo/configure-tempo-data-source) to see how to configure the visualization to link traces with profiles.

## Examples

Check out the [examples](https://github.com/grafana/pyroscope/tree/main/examples/tracing/tempo) directory for a complete demo application of span profiles in multiple languages.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Span profiles with Traces to profiles for Java
menuTitle: Span profiles with Traces to profiles (Java)
description: Learn about and configure Span profiles with Traces to profiles in Grafana for the Java language.
weight: 100
weight: 101
---

# Span profiles with Traces to profiles for Java
aleks-p marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -13,7 +13,7 @@ weight: 100

Your applications must be instrumented for profiling and tracing before you can use span profiles.

* Profiling: Your application must be instrumented with Pyroscope's Java client SDK. If you haven't done this yet, please refer to the [Java ()]({{< relref "../language-sdks/java" >}}) guide.
* Profiling: Your application must be instrumented with Pyroscope's Java client SDK. If you haven't done this yet, please refer to the [Java]({{< relref "../language-sdks/java" >}}) guide.
* Tracing: Your application must be instrumented with OpenTelemetry traces. If you haven't done this yet, please refer to the [OpenTelemetry](https://opentelemetry.io/docs/java/getting-started/) guide.
aleks-p marked this conversation as resolved.
Show resolved Hide resolved

## OpenTelemetry support
aleks-p marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: Span profiles with Traces to profiles for Ruby
menuTitle: Span profiles with Traces to profiles (Ruby)
description: Learn about and configure Span profiles with Traces to profiles in Grafana for the Ruby language.
weight: 100
weight: 102
---

# Span profiles with Traces to profiles for Ruby
aleks-p marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -13,7 +13,7 @@ weight: 100

Your applications must be instrumented for profiling and tracing before you can use span profiles.

* Profiling: Your application must be instrumented with Pyroscope's Ruby SDK. If you haven't done this yet, please refer to the [Ruby (push mode)]({{< relref "../language-sdks/ruby" >}}) guide.
* Profiling: Your application must be instrumented with Pyroscope's Ruby SDK. If you haven't done this yet, please refer to the [Ruby]({{< relref "../language-sdks/ruby" >}}) guide.
* Tracing: Your application must be instrumented with OpenTelemetry traces. If you haven't done this yet, please refer to the [OpenTelemetry](https://opentelemetry.io/docs/ruby/getting-started/) guide.
aleks-p marked this conversation as resolved.
Show resolved Hide resolved

## OpenTelemetry support
knylander-grafana marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,4 +59,9 @@ services:
build:
context: .
dockerfile: Dockerfile.load-generator

depends_on:
- pyroscope
- us-east
- eu-north
- ap-south
- ap-south-alpine
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,11 @@
<Nullable>enable</Nullable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Pyroscope" Version="0.8.0" />
<PackageReference Include="OpenTelemetry.Exporter.Console" Version="1.8.0" />
<PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.8.0" />
<PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.8.0" />
<PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.8.1" />
<PackageReference Include="Pyroscope.Tracing.OpenTelemetry" Version="0.1.0" />
<PackageReference Include="System.Diagnostics.DiagnosticSource" Version="8.0.1" />
</ItemGroup>
</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,22 @@
using System.Collections.Generic;
using System.IO;
using System.Threading;
using System.Collections;

using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Builder;
using Microsoft.Extensions.DependencyInjection;

namespace Example;
using OpenTelemetry.Trace;

using System.Collections;
using Microsoft.AspNetCore.Builder;
using Pyroscope.Tracing.OpenTelemetry;

namespace Example;

public static class Program
{
private static readonly List<FileStream> Files = new();
public const string CustomActivitySourceName = "Example.ScooterService";
public static void Main(string[] args)
{
for (int i = 0; i < 1024; i++)
Expand All @@ -20,12 +26,28 @@ public static void Main(string[] args)
}
object globalLock = new();
var strings = new List<string>();

var orderService = new OrderService();
var bikeService = new BikeService(orderService);
var scooterService = new ScooterService(orderService);
var carService = new CarService(orderService);

var app = WebApplication.CreateBuilder(args).Build();
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddOpenTelemetry()
.WithTracing(b =>
{
b
.AddAspNetCoreInstrumentation()
.AddSource(CustomActivitySourceName)
.AddConsoleExporter()
.AddOtlpExporter()
.AddProcessor(
new PyroscopeSpanProcessor.Builder()
.WithRootSpanOnly(true)
.Build());
});
var app = builder.Build();

app.MapGet("/bike", () =>
{
bikeService.Order(1);
Expand Down Expand Up @@ -99,7 +121,7 @@ public static void Main(string[] args)
{
throw new Exception("foobar" + i);
}
catch (Exception e)
catch (Exception)
{
}
}
Expand All @@ -125,4 +147,4 @@ public static void Main(string[] args)

app.Run();
}
}
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
using System.Diagnostics;

namespace Example;

internal class ScooterService
{
private static readonly ActivitySource CustomActivity = new(Program.CustomActivitySourceName);

private readonly OrderService _orderService;

public ScooterService(OrderService orderService)
Expand All @@ -11,6 +15,25 @@ public ScooterService(OrderService orderService)

public void Order(int searchRadius)
{
using var activity = CustomActivity.StartActivity("OrderScooter");
activity?.SetTag("type", "scooter");
for (long i = 0; i < 2000000000; i++)
{
}
OrderInternal(searchRadius);
DoSomeOtherWork();
}

private void OrderInternal(int searchRadius)
{
using var activity = CustomActivity.StartActivity("OrderScooterInternal");
_orderService.FindNearestVehicle(searchRadius, "scooter");
}
}

private void DoSomeOtherWork()
{
for (long i = 0; i < 1000000000; i++)
{
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,6 @@ ENV PYROSCOPE_PROFILING_ENABLED=1
ENV PYROSCOPE_PROFILING_ALLOCATION_ENABLED=true
ENV PYROSCOPE_PROFILING_CONTENTION_ENABLED=true
ENV PYROSCOPE_PROFILING_EXCEPTION_ENABLED=true
ENV RIDESHARE_LISTEN_PORT=5000


CMD ["dotnet", "/dotnet/example.dll"]
CMD sh -c "ASPNETCORE_URLS=http://*:${RIDESHARE_LISTEN_PORT} exec dotnet /dotnet/example.dll"
21 changes: 21 additions & 0 deletions examples/tracing/tempo/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,26 @@ services:
context: ../../language-sdk-instrumentation/java/rideshare
dockerfile: Dockerfile.otel-instrumentation

rideshare-dotnet-eu-west:
platform: linux/amd64
ports:
- 5000
hostname: rideshare-dotnet-eu-west
environment:
<<: *env
OTEL_TRACES_EXPORTER: otlp
OTEL_EXPORTER_OTLP_ENDPOINT: http://tempo:4317
OTEL_SERVICE_NAME: rideshare.dotnet.push.app
OTEL_METRICS_EXPORTER: none
OTEL_TRACES_SAMPLER: always_on
OTEL_PROPAGATORS: tracecontext
OTEL_RESOURCE_ATTRIBUTES: host.name=rideshare-dotnet-eu-west
PYROSCOPE_LABELS: hostname=rideshare-dotnet-eu-west
REGION: eu-west
build:
context: ../../language-sdk-instrumentation/dotnet/rideshare
dockerfile: Dockerfile

load-generator:
environment: *env
build:
Expand All @@ -52,6 +72,7 @@ services:
- http://rideshare-go-ap-south:5000
- http://rideshare-go-eu-north:5000
- http://rideshare-java-us-east:5000
- http://rideshare-dotnet-eu-west:5000

grafana:
image: grafana/grafana-dev:10.3.0-151740
Expand Down
Loading