diff --git a/docs/sources/configure-client/trace-span-profiles/_index.md b/docs/sources/configure-client/trace-span-profiles/_index.md index d4f91acdd5..a83a73d91a 100644 --- a/docs/sources/configure-client/trace-span-profiles/_index.md +++ b/docs/sources/configure-client/trace-span-profiles/_index.md @@ -21,10 +21,11 @@ Key benefits and features: Get started: - Configure Pyroscope: Begin sending profiling data to unlock the full potential of Span Profiles -- Client-Side Packages: Easily link traces and profiles using available packages for Go, Ruby, and Java +- Client-Side Packages: Easily link traces and profiles using available packages for Go, Java, Ruby and .NET - Go: [Span profiles with Traces to profiles (Go)]({{< relref "./go-span-profiles" >}}) - Java: [Span profiles with Traces to profiles (Java)]({{< relref "./java-span-profiles" >}}) - Ruby: [Span profiles with Traces to profiles (Ruby)]({{< relref "./ruby-span-profiles" >}}) + - .NET: [Span profiles with Traces to profiles (.NET)]({{< relref "./dotnet-span-profiles" >}}) - Grafana Tempo: Visualize and analyze Span Profiles within the Grafana using a Tempo data source. To learn more, check out our product announcement blog: [Introducing Span Profiles](/blog/2024/02/06/combining-tracing-and-profiling-for-enhanced-observability-introducing-span-profiles/). diff --git a/docs/sources/configure-client/trace-span-profiles/dotnet-span-profiles.md b/docs/sources/configure-client/trace-span-profiles/dotnet-span-profiles.md new file mode 100644 index 0000000000..e864d683bd --- /dev/null +++ b/docs/sources/configure-client/trace-span-profiles/dotnet-span-profiles.md @@ -0,0 +1,79 @@ +--- +title: Span profiles with Traces to profiles for .NET +menuTitle: Span profiles with Traces to profiles (.NET) +description: Learn about and configure Span profiles with Traces to profiles in Grafana for .NET applications. +weight: 103 +--- + +# Span profiles with Traces to profiles for .NET + +Span Profiles represents a major shift in profiling methodology, enabling deeper analysis of both tracing and profiling data. +Traditional continuous profiling provides an application-wide view over fixed intervals. +In contrast, Span Profiles delivers focused, dynamic analysis on specific execution scopes within applications, such as individual requests or specific trace spans. + +This shift enables a more granular view of performance, enhancing the utility of profiles by linking them directly with traces for a comprehensive understanding of application behavior. As a result, engineering teams can more efficiently identify and address performance bottlenecks. + +To learn more about Span Profiles, refer to [Combining tracing and profiling for enhanced observability: Introducing Span Profiles](/blog/2024/02/06/combining-tracing-and-profiling-for-enhanced-observability-introducing-span-profiles/). + +![span-profiles screenshot](https://grafana.com/static/img/docs/tempo/profiles/tempo-profiles-Span-link-profile-data-source.png) + +Pyroscope integrates with distributed tracing systems supporting the [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/net/getting-started/) standard. +This integration lets you link traces with the profiling data and find resource usage for specific lines of code for your trace spans. + +{{< admonition type="note" >}} +* Only CPU profiling is supported at the moment. +* Because of how sampling profilers work, spans shorter than the sample interval may not be captured. +{{< /admonition >}} + +To use Span Profiles, you need to: + +* [Configure Pyroscope to send profiling data]({{< relref "../../configure-client" >}}) +* Configure a client-side package to link traces and profiles: [.NET](https://github.com/grafana/pyroscope-dotnet/tree/main/Pyroscope/Pyroscope.OpenTelemetry) +* [Configure the Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles](/grafana-cloud/connect-externally-hosted/data-sources/tempo/configure-tempo-data-source/) + +## Before you begin + +Your applications must be instrumented for profiling and tracing before you can use span profiles. + +* Profiling: Your application must be instrumented with Pyroscope's .NET instrumentation library. Refer to the [.NET]({{< relref "../language-sdks/dotnet" >}}) guide for instructions. +* Tracing: Your application must be instrumented with OpenTelemetry traces. Refer to the [OpenTelemetry](https://opentelemetry.io/docs/net/getting-started/) guide for isntructions. + +{{< admonition type="note" >}} +Span profiles in .NET are only supported using [OpenTelemetry manual instrumentation](https://opentelemetry.io/docs/languages/net/instrumentation/) +because Pyroscope's .NET profiler and OpenTelemetry's auto instrumentation are based on separate .NET CLR profilers. +{{< /admonition >}} + +## Configure the `Pyroscope.OpenTelemetry` package + +To start collecting Span Profiles for your .NET application, you need to include [Pyroscope.OpenTelemetry](https://github.com/grafana/pyroscope-dotnet/tree/main/Pyroscope/Pyroscope.OpenTelemetry) in your code. + +This package provides a [`SpanProcessor`](https://github.com/open-telemetry/opentelemetry-dotnet/blob/main/src/OpenTelemetry/BaseProcessor.cs) implementation, which connects the two telemetry signals (traces and profiles) together. + +```shell +dotnet add package Pyroscope.OpenTelemetry +``` + +Next, create and register the `PyroscopeSpanProcessor`: +```csharp +builder.Services.AddOpenTelemetry() + .WithTracing(b => + { + b + .AddAspNetCoreInstrumentation() + .AddConsoleExporter() + .AddOtlpExporter() + .AddProcessor(new Pyroscope.OpenTelemetry.PyroscopeSpanProcessor()); + }); +``` + +With the span processor registered, spans created automatically (for example, HTTP handlers) and manually (`ActivitySource.StartActivity()`) have profiling data associated with them. + +## View the span profiles in Grafana Tempo + +To view the span profiles in Grafana Tempo, you need to have a Grafana instance running and a data source configured to link traces and profiles. + +Refer to the [data source configuration documentation](/docs/grafana/datasources/tempo/configure-tempo-data-source) to see how to configure the visualization to link traces with profiles. + +## Examples + +Check out the [examples](https://github.com/grafana/pyroscope/tree/main/examples/tracing/tempo) directory for a complete demo application of span profiles in multiple languages. diff --git a/docs/sources/configure-client/trace-span-profiles/go-span-profiles.md b/docs/sources/configure-client/trace-span-profiles/go-span-profiles.md index fad1bb908b..2b5175af58 100644 --- a/docs/sources/configure-client/trace-span-profiles/go-span-profiles.md +++ b/docs/sources/configure-client/trace-span-profiles/go-span-profiles.md @@ -20,11 +20,19 @@ To learn more about Span Profiles, refer to [Combining tracing and profiling for ![span-profiles screenshot](https://grafana.com/static/img/docs/tempo/profiles/tempo-profiles-Span-link-profile-data-source.png) +Pyroscope can integrate with distributed tracing systems supporting the [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/go/getting-started/) standard. +This integration lets you link traces with the profiling data and find resource usage for specific lines of code for your trace spans. + +{{% admonition type="note"%}} +* Only CPU profiling is supported. +* Because of how sampling profilers work, spans shorter than the sample interval may not be captured. Go CPU profiler probes stack traces 100 times per second, meaning that spans shorter than 10ms may not be captured. +{{% /admonition %}} + To use Span Profiles, you need to: * [Configure Pyroscope to send profiling data]({{< relref "../../configure-client" >}}) * Configure a client-side package to link traces and profiles: [Go](https://github.com/grafana/otel-profiling-go) -* [Configure Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles](/grafana-cloud/connect-externally-hosted/data-sources/tempo/configure-tempo-data-source/) +* [Configure the Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles](/grafana-cloud/connect-externally-hosted/data-sources/tempo/configure-tempo-data-source/) ## Before you begin @@ -33,16 +41,6 @@ Your applications must be instrumented for profiling and tracing before you can * Profiling: Your application must be instrumented with Pyroscope's Go SDK. If you haven't done this yet, please refer to the [Go (push mode)]({{< relref "../language-sdks/go_push" >}}) guide. * Tracing: Your application must be instrumented with OpenTelemetry traces. If you haven't done this yet, please refer to the [OpenTelemetry](https://opentelemetry.io/docs/go/getting-started/) guide. -### OpenTelemetry support - -Pyroscope can integrate with distributed tracing systems supporting [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/go/getting-started/) standard, which allows you to -link traces with the profiling data, and find resource usage for specific lines of code for your trace spans. - -{{% admonition type="note"%}} - * Only CPU profiling is supported. - * Because of how sampling profilers work, spans shorter than the sample interval may not be captured. Go CPU profiler probes stack traces 100 times per second, meaning that spans shorter than 10ms may not be captured. -{{% /admonition %}} - ## Configure the `otel-profiling-go` package To start collecting Span Profiles for your Go application, you need to include the [`otel-profiling-go`](https://github.com/pyroscope-io/otel-profiling-go) package in your code. diff --git a/docs/sources/configure-client/trace-span-profiles/java-span-profiles.md b/docs/sources/configure-client/trace-span-profiles/java-span-profiles.md index be3f2bc592..79d499a3b0 100644 --- a/docs/sources/configure-client/trace-span-profiles/java-span-profiles.md +++ b/docs/sources/configure-client/trace-span-profiles/java-span-profiles.md @@ -2,30 +2,41 @@ title: Span profiles with Traces to profiles for Java menuTitle: Span profiles with Traces to profiles (Java) description: Learn about and configure Span profiles with Traces to profiles in Grafana for the Java language. -weight: 100 +weight: 101 --- # Span profiles with Traces to profiles for Java -![span-profiles screenshot](https://grafana.com/static/img/docs/tempo/profiles/tempo-profiles-Span-link-profile-data-source.png) - -## Before you begin +Span Profiles represents a major shift in profiling methodology, enabling deeper analysis of both tracing and profiling data. +Traditional continuous profiling provides an application-wide view over fixed intervals. +In contrast, Span Profiles delivers focused, dynamic analysis on specific execution scopes within applications, such as individual requests or specific trace spans. -Your applications must be instrumented for profiling and tracing before you can use span profiles. +This shift enables a more granular view of performance, enhancing the utility of profiles by linking them directly with traces for a comprehensive understanding of application behavior. As a result, engineering teams can more efficiently identify and address performance bottlenecks. -* Profiling: Your application must be instrumented with Pyroscope's Java client SDK. If you haven't done this yet, please refer to the [Java ()]({{< relref "../language-sdks/java" >}}) guide. -* Tracing: Your application must be instrumented with OpenTelemetry traces. If you haven't done this yet, please refer to the [OpenTelemetry](https://opentelemetry.io/docs/java/getting-started/) guide. +To learn more about Span Profiles, refer to [Combining tracing and profiling for enhanced observability: Introducing Span Profiles](/blog/2024/02/06/combining-tracing-and-profiling-for-enhanced-observability-introducing-span-profiles/). -## OpenTelemetry support +![span-profiles screenshot](https://grafana.com/static/img/docs/tempo/profiles/tempo-profiles-Span-link-profile-data-source.png) -Pyroscope can integrate with distributed tracing systems supporting [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/java/getting-started/) standard which allows you to -link traces with the profiling data, and find resource usage for specific lines of code for your trace spans +Pyroscope integrates with distributed tracing systems supporting the [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/java/getting-started/) standard. +This integration lets you link traces with the profiling data and find resource usage for specific lines of code for your trace spans. {{< admonition type="note" >}} - * Only CPU profiling is supported at the moment. - * Because of how sampling profilers work, spans shorter than the sample interval may not be captured. Java CPU profiler probes stack traces 100 times per second, meaning that spans shorter than 10ms may not be captured. +* Only CPU profiling is supported at the moment. +* Because of how sampling profilers work, spans shorter than the sample interval may not be captured. {{< /admonition >}} +To use Span Profiles, you need to: + +* [Configure Pyroscope to send profiling data]({{< relref "../../configure-client" >}}) +* Configure a client-side package to link traces and profiles: [Java](https://github.com/grafana/otel-profiling-java) +* [Configure the Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles](/grafana-cloud/connect-externally-hosted/data-sources/tempo/configure-tempo-data-source/) + +## Before you begin + +Your applications must be instrumented for profiling and tracing before you can use span profiles. + +* Profiling: Your application must be instrumented with Pyroscope's Java client SDK. Refer to the [Java]({{< relref "../language-sdks/java" >}}) guide for instructions. +* Tracing: Your application must be instrumented with OpenTelemetry traces. Refer to the [OpenTelemetry](https://opentelemetry.io/docs/java/getting-started/) guide for instructions. ## Configure the otel-profiling-java package diff --git a/docs/sources/configure-client/trace-span-profiles/ruby-span-profiles.md b/docs/sources/configure-client/trace-span-profiles/ruby-span-profiles.md index 5ff2a3d731..b02b98ae76 100644 --- a/docs/sources/configure-client/trace-span-profiles/ruby-span-profiles.md +++ b/docs/sources/configure-client/trace-span-profiles/ruby-span-profiles.md @@ -2,30 +2,41 @@ title: Span profiles with Traces to profiles for Ruby menuTitle: Span profiles with Traces to profiles (Ruby) description: Learn about and configure Span profiles with Traces to profiles in Grafana for the Ruby language. -weight: 100 +weight: 102 --- # Span profiles with Traces to profiles for Ruby -![span-profiles screenshot](https://grafana.com/static/img/docs/tempo/profiles/tempo-profiles-Span-link-profile-data-source.png) - -## Before you begin +Span Profiles represents a major shift in profiling methodology, enabling deeper analysis of both tracing and profiling data. +Traditional continuous profiling provides an application-wide view over fixed intervals. +In contrast, Span Profiles delivers focused, dynamic analysis on specific execution scopes within applications, such as individual requests or specific trace spans. -Your applications must be instrumented for profiling and tracing before you can use span profiles. +This shift enables a more granular view of performance, enhancing the utility of profiles by linking them directly with traces for a comprehensive understanding of application behavior. As a result, engineering teams can more efficiently identify and address performance bottlenecks. -* Profiling: Your application must be instrumented with Pyroscope's Ruby SDK. If you haven't done this yet, please refer to the [Ruby (push mode)]({{< relref "../language-sdks/ruby" >}}) guide. -* Tracing: Your application must be instrumented with OpenTelemetry traces. If you haven't done this yet, please refer to the [OpenTelemetry](https://opentelemetry.io/docs/ruby/getting-started/) guide. +To learn more about Span Profiles, refer to [Combining tracing and profiling for enhanced observability: Introducing Span Profiles](/blog/2024/02/06/combining-tracing-and-profiling-for-enhanced-observability-introducing-span-profiles/). -## OpenTelemetry support +![span-profiles screenshot](https://grafana.com/static/img/docs/tempo/profiles/tempo-profiles-Span-link-profile-data-source.png) -Pyroscope can integrate with distributed tracing systems supporting [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/ruby/getting-started/) standard which allows you to -link traces with the profiling data, and find resource usage for specific lines of code for your trace spans +Pyroscope integrates with distributed tracing systems supporting the [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/ruby/getting-started/) standard. +This integration lets you link traces with the profiling data and find resource usage for specific lines of code for your trace spans. {{< admonition type="note" >}} - * Only CPU profiling is supported at the moment. - * Because of how sampling profilers work, spans shorter than the sample interval may not be captured. Ruby CPU profiler probes stack traces 100 times per second, meaning that spans shorter than 10ms may not be captured. +* Only CPU profiling is supported at the moment. +* Because of how sampling profilers work, spans shorter than the sample interval may not be captured. {{< /admonition >}} +To use Span Profiles, you need to: + +* [Configure Pyroscope to send profiling data]({{< relref "../../configure-client" >}}) +* Configure a client-side package to link traces and profiles: [Ruby](https://github.com/grafana/otel-profiling-ruby) +* [Configure the Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles](/grafana-cloud/connect-externally-hosted/data-sources/tempo/configure-tempo-data-source/) + +## Before you begin + +Your applications must be instrumented for profiling and tracing before you can use span profiles. + +* Profiling: Your application must be instrumented with Pyroscope's Ruby SDK. Refer to the [Ruby]({{< relref "../language-sdks/ruby" >}}) guide for instructions. +* Tracing: Your application must be instrumented with OpenTelemetry traces. Refer to the [OpenTelemetry](https://opentelemetry.io/docs/ruby/getting-started/) guide for instructions. ## Configure the otel-profiling-ruby package diff --git a/examples/language-sdk-instrumentation/dotnet/rideshare/docker-compose.yml b/examples/language-sdk-instrumentation/dotnet/rideshare/docker-compose.yml index d628ed4b26..7c64baad09 100644 --- a/examples/language-sdk-instrumentation/dotnet/rideshare/docker-compose.yml +++ b/examples/language-sdk-instrumentation/dotnet/rideshare/docker-compose.yml @@ -59,4 +59,9 @@ services: build: context: . dockerfile: Dockerfile.load-generator - + depends_on: + - pyroscope + - us-east + - eu-north + - ap-south + - ap-south-alpine diff --git a/examples/language-sdk-instrumentation/dotnet/rideshare/example/Example.csproj b/examples/language-sdk-instrumentation/dotnet/rideshare/example/Example.csproj index 1e88214f06..95e3a743d9 100644 --- a/examples/language-sdk-instrumentation/dotnet/rideshare/example/Example.csproj +++ b/examples/language-sdk-instrumentation/dotnet/rideshare/example/Example.csproj @@ -7,6 +7,11 @@ enable - + + + + + + diff --git a/examples/language-sdk-instrumentation/dotnet/rideshare/example/Program.cs b/examples/language-sdk-instrumentation/dotnet/rideshare/example/Program.cs index a4808f8259..83beb1a632 100644 --- a/examples/language-sdk-instrumentation/dotnet/rideshare/example/Program.cs +++ b/examples/language-sdk-instrumentation/dotnet/rideshare/example/Program.cs @@ -2,16 +2,22 @@ using System.Collections.Generic; using System.IO; using System.Threading; +using System.Collections; + using Microsoft.AspNetCore.Http; +using Microsoft.AspNetCore.Builder; +using Microsoft.Extensions.DependencyInjection; -namespace Example; +using OpenTelemetry.Trace; -using System.Collections; -using Microsoft.AspNetCore.Builder; +using Pyroscope.OpenTelemetry; + +namespace Example; public static class Program { private static readonly List Files = new(); + public const string CustomActivitySourceName = "Example.ScooterService"; public static void Main(string[] args) { for (int i = 0; i < 1024; i++) @@ -20,12 +26,25 @@ public static void Main(string[] args) } object globalLock = new(); var strings = new List(); + var orderService = new OrderService(); var bikeService = new BikeService(orderService); var scooterService = new ScooterService(orderService); var carService = new CarService(orderService); - var app = WebApplication.CreateBuilder(args).Build(); + var builder = WebApplication.CreateBuilder(args); + builder.Services.AddOpenTelemetry() + .WithTracing(b => + { + b + .AddAspNetCoreInstrumentation() + .AddSource(CustomActivitySourceName) + .AddConsoleExporter() + .AddOtlpExporter() + .AddProcessor(new PyroscopeSpanProcessor()); + }); + var app = builder.Build(); + app.MapGet("/bike", () => { bikeService.Order(1); @@ -99,7 +118,7 @@ public static void Main(string[] args) { throw new Exception("foobar" + i); } - catch (Exception e) + catch (Exception) { } } @@ -125,4 +144,4 @@ public static void Main(string[] args) app.Run(); } -} \ No newline at end of file +} diff --git a/examples/language-sdk-instrumentation/dotnet/rideshare/example/ScooterService.cs b/examples/language-sdk-instrumentation/dotnet/rideshare/example/ScooterService.cs index c41ca593d1..a67fdbb993 100644 --- a/examples/language-sdk-instrumentation/dotnet/rideshare/example/ScooterService.cs +++ b/examples/language-sdk-instrumentation/dotnet/rideshare/example/ScooterService.cs @@ -1,7 +1,11 @@ +using System.Diagnostics; + namespace Example; internal class ScooterService { + private static readonly ActivitySource CustomActivity = new(Program.CustomActivitySourceName); + private readonly OrderService _orderService; public ScooterService(OrderService orderService) @@ -11,6 +15,25 @@ public ScooterService(OrderService orderService) public void Order(int searchRadius) { + using var activity = CustomActivity.StartActivity("OrderScooter"); + activity?.SetTag("type", "scooter"); + for (long i = 0; i < 2000000000; i++) + { + } + OrderInternal(searchRadius); + DoSomeOtherWork(); + } + + private void OrderInternal(int searchRadius) + { + using var activity = CustomActivity.StartActivity("OrderScooterInternal"); _orderService.FindNearestVehicle(searchRadius, "scooter"); } -} \ No newline at end of file + + private void DoSomeOtherWork() + { + for (long i = 0; i < 1000000000; i++) + { + } + } +} diff --git a/examples/language-sdk-instrumentation/dotnet/rideshare/musl.Dockerfile b/examples/language-sdk-instrumentation/dotnet/rideshare/musl.Dockerfile index e3218b7795..15e29b6f04 100644 --- a/examples/language-sdk-instrumentation/dotnet/rideshare/musl.Dockerfile +++ b/examples/language-sdk-instrumentation/dotnet/rideshare/musl.Dockerfile @@ -22,6 +22,6 @@ ENV PYROSCOPE_PROFILING_ENABLED=1 ENV PYROSCOPE_PROFILING_ALLOCATION_ENABLED=true ENV PYROSCOPE_PROFILING_CONTENTION_ENABLED=true ENV PYROSCOPE_PROFILING_EXCEPTION_ENABLED=true +ENV RIDESHARE_LISTEN_PORT=5000 - -CMD ["dotnet", "/dotnet/example.dll"] +CMD sh -c "ASPNETCORE_URLS=http://*:${RIDESHARE_LISTEN_PORT} exec dotnet /dotnet/example.dll" diff --git a/examples/tracing/tempo/docker-compose.yml b/examples/tracing/tempo/docker-compose.yml index 7a45dc09b5..4c92e2ef6a 100644 --- a/examples/tracing/tempo/docker-compose.yml +++ b/examples/tracing/tempo/docker-compose.yml @@ -42,6 +42,26 @@ services: context: ../../language-sdk-instrumentation/java/rideshare dockerfile: Dockerfile.otel-instrumentation + rideshare-dotnet-eu-west: + platform: linux/amd64 + ports: + - 5000 + hostname: rideshare-dotnet-eu-west + environment: + <<: *env + OTEL_TRACES_EXPORTER: otlp + OTEL_EXPORTER_OTLP_ENDPOINT: http://tempo:4317 + OTEL_SERVICE_NAME: rideshare.dotnet.push.app + OTEL_METRICS_EXPORTER: none + OTEL_TRACES_SAMPLER: always_on + OTEL_PROPAGATORS: tracecontext + OTEL_RESOURCE_ATTRIBUTES: host.name=rideshare-dotnet-eu-west + PYROSCOPE_LABELS: hostname=rideshare-dotnet-eu-west + REGION: eu-west + build: + context: ../../language-sdk-instrumentation/dotnet/rideshare + dockerfile: Dockerfile + load-generator: environment: *env build: @@ -52,6 +72,7 @@ services: - http://rideshare-go-ap-south:5000 - http://rideshare-go-eu-north:5000 - http://rideshare-java-us-east:5000 + - http://rideshare-dotnet-eu-west:5000 grafana: image: grafana/grafana-dev:10.3.0-151740