Example and docs for Python span profiles (#3252)

* Add example for Python span profiles * Add docs for Python span profiles * Apply suggestions from code review Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> * Update docs/sources/configure-client/trace-span-profiles/_index.md Co-authored-by: Anton Kolesnikov <anton.e.kolesnikov@gmail.com> --------- Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com> Co-authored-by: Anton Kolesnikov <anton.e.kolesnikov@gmail.com>
grafana · Apr 26, 2024 · 960fc32 · 960fc32
1 parent 5c34f63
commit 960fc32
Show file tree

Hide file tree

Showing 6 changed files with 121 additions and 6 deletions.
diff --git a/docs/sources/configure-client/trace-span-profiles/_index.md b/docs/sources/configure-client/trace-span-profiles/_index.md
@@ -21,11 +21,12 @@ Key benefits and features:
 Get started:
 
 - Configure Pyroscope: Begin sending profiling data to unlock the full potential of Span Profiles
-- Client-Side Packages: Easily link traces and profiles using available packages for Go, Java, Ruby and .NET
+- Client-side packages: Easily link traces and profiles using available packages for Go, Java, Ruby, .NET, and Python
   - Go: [Span profiles with Traces to profiles (Go)]({{< relref "./go-span-profiles" >}})
   - Java: [Span profiles with Traces to profiles (Java)]({{< relref "./java-span-profiles" >}})
   - Ruby: [Span profiles with Traces to profiles (Ruby)]({{< relref "./ruby-span-profiles" >}})
   - .NET: [Span profiles with Traces to profiles (.NET)]({{< relref "./dotnet-span-profiles" >}})
+  - Python: [Span profiles with Traces to profiles (Python)]({{< relref "./python-span-profiles" >}})
 - Grafana Tempo: Visualize and analyze Span Profiles within the Grafana using a Tempo data source.
 
 To learn more, check out our product announcement blog: [Introducing Span Profiles](/blog/2024/02/06/combining-tracing-and-profiling-for-enhanced-observability-introducing-span-profiles/).
diff --git a/docs/sources/configure-client/trace-span-profiles/dotnet-span-profiles.md b/docs/sources/configure-client/trace-span-profiles/dotnet-span-profiles.md
@@ -17,7 +17,7 @@ To learn more about Span Profiles, refer to [Combining tracing and profiling for
 
 ![span-profiles screenshot](https://grafana.com/static/img/docs/tempo/profiles/tempo-profiles-Span-link-profile-data-source.png)
 
-Pyroscope integrates with distributed tracing systems supporting the [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/net/getting-started/) standard.
+Pyroscope integrates with distributed tracing systems supporting the [**OpenTelemetry**](https://opentelemetry.io/docs/languages/net/getting-started/) standard.
 This integration lets you link traces with the profiling data and find resource usage for specific lines of code for your trace spans.
 
 {{< admonition type="note" >}}
@@ -36,7 +36,7 @@ To use Span Profiles, you need to:
 Your applications must be instrumented for profiling and tracing before you can use span profiles.
 
 * Profiling: Your application must be instrumented with Pyroscope's .NET instrumentation library. Refer to the [.NET]({{< relref "../language-sdks/dotnet" >}}) guide for instructions.
-* Tracing: Your application must be instrumented with OpenTelemetry traces. Refer to the [OpenTelemetry](https://opentelemetry.io/docs/net/getting-started/) guide for isntructions.
+* Tracing: Your application must be instrumented with OpenTelemetry traces. Refer to the [OpenTelemetry](https://opentelemetry.io/docs/languages/net/getting-started/) guide for isntructions.
 
 {{< admonition type="note" >}}
 Span profiles in .NET are only supported using [OpenTelemetry manual instrumentation](https://opentelemetry.io/docs/languages/net/instrumentation/)

diff --git a/docs/sources/configure-client/trace-span-profiles/python-span-profiles.md b/docs/sources/configure-client/trace-span-profiles/python-span-profiles.md
@@ -0,0 +1,78 @@
+---
+title: Span profiles with Traces to profiles for Python
+menuTitle: Span profiles with Traces to profiles (Python)
+description: Learn about and configure Span profiles with Traces to profiles in Grafana for Python applications.
+weight: 104
+---
+
+# Span profiles with Traces to profiles for Python
+
+Span Profiles represents a major shift in profiling methodology, enabling deeper analysis of both tracing and profiling data.
+Traditional continuous profiling provides an application-wide view over fixed intervals.
+In contrast, Span Profiles delivers focused, dynamic analysis on specific execution scopes within applications, such as individual requests or specific trace spans.
+
+This shift enables a more granular view of performance, enhancing the utility of profiles by linking them directly with traces for a comprehensive understanding of application behavior. As a result, engineering teams can more efficiently identify and address performance bottlenecks.
+
+To learn more about Span Profiles, refer to [Combining tracing and profiling for enhanced observability: Introducing Span Profiles](/blog/2024/02/06/combining-tracing-and-profiling-for-enhanced-observability-introducing-span-profiles/).
+
+![span-profiles screenshot](https://grafana.com/static/img/docs/tempo/profiles/tempo-profiles-Span-link-profile-data-source.png)
+
+Pyroscope integrates with distributed tracing systems supporting the [**OpenTelemetry**](https://opentelemetry.io/docs/languages/python/getting-started/) standard.
+This integration lets you link traces with the profiling data and find resource usage for specific lines of code for your trace spans.
+
+{{< admonition type="note" >}}
+* Only CPU profiling is supported at the moment.
+* Because of how sampling profilers work, spans shorter than the sample interval may not be captured.
+{{< /admonition >}}
+
+To use Span Profiles, you need to:
+
+* [Configure Pyroscope to send profiling data]({{< relref "../../configure-client" >}})
+* Configure a client-side package to link traces and profiles: [Python](https://github.com/grafana/otel-profiling-python)
+* [Configure the Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles](/grafana-cloud/connect-externally-hosted/data-sources/tempo/configure-tempo-data-source/)
+
+## Before you begin
+
+Your applications must be instrumented for profiling and tracing before you can use span profiles.
+
+* Profiling: Your application must be instrumented with Pyroscope's Python instrumentation library. Refer to the [Python]({{< relref "../language-sdks/python" >}}) guide for instructions.
+* Tracing: Your application must be instrumented with OpenTelemetry traces. Refer to the [OpenTelemetry](https://opentelemetry.io/docs/languages/python/getting-started/) guide for isntructions.
+
+## Configure the `pyroscope-otel` package
+
+To start collecting Span Profiles for your Python application, you need to include [pyroscope-otel](https://github.com/grafana/otel-profiling-python) in your code.
+
+This package provides a [`SpanProcessor`](https://github.com/open-telemetry/opentelemetry-python/blob/d213e02941039d4383abc3608b75404ce84725b1/opentelemetry-sdk/src/opentelemetry/sdk/trace/__init__.py#L85) implementation, which connects the two telemetry signals (traces and profiles) together.
+
+```shell
+pip install pyroscope-otel
+```
+
+Next, create and register the `PyroscopeSpanProcessor`:
+```python
+# import span processor
+from pyroscope-otel import PyroscopeSpanProcessor
+
+# obtain a OpenTelemetry tracer provider
+from opentelemetry import trace
+from opentelemetry.sdk.trace import TracerProvider
+provider = TracerProvider()
+
+# register the span processor
+provider.add_span_processor(PyroscopeSpanProcessor())
+
+# register the tracer provider
+trace.set_tracer_provider(provider)
+```
+
+With the span processor registered, spans created automatically (for example, HTTP handlers) and manually will have profiling data associated with them.
+
+## View the span profiles in Grafana Tempo
+
+To view the span profiles in Grafana Tempo, you need to have a Grafana instance running and a data source configured to link traces and profiles.
+
+Refer to the [data source configuration documentation](/docs/grafana/datasources/tempo/configure-tempo-data-source) to see how to configure the visualization to link traces with profiles.
+
+## Examples
+
+Check out the [examples](https://github.com/grafana/pyroscope/tree/main/examples/tracing/tempo) directory for a complete demo application of span profiles in multiple languages.
diff --git a/examples/language-sdk-instrumentation/python/rideshare/flask/Dockerfile b/examples/language-sdk-instrumentation/python/rideshare/flask/Dockerfile
@@ -1,10 +1,10 @@
 FROM python:3.9
 
-RUN pip3 install flask pyroscope-io==0.8.6
+RUN pip3 install flask pyroscope-io==0.8.7 pyroscope-otel==0.1.0
+RUN pip3 install opentelemetry-api opentelemetry-sdk opentelemetry-instrumentation-flask opentelemetry-exporter-otlp-proto-grpc
 
 ENV FLASK_ENV=development
 ENV PYTHONUNBUFFERED=1
 
 COPY lib ./lib
 CMD [ "python", "lib/server.py" ]
-
diff --git a/examples/language-sdk-instrumentation/python/rideshare/flask/lib/server.py b/examples/language-sdk-instrumentation/python/rideshare/flask/lib/server.py
@@ -1,11 +1,27 @@
 import os
-import time
+
 import pyroscope
 from flask import Flask
+
+# OpenTelemetry
+from opentelemetry.instrumentation.flask import FlaskInstrumentor
+from opentelemetry import trace
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import BatchSpanProcessor
+from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
+from pyroscope.otel import PyroscopeSpanProcessor
+
 from bike.bike import order_bike
 from car.car import order_car
 from scooter.scooter import order_scooter
 
+provider = TracerProvider()
+provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
+provider.add_span_processor(PyroscopeSpanProcessor())
+
+# Sets the global default tracer provider
+trace.set_tracer_provider(provider)
+
 app_name = os.getenv("PYROSCOPE_APPLICATION_NAME", "flask-ride-sharing-app")
 server_addr = os.getenv("PYROSCOPE_SERVER_ADDRESS", "http://pyroscope:4040")
 basic_auth_username = os.getenv("PYROSCOPE_BASIC_AUTH_USER", "")
@@ -23,6 +39,7 @@
 )
 
 app = Flask(__name__)
+FlaskInstrumentor().instrument_app(app)
 
 @app.route("/bike")
 def bike():

diff --git a/examples/tracing/tempo/docker-compose.yml b/examples/tracing/tempo/docker-compose.yml
@@ -62,6 +62,24 @@ services:
       context: ../../language-sdk-instrumentation/dotnet/rideshare
       dockerfile: Dockerfile
 
+  rideshare-python-eu-east:
+    ports:
+      - 5000
+    hostname: rideshare-python-eu-east
+    environment:
+      <<: *env
+      OTEL_TRACES_EXPORTER: otlp
+      OTEL_EXPORTER_OTLP_ENDPOINT: http://tempo:4317
+      OTEL_SERVICE_NAME: rideshare.python.push.app
+      OTEL_METRICS_EXPORTER: none
+      OTEL_TRACES_SAMPLER: always_on
+      OTEL_PROPAGATORS: tracecontext
+      PYROSCOPE_LABELS: hostname=rideshare-python-us-east
+      REGION: us-east
+    build:
+      context: ../../language-sdk-instrumentation/python/rideshare/flask
+      dockerfile: Dockerfile
+
   load-generator:
     environment: *env
     build:
@@ -73,6 +91,7 @@ services:
       - http://rideshare-go-eu-north:5000
       - http://rideshare-java-us-east:5000
       - http://rideshare-dotnet-eu-west:5000
+      - http://rideshare-python-eu-east:5000
 
   grafana:
     image: grafana/grafana-dev:10.3.0-151740