Skip to content

Commit

Permalink
[DOC] Update Configure the client and span profiles doc (#3028)
Browse files Browse the repository at this point in the history 
* Update Configure the client page

* Update Span Profiles doc

* Apply suggestions from code review

* fix broken link from validator

* Update _index.md

* Update docs/sources/configure-client/go-span-profiles.md
  • Loading branch information
@knylander-grafana
knylander-grafana authored Feb 29, 2024
1 parent c723cba commit 87423e4
Show file tree
Hide file tree
Showing 2 changed files with 65 additions and 50 deletions.
53 changes: 33 additions & 20 deletions docs/sources/configure-client/_index.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,47 +2,61 @@
aliases:
- /docs/phlare/latest/operators-guide/configure-agent/
- /docs/phlare/latest/configure-client/
title: "Sending profiles from your application"
title: "Configure the client to send profiles"
menuTitle: "Configure the client"
description: "Learn how to configure the client to send profiles from your application."
weight: 35
---

# Sending profiles from your application
# Configure the client to send profiles

Pyroscope is a continuous profiling database that allows you to analyze the performance of your applications. When sending profiles to Pyroscope, you can choose between two primary methods: SDK Instrumentation and Auto-Instrumentation using the Grafana agent. This document will explain these two techniques and guide you when to choose each one.
Pyroscope is a continuous profiling database that allows you to analyze the performance of your applications.
When sending profiles to Pyroscope, you can choose between two primary methods: SDK Instrumentation and auto-instrumentation using the Grafana Agent.

This document explains these two techniques and guide you when to choose each one.

![Pyroscope agent server diagram](https://grafana.com/media/docs/pyroscope/pyroscope_client_server_diagram.png)

## Grafana Agent (Auto-Instrumentation)
## About auto-instrumentation with Grafana Agent

The Grafana Agent is a component that runs alongside your application and periodically gathers profiling data from it.
This method is suitable when you want to collect profiles from existing applications without modifying their source code.
This approach is simplified with the eBPF profiling option that doesn't necessitate pull or push mechanisms.

The Grafana agent is a component that runs alongside your application and periodically gathers profiling data from it. This method is suitable when you want to collect profiles from existing applications without modifying their source code. This approach is made even simpler with the eBPF profiling option that doesn't necessitate pull or push mechanisms. Here's how it works:
Here's how it works:

1. Install and configure the Grafana agent on the same machine or container where your application is running
2. The agent will periodically retrieve your application's performance profiling data, regardless of the language or technology stack your application is using
1. Install and configure the Grafana Agent on the same machine or container where your application is running
2. The Agent periodically retrieves your application's performance profiling data, regardless of the language or technology stack your application is using
3. The captured profiles are then sent to the Pyroscope server for storage and analysis

Using the Grafana agent provides a hassle-free option, especially when dealing with multiple applications or microservices, allowing you to centralize the profiling process without changing your application's codebase.
Using the Grafana Agent provides a hassle-free option, especially when dealing with multiple applications or microservices, allowing you to centralize the profiling process without changing your application's codebase.

## About instrumentation with Pyroscope SDKs

## Pyroscope SDKs (SDK Instrumentation)
Alternatively, Pyroscope SDKs offer you the ability to instrument your application directly for more precise profiling.
Use the SDKs when you want complete control over the profiling process or when the application you are profiling is written in a language supported by the SDKs (for example, Java, Python, .NET, and others).

Alternatively, Pyroscope SDKs offer you the ability to instrument your application directly for more precise profiling. This mode is suitable when you wish to have complete control over the profiling process or when the application you are profiling is written in a language supported by the SDKs (e.g. Java, Python, .NET, etc.). Here's how to use Pyroscope SDKs:
Here's how to use Pyroscope SDKs:

1. Install the relevant Pyroscope SDK for your application's programming language (e.g. pip package, npm package, Ruby gem, etc.)
1. Install the relevant Pyroscope SDK for your application's programming language (for example, pip package, npm package, Ruby gem).
2. Instrument your application's code using the SDK to capture the necessary profiling data
3. SDK will automatically periodically push the captured profiles to the Pyroscope server for storage and analysis
3. SDK automatically periodically push the captured profiles to the Pyroscope server for storage and analysis

By using the Pyroscope SDKs, you have the flexibility to customize the profiling process according to your application's specific requirements. You can selectively profile specific sections of code or send profiles at different intervals, depending on your needs.
By using the Pyroscope SDKs, you have the flexibility to customize the profiling process according to your application's specific requirements.
You can selectively profile specific sections of code or send profiles at different intervals, depending on your needs.

## Choosing the Grafana agent or Pyroscope SDK to send profiles
## Choose Grafana Agent or Pyroscope SDK to send profiles

The choice between using Grafana Agent (Auto-Instrumentation) or Pyroscope SDKs (SDK Instrumentation) depends on your specific use case and requirements. Here are some factors to consider when making the choice:
You can use Grafana Agent for auto-instrumentation or the Pyroscope instrumentation SDKs.
The method you choose depends on your specific use case and requirements.

- Ease of setup: The Grafana agent is an ideal choice for a quick and straightforward setup without modifying your application's code. Note that eBPF profiling supports some languages (i.e. Golang) better than others, but more robust support for Python, Java, and other languages is coming soon!
- Language support: If your application is written in a language supported by the Pyroscope SDKs, and you want more control over the profiling process, using the SDKs is recommended.
Here are some factors to consider when making the choice:

- Ease of setup: The Grafana Agent is an ideal choice for a quick and straightforward setup without modifying your application's code. Note that eBPF profiling supports some languages (for example, Golang) better than others. More robust support for Python, Java, and other languages is coming soon!
- Language support: If you want more control over the profiling process and your application is written in a language supported by the Pyroscope SDKs, consider using the SDKs.
- Flexibility: The Pyroscope SDKs offer greater flexibility in terms of customizing the profiling process and capturing specific sections of code with labels. If you have particular profiling needs or want to fine-tune the data collection process, the SDKs would be your best bet.

To get started choose one of the integrations below:
To get started, choose one of the integrations below:
<table>
<tr>
<td align="center"><a href="https://grafana.com/docs/pyroscope/latest/configure-client/grafana-agent/go_pull"><img src="https://github-production-user-asset-6210df.s3.amazonaws.com/223048/257522425-48683963-91ae-4caf-8c52-ce131e25bd65.png" width="100px;" alt=""/><br />
Expand Down Expand Up @@ -95,5 +109,4 @@ To get started choose one of the integrations below:
</tr>
</table>


If you have more questions feel free to reach out in our Slack channel or create an issue on GitHub and the Pyroscope team will help!
If you have more questions, feel free to reach out in [our Slack channel](https://grafana.slack.com/) or create an [issue on GitHub](https://github.com/grafana/pyroscope) and the Pyroscope team will help!
62 changes: 32 additions & 30 deletions docs/sources/configure-client/go-span-profiles.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,38 +7,52 @@ weight: 100

# Span profiles with Traces to profiles for Go

Span Profiles represents a major shift in profiling methodology, enabling deeper analysis of both tracing and profiling data.
Traditional continuous profiling provides an application-wide view over fixed intervals.
In contrast, Span Profiles delivers focused, dynamic analysis on specific execution scopes within applications, such as individual requests or specific trace spans.

This shift enables a more granular view of performance, enhancing the utility of profiles by linking them directly with traces for a comprehensive understanding of application behavior. As a result, engineering teams can more efficiently identify and address performance bottlenecks.

To learn more about Span Profiles, refer to [Combining tracing and profiling for enhanced observability: Introducing Span Profiles](/blog/2024/02/06/combining-tracing-and-profiling-for-enhanced-observability-introducing-span-profiles/).

![span-profiles screenshot](https://grafana.com/static/img/docs/tempo/profiles/tempo-profiles-Span-link-profile-data-source.png)

To use Span Profiles, you need to:

* [Configure Pyroscope to send profiling data]({{< relref "../configure-client" >}})
* Configure a client-side package to link traces and profiles: [Go](https://github.com/grafana/otel-profiling-go), [Ruby](https://github.com/grafana/otel-profiling-ruby), and [Java](https://github.com/grafana/otel-profiling-java)
* [Configure Tempo data source in Grafana or Grafana Cloud to discover linked traces and profiles](/grafana-cloud/connect-externally-hosted/data-sources/tempo/configure-tempo-data-source/)

## Before you begin

Your applications must be instrumented for profiling and tracing before you can use span profiles.
Your applications must be instrumented for profiling and tracing before you can use span profiles.

* Profiling: Your application must be instrumented with Pyroscopes Go SDK. If you haven't done this yet, please refer to the [Go (push mode)]({{< relref "../configure-client/language-sdks/go_push" >}}) guide.
* Tracing: Your application must be instrumented with OpenTelemetry traces. If you haven't done this yet, please refer to the [OpenTelemetry](https://opentelemetry.io/docs/go/getting-started/) guide.

## OpenTelemetry support
### OpenTelemetry support

Pyroscope can integrate with distributed tracing systems supporting [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/go/getting-started/) standard which allows you to
link traces with the profiling data, and find resource usage for specific lines of code for your trace spans
Pyroscope can integrate with distributed tracing systems supporting [**OpenTelemetry**](https://opentelemetry.io/docs/instrumentation/go/getting-started/) standard, which allows you to
link traces with the profiling data, and find resource usage for specific lines of code for your trace spans.

:::note
* Only CPU profiling is supported at the moment.
{{% admonition type="note"%}}
* Only CPU profiling is supported.
* Because of how sampling profilers work, spans shorter than the sample interval may not be captured. Go CPU profiler probes stack traces 100 times per second, meaning that spans shorter than 10ms may not be captured.
:::

{{% /admonition %}}

## Configure the otel-profiling-go package
## Configure the `otel-profiling-go` package

To start collecting Span Profiles for your Go application, you need to include [otel-profiling-go](https://github.com/pyroscope-io/otel-profiling-go) in your code.
To start collecting Span Profiles for your Go application, you need to include the [`otel-profiling-go`](https://github.com/pyroscope-io/otel-profiling-go) package in your code.

This package is a `TracerProvider` implementation, that labels profiling data with span IDs which makes it possible to query for span-specific profiling data in Grafana Tempo UI.
This package is a `TracerProvider` implementation that labels profiling data with span IDs. This makes it possible to query for span-specific profiling data with a Tempo data source configured in Grafana or Grafana Cloud.

```shell
# Make sure you also upgrade pyroscope server to version 0.14.0 or higher.
go get github.com/grafana/otel-profiling-go
```

Next you need to create and configure the tracer provider:
Next, you need to create and configure the tracer provider:

```go
package main

Expand All @@ -64,37 +78,25 @@ func main() {
}
```

Now that we set up the tracer, we can create a new trace from anywhere and the profiler will automatically capture profiles for it.
Now that you set up the tracer, you can create a new trace from anywhere and the profiler automatically captures profiles for it.
```go
ctx, span := otel.Tracer("tracerName").Start(ctx, "ExampleSpan")
defer span.End()

// Your code goes here.
```

## View the span profiles in Grafana Tempo

To view the span profiles in Grafana Tempo, you need to have a Grafana instance running and a data source configured to link trace spans and profiles.

Refer to the [data source configuration documentation](/docs/grafana/datasources/tempo/configure-tempo-data-source) to see how to configure the visualization to link trace spans with profiles.

To use a simple configuration, follow these steps:

1. Select a Pyroscope data source from the Data source drop-down.

2. Optional: Choose any tags to use in the query. If left blank, the default values of service.name and service.namespace are used.

The tags you configure must be present in the spans attributes or resources for a trace to profiles span link to appear. You can optionally configure a new name for the tag. This is useful for example if the tag has dots in the name and the target data source doesn’t allow using dots in labels. In that case you can for example remap service.name to service_name.
## View the span profiles in Grafana or Grafana Cloud

3. Select one or more profile types to use in the query. Select the drop-down and choose options from the menu.
To view the span profiles in Grafana Tempo, you need to have a Grafana instance with a Tempo data source configured to link trace spans and profiles.
Refer to the configuration documentation for [Grafana](/docs/grafana/<GRAFANA_VERSION>/datasources/tempo/configure-tempo-data-source) or [Grafana Cloud](/docs/grafana-cloud/connect-externally-hosted/data-sources/tempo/configure-tempo-data-source).

The profile type or app must be selected for the query to be valid. Grafana doesn’t show any data if the profile type or app isn’t selected when a query runs.
To learn how to set up Traces to profiles and view the span profiles, refer to [Traces to profiles]({{< relref "../view-and-analyze-profile-data/profile-tracing/traces-to-profiles" >}}).

![span-profiles configuration](https://grafana.com/static/img/docs/tempo/profiles/Tempo-data-source-profiles-Settings.png)

## Examples

Check out the [examples](https://github.com/grafana/pyroscope/tree/main/examples/tracing/tempo) directory in our repository to
Check out the [examples directory](https://github.com/grafana/pyroscope/tree/main/examples/tracing/tempo) in the Pyroscope GitHub repository to
find a complete example application that demonstrates tracing integration features.

<!-- ## Using tracing exemplars manually
Expand Down

0 comments on commit 87423e4

Please sign in to comment.