Skip to content

Commit

Permalink
docs: add documentation for Pyroscope SDK profiles through Alloy (#3658)
Browse files Browse the repository at this point in the history
* docs: add documentation for Pyroscope SDK profiles through Alloy

Co-authored-by: Kim Nylander <104772500+knylander-grafana@users.noreply.github.com>
  • Loading branch information
marcsanmi and knylander-grafana authored Nov 13, 2024
1 parent bb45e2e commit 44b9335
Show file tree
Hide file tree
Showing 3 changed files with 240 additions and 7 deletions.
28 changes: 21 additions & 7 deletions docs/sources/configure-client/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,12 @@ weight: 300
# Configure the client to send profiles

Pyroscope is a continuous profiling database that allows you to analyze the performance of your applications.
When sending profiles to Pyroscope, you can choose between two primary methods: SDK instrumentation and auto-instrumentation using Grafana Alloy.
When sending profiles to Pyroscope, you can choose between three methods:
1. Auto-instrumentation using Grafana Alloy
2. SDK instrumentation
3. SDK instrumentation through Grafana Alloy

This document explains these two techniques and guide you when to choose each one.
This document explains these techniques and guide you when to choose each one.

![Pyroscope agent server diagram](https://grafana.com/media/docs/pyroscope/pyroscope_client_server_diagram_09_18_2024.png)

Expand Down Expand Up @@ -57,16 +60,27 @@ Here's how to use Pyroscope SDKs:
By using the Pyroscope SDKs, you have the flexibility to customize the profiling process according to your application's specific requirements.
You can selectively profile specific sections of code or send profiles at different intervals, depending on your needs.

## Choose Grafana Alloy or Pyroscope SDK to send profiles
## About instrumentation with Pyroscope SDKs through Alloy

You can use Grafana Alloy for auto-instrumentation or the Pyroscope instrumentation SDKs.
Pyroscope SDKs can be configured to send profiles to Grafana Alloy first, which then forwards them to the Pyroscope server. This method combines the flexibility of SDK instrumentation with Alloy's infrastructure benefits.

Here's how it works:
1. Your application is instrumented with Pyroscope SDKs
2. Instead of sending profiles directly to Pyroscope, the SDK sends them to Alloy's `pyroscope.receive_http` component
3. Alloy processes and forwards the profiles to the Pyroscope server

By sending profiles through Alloy, you benefit from lower latency as profiles are sent to a local Alloy instance instead of directly over the internet to Grafana Cloud. Your application code remains focused on instrumentation while infrastructure concerns like authentication and routing are handled by Alloy's configuration. This separation allows for centralized management of metadata, where you can enrich profiles with infrastructure labels such as Kubernetes metadata or environment tags without modifying application code.

## Choose the right profiling method

You can use Grafana Alloy for auto-instrumentation, the Pyroscope instrumentation SDKs directly, or SDKs through Alloy.
The method you choose depends on your specific use case and requirements.

Here are some factors to consider when making the choice:

- Ease of setup: Grafana Alloy is an ideal choice for a quick and straightforward setup without modifying your application's code. eBPF profiling supports some languages (for example, Golang, Python) better than others. More robust support for Java and other languages is in development.
- Language support: If you want more control over the profiling process and your application is written in a language supported by the Pyroscope SDKs, consider using the SDKs.
- Flexibility: The Pyroscope SDKs offer greater flexibility in terms of customizing the profiling process and capturing specific sections of code with labels. If you have particular profiling needs or want to fine-tune the data collection process, the SDKs would be your best bet.
- Ease of setup: Grafana Alloy is an ideal choice for a quick and straightforward setup without modifying your application's code. eBPF profiling supports some languages (for example, Golang, Python) better than others. More robust support for Java and other languages is in development. Using SDKs through Alloy adds minimal setup complexity while providing infrastructure benefits.
- Language support: If you want more control over the profiling process and your application is written in a language supported by the Pyroscope SDKs, consider using the SDKs - either directly or through Alloy depending on your infrastructure needs.
- Flexibility: The Pyroscope SDKs offer greater flexibility in terms of customizing the profiling process and capturing specific sections of code with labels. If you have particular profiling needs or want to fine-tune the data collection process, the SDKs would be your best bet. When used with Alloy, you gain additional infrastructure flexibility without compromising SDK capabilities.

To get started, choose one of the integrations below:
<table>
Expand Down
14 changes: 14 additions & 0 deletions docs/sources/configure-client/grafana-alloy/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,20 @@ In pull mode, the collector periodically retrieves profiles from Golang applicat
1. Ensure Alloy is set to pull mode and targeting the correct pprof endpoints. For step-by-step instructions, visit the [Go (Pull Mode)](https://grafana.com/docs/pyroscope/<PYROSCOPE_VERSION>/configure-client/grafana-alloy/go_pull) documentation.
1. The collector queries the pprof endpoints of your Golang application, collects the profiles, and forwards them to the Pyroscope server.

## Receive profiles from Pyroscope SDKs

Alloy can receive profiles from applications instrumented with Pyroscope SDKs through the `pyroscope.receive_http` component. This approach provides several key advantages:
- Improved performance by sending profiles to a local Alloy instance instead of over the internet to Grafana Cloud, reducing latency and application impact
- Separation of infrastructure concerns from application code - developers don't need to handle authentication, tenant configuration, or infrastructure labels in their code
- Centralized management of authentication and metadata enrichment (for example, Kubernetes labels, business labels)

This capability is not available in the legacy Grafana Agent.

### Set up profile receiving

1. Configure your application with a Pyroscope SDK pointing to receive endpoint for Alloy.
2. For step-by-step instructions, refer to the [Receive profiles from Pyroscope SDKs](https://grafana.com/docs/pyroscope/<PYROSCOPE_VERSION>/configure-client/grafana-alloy/receive_profiles) documentation.

## Next steps

Whether using eBPF for versatile system and application profiling or relying on Golang's built-in pprof endpoints in pull mode, Alloy collectors offer streamlined processes to gather essential profiling data.
Expand Down
205 changes: 205 additions & 0 deletions docs/sources/configure-client/grafana-alloy/receive_profiles.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,205 @@
---
title: Receive profiles from Pyroscope SDKs
menuTitle: Receive SDK profiles
description: Learn how to configure Grafana Alloy to receive profiles from applications using Pyroscope SDKs.
weight: 10
---

# Receive profiles from Pyroscope SDKs

The `pyroscope.receive_http` component in Alloy receives profiles from applications instrumented with Pyroscope SDKs. This approach provides several benefits:
- Lower latency by sending profiles to a local Alloy instance instead of over internet
- Separation of infrastructure concerns (auth, routing) from application code
- Centralized management of authentication and metadata enrichment

For more information about this component, refer to the [pyroscope.receive_http component](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/pyroscope/pyroscope.receive_http/) documentation.

{{< admonition type="note" >}}
The `pyroscope.receive_http` component is currently in public preview. To use this component, set the `--stability.level` flag to `public-preview`. For more information about Alloy's run usage, refer to the [run command documentation](https://grafana.com/docs/grafana-cloud/send-data/alloy/reference/cli/run/#the-run-command) documentation.
{{< /admonition >}}

To set up profile receiving, you need to:
1. Configure Alloy components
2. Configure your application's SDK
3. Start Alloy

## Configure Alloy components

The configuration requires at least two components:
- [`pyroscope.receive_http`](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/pyroscope/pyroscope.receive_http/) to receive profiles via HTTP
- [`pyroscope.write`](https://grafana.com/docs/alloy/<ALLOY_VERSION>/reference/components/pyroscope/pyroscope.write/) to forward profiles to Pyroscope

Here's a basic configuration that sets up a simple profile collection pipeline.
It creates a receiver to collect profiles from your applications and forwards them through a writer component to send them to the Pyroscope backend:

```alloy
// Receives profiles over HTTP
pyroscope.receive_http "default" {
http {
listen_address = "0.0.0.0"
listen_port = 9999
}
forward_to = [pyroscope.write.backend.receiver]
}
// Forwards profiles to Pyroscope
pyroscope.write "backend" {
endpoint {
url = "http://pyroscope:4040"
}
}
```

## Configure application SDK
Update your application's SDK configuration to point to Alloy's receive endpoint instead of Pyroscope directly. For example, in Go:
```go
config := pyroscope.Config{
ApplicationName: "my.service.cpu",
ServerAddress: "http://localhost:9999", // Alloy's receive endpoint
}
```
Check your specific language SDK documentation for the exact configuration options.

## Examples

The examples in this section provide samples you can use as a starting point for your own configurations.

### Basic receiving setup

This example shows a basic setup receiving profiles on port 9090 and forwarding them to a local Pyroscope instance:


```alloy
pyroscope.receive_http "default" {
http {
listen_address = "0.0.0.0"
listen_port = 9090
}
forward_to = [pyroscope.write.backend.receiver]
}
pyroscope.write "production" {
endpoint {
url = "http://localhost:4040"
}
}
```

### Authentication

To send profiles to an authenticated Pyroscope endpoint:

```alloy
pyroscope.write "production" {
endpoint {
url = "http://pyroscope:4040"
basic_auth {
username = env("PYROSCOPE_USERNAME")
password = env("PYROSCOPE_PASSWORD")
}
}
}
```

### Adding external labels
External labels are added to all profiles forwarded through the write component. This is useful for adding infrastructure metadata:
```alloy
pyroscope.receive_http "default" {
http {
listen_address = "0.0.0.0"
listen_port = 9999
}
forward_to = [pyroscope.write.backend.receiver]
}
pyroscope.write "backend" {
endpoint {
url = "http://pyroscope:4040"
}
external_labels = {
"env" = "production",
"region" = "us-west-1",
"instance" = env("HOSTNAME"),
"cluster" = "main",
}
}
```

### Multiple destinations
Forward received profiles to multiple destinations - useful for testing or migration scenarios:
```alloy
pyroscope.receive_http "default" {
http {
listen_address = "0.0.0.0"
listen_port = 9999
}
forward_to = [pyroscope.write.staging.receiver, pyroscope.write.production.receiver]
}
// Send profiles to staging
pyroscope.write "staging" {
endpoint {
url = "http://pyroscope-staging:4040"
}
external_labels = {
"env" = "staging",
}
}
// Send profiles to production
pyroscope.write "production" {
endpoint {
url = "http://pyroscope-production:4041"
}
external_labels = {
"env" = "production",
}
}
```
{{< admonition type="note" >}}
This configuration will duplicate the received profiles and send a copy to each configured `pyroscope.write` component.
{{< /admonition >}}

Another approach is to configure multiple receivers with multiple destinations:
```alloy
pyroscope.receive_http "staging" {
http {
listen_address = "0.0.0.0"
listen_port = 9998
}
forward_to = [pyroscope.write.staging.receiver]
}
pyroscope.receive_http "production" {
http {
listen_address = "0.0.0.0"
listen_port = 9999
}
forward_to = [pyroscope.write.production.receiver]
}
// Send profiles to staging
pyroscope.write "staging" {
endpoint {
url = "http://pyroscope-staging:4040"
}
external_labels = {
"env" = "staging",
}
}
// Send profiles to production
pyroscope.write "production" {
endpoint {
url = "http://pyroscope-production:4041"
}
external_labels = {
"env" = "production",
}
}
```

For more information about component configuration options, refer to:

- pyroscope.receive_http [documentation](https://grafana.com/docs/alloy/latest/reference/components/pyroscope/pyroscope.write/)
- pyroscope.write [documentation](https://grafana.com/docs/alloy/latest/reference/components/pyroscope/pyroscope.receive_http/)

0 comments on commit 44b9335

Please sign in to comment.