diff --git a/docs/how_to_guides/telemetry.mdx b/docs/how_to_guides/telemetry.mdx index 50b21ce63..100670aac 100644 --- a/docs/how_to_guides/telemetry.mdx +++ b/docs/how_to_guides/telemetry.mdx @@ -2,7 +2,11 @@ import Dashboard from './assets/dashboard.png'; import ViewTraces from './assets/view_traces.mp4'; import GrafanaOtelConfig from './assets/grafana_otel_config.mp4'; -# Capture Metrics for your Guards +# Overview + +In this document, we explain how to set up Guardrails with [OpenTelemetry (OTEL)](https://opentelemetry.io/) using either Grafana or a self-hosted OTEL collector. With this functionality enabled, you can measure latency of Guards, Large Language Models (LLMs), scuccess rates, and other metrics for your Guardrails-protected LLM calls. + +## Metrics you can capture using OTEL This package is instrumented using the OpenTelemetry Python SDK. By viewing the captured traces and derived metrics, we're able to get useful insights into how our Guards, and our LLM apps in general perform. Among other things, we're able to find: @@ -12,12 +16,11 @@ This package is instrumented using the OpenTelemetry Python SDK. By viewing the 4. The rate at which validators Pass and Fail, within a Guard and across Guards 5. Deep dives into singular guard and validator calls -Since we are using OpenTelemetry, traces and metrics can be written to any OpenTelemetry enabled service or OTLP endpoint. This includes all major metrics providers like Grafana, New Relic, Prometheus, and Splunk. +Since we are using OpenTelemetry, traces and metrics can be written to any OpenTelemetry-enabled service or OTLP endpoint. This includes all major metrics providers like Grafana, New Relic, Prometheus, and Splunk. -This guide will show how to set up your python project to log traces to Grafana and an OTEL collector. +This guide will show how to set up your Python project to log traces to Grafana and to a self-hosted OTEL collector. For other OTEL endpoints, consult your metrics provider's documentation on OTEL support. - -## Setup with Grafana +## Configure OTEL for Grafana Grafana Cloud is a free offering by Grafana that's easy to setup and is our preferred location for storing metrics. @@ -41,11 +44,11 @@ OTEL_EXPORTER_OTLP_HEADERS ### Setup a Guard with Telemetry -We first have to install the ```ValidLength``` guardrail from Guardrails Hub. +1. First, install the ```ValidLength``` guardrail from Guardrails Hub. ```guardrails hub install hub://guardrails/valid_length``` -Then, set up your Guard the default tracer provided in the guardrails library. You can still use your desired validators +2. Next, set up your Guard the default tracer provided in the guardrails library. You can still use your desired validators:
main.py