Patronus Evaluate API Integration (#834)

varjoshi · Pouyanpi · web-flow · commit 903a3879e22e · 2024-11-04T17:20:57.000+01:00
* Patronus Evaluate API Integration

* Address comments - tests will be added separately

* Add missing tests

* Remove print statements
---------

Signed-off-by: Pouyan &lt;13303554+Pouyanpi@users.noreply.github.com&gt;
Co-authored-by: Pouyan &lt;13303554+Pouyanpi@users.noreply.github.com&gt;
diff --git a/docs/user_guides/community/patronus-evaluate-api.md b/docs/user_guides/community/patronus-evaluate-api.md
@@ -0,0 +1,75 @@
+# Patronus Evaluate API Integration
+
+NeMo Guardrails supports using [Patronus AI](www.patronus.ai)'s Evaluate API as an output rail. The Evaluate API gives you access to Patronus' powerful suite of fully-managed in-house evaluation models, including [Lynx](patronus-lynx.md), Judge (a hosted LLM-as-a-Judge model), Toxicity, PII, and PHI models, and a suite of specialized RAG evaluators with
+industry-leading performance on metrics like Answer Relevance, Context Relevance, Context Sufficiency, and Hallucination.
+
+Patronus also has Managed configurations of the Judge evaluator, which you can use to detect AI failures like prompt injection and brand misalignment in order to prevent problematic bot responses from being returned to users.
+
+## Setup
+
+1. Sign up for an account on [app.patronus.ai](https://app.patronus.ai).
+2. You can follow the Quick Start guide [here](https://docs.patronus.ai/docs/quickstart-guide) to get onboarded.
+3. Create an API Key and save it somewhere safe.
+
+## Usage
+
+Here's how to use the Patronus Evaluate API as an output rail:
+
+1. Get a Patronus API key and set it to the PATRONUS_API_KEY variable in your environment.
+
+2. Add the guardrail `patronus api check output` to your output rails in `config.yml`:
+
+```yaml
+rails:
+  output:
+    flows:
+      - patronus api check output
+```
+
+3. Add a rails config for Patronus in `config.yml`:
+
+```yaml
+rails:
+  config:
+    patronus:
+      output:
+        evaluate_config:
+          success_strategy: "all_pass"
+          params:
+            {
+              evaluators:
+                [
+                  { "evaluator": "lynx" },
+                  {
+                    "evaluator": "answer-relevance",
+                    "explain_strategy": "on-fail",
+                  },
+                ],
+              tags: { "retrieval_configuration": "ast-123" },
+            }
+```
+
+The `evaluate_config` has two top-level arguments: `success_strategy` and `params`.
+
+In `params` you can pass the relevant arguments to the Patronus Evaluate API. The schema is the same as the API documentation [here](https://docs.patronus.ai/reference/evaluate_v1_evaluate_post), so as new API parameters are added and new values are supported, you can readily add them to your NeMo Guardrails configuration.
+
+Note that you can pass in multiple evaluators to the Patronus Evaluate API. By setting `success_strategy` to "all_pass",
+every single evaluator called in the Evaluate API must pass for the rail to pass successfully. If you set it to "any_pass", then only one evaluator needs to pass.
+
+## Additional Information
+
+For now, the Evaluate API Integration only looks at whether the evaluators return Pass or Fail in the API response. However, most evaluators return a score between 0 and 1, where by default a score below 0.5 indicates a Fail and score above 0.5 indicates a Pass. But you can use the score directly to adjust how sensitive your pass/fail threshold should be. The API response can also include explanations of why the rail passed or failed that could be surfaced to a user (set `explain_strategy` in the evaluator object). Some evaluators even include spans of problematic keywords or sentences where issues like hallucinations are present, so you can scrub them out before returning the bot response.
+
+Here's the `patronus api check output` flow, showing how the action is executed:
+
+```colang
+define bot inform answer unknown
+  "I don't know the answer to that."
+
+define flow patronus api check output
+  $patronus_response = execute PatronusApiCheckOutputAction
+  $evaluation_passed = $patronus_response["pass"]
+
+  if not $evaluation_passed
+    bot inform answer unknown
+```
diff --git a/docs/user_guides/llm-support.md b/docs/user_guides/llm-support.md
@@ -37,6 +37,8 @@ If you want to use an LLM and you cannot see a prompt in the [prompts folder](ht
 | Got It AI RAG TruthChecker _(LLM independent)_         | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark: | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:                 |
 | Patronus Lynx RAG Hallucination detection _(LLM independent)_         | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark: | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:                 |
 | GCP Text Moderation _(LLM independent)_         | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark: | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:                 |
+| Patronus Evaluate API _(LLM independent)_         | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark:        | :heavy_check_mark: | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:   | :heavy_check_mark:                 |
+
 
 Table legend:
 - :heavy_check_mark: - Supported (_The feature is fully supported by the LLM based on our experiments and tests_)
diff --git a/examples/configs/patronusai/evaluate_api_config.yml b/examples/configs/patronusai/evaluate_api_config.yml
@@ -0,0 +1,26 @@
+models:
+  - type: main
+    engine: openai
+    model: gpt-3.5-turbo-instruct
+
+rails:
+  output:
+    flows:
+      - patronus api check output
+  config:
+    patronus:
+      output:
+        evaluate_config:
+          success_strategy: "all_pass"
+          params:
+            {
+              evaluators:
+                [
+                  { "evaluator": "lynx" },
+                  {
+                    "evaluator": "answer-relevance",
+                    "explain_strategy": "on-fail",
+                  },
+                ],
+              tags: { "hello": "world" },
+            }
diff --git a/examples/configs/patronusai/lynx_config.yml b/examples/configs/patronusai/lynx_config.yml
diff --git a/nemoguardrails/library/patronusai/actions.py b/nemoguardrails/library/patronusai/actions.py
@@ -14,9 +14,11 @@
 # limitations under the License.
 
 import logging
+import os
 import re
-from typing import List, Optional, Tuple, Union
+from typing import List, Literal, Optional, Tuple, Union
 
+import aiohttp
 from langchain_core.language_models.llms import BaseLLM
 
 from nemoguardrails.actions import action
@@ -106,5 +108,140 @@ async def patronus_lynx_check_output_hallucination(
         )
 
     hallucination, reasoning = parse_patronus_lynx_response(result)
-    print(f"Hallucination: {hallucination}, Reasoning: {reasoning}")
     return {"hallucination": hallucination, "reasoning": reasoning}
+
+
+def check_guardrail_pass(
+    response: Optional[dict], success_strategy: Literal["all_pass", "any_pass"]
+) -> bool:
+    """
+    Check if evaluations in the Patronus API response pass based on the success strategy.
+    "all_pass" requires all evaluators to pass for success.
+    "any_pass" requires only one evaluator to pass for success.
+    """
+    if not response or "results" not in response:
+        return False
+
+    evaluations = response["results"]
+
+    if success_strategy == "all_pass":
+        return all(
+            "evaluation_result" in result
+            and isinstance(result["evaluation_result"], dict)
+            and result["evaluation_result"].get("pass", False)
+            for result in evaluations
+        )
+    return any(
+        "evaluation_result" in result
+        and isinstance(result["evaluation_result"], dict)
+        and result["evaluation_result"].get("pass", False)
+        for result in evaluations
+    )
+
+
+async def patronus_evaluate_request(
+    api_params: dict,
+    user_input: Optional[str] = None,
+    bot_response: Optional[str] = None,
+    provided_context: Optional[Union[str, List[str]]] = None,
+) -> Optional[dict]:
+    """
+    Make a call to the Patronus Evaluate API.
+
+    Returns a dictionary of the API response JSON if successful, or None if a server error occurs.
+        * Server errors will cause the guardrail to block the bot response
+
+    Raises a ValueError for client errors (400-499), as these indicate invalid requests.
+    """
+    api_key = os.environ.get("PATRONUS_API_KEY")
+
+    if api_key is None:
+        raise ValueError("PATRONUS_API_KEY environment variable not set.")
+
+    if "evaluators" not in api_params:
+        raise ValueError(
+            "The Patronus Evaluate API parameters must contain an 'evaluators' field"
+        )
+    evaluators = api_params["evaluators"]
+    if not isinstance(evaluators, list):
+        raise ValueError(
+            "The Patronus Evaluate API parameter 'evaluators' must be a list"
+        )
+
+    for evaluator in evaluators:
+        if not isinstance(evaluator, dict):
+            raise ValueError(
+                "Each object in the 'evaluators' list must be a dictionary"
+            )
+        if "evaluator" not in evaluator:
+            raise ValueError(
+                "Each dictionary in the 'evaluators' list must contain the 'evaluator' field"
+            )
+
+    data = {
+        **api_params,
+        "evaluated_model_input": user_input,
+        "evaluated_model_output": bot_response,
+        "evaluated_model_retrieved_context": provided_context,
+    }
+
+    url = "https://api.patronus.ai/v1/evaluate"
+    headers = {
+        "X-API-KEY": api_key,
+        "Content-Type": "application/json",
+    }
+
+    async with aiohttp.ClientSession() as session:
+        async with session.post(
+            url=url,
+            headers=headers,
+            json=data,
+        ) as response:
+            if 400 <= response.status < 500:
+                raise ValueError(
+                    f"The Patronus Evaluate API call failed with status code {response.status}. "
+                    f"Details: {await response.text()}"
+                )
+
+            if response.status != 200:
+                log.error(
+                    "The Patronus Evaluate API call failed with status code %s. Details: %s",
+                    response.status,
+                    await response.text(),
+                )
+                return None
+
+            response_json = await response.json()
+            return response_json
+
+
+@action(name="patronus_api_check_output")
+async def patronus_api_check_output(
+    llm_task_manager: LLMTaskManager,
+    context: Optional[dict] = None,
+) -> dict:
+    """
+    Check the user message, bot response, and/or provided context
+    for issues based on the Patronus Evaluate API
+    """
+    user_input = context.get("user_message")
+    bot_response = context.get("bot_message")
+    provided_context = context.get("relevant_chunks")
+
+    patronus_config = llm_task_manager.config.rails.config.patronus.output
+    evaluate_config = getattr(patronus_config, "evaluate_config", {})
+    success_strategy: Literal["all_pass", "any_pass"] = getattr(
+        evaluate_config, "success_strategy", "all_pass"
+    )
+    api_params = getattr(evaluate_config, "params", {})
+    response = await patronus_evaluate_request(
+        api_params=api_params,
+        user_input=user_input,
+        bot_response=bot_response,
+        provided_context=provided_context,
+    )
+    return {
+        "pass": check_guardrail_pass(
+            response=response, success_strategy=success_strategy
+        )
+    }
diff --git a/nemoguardrails/library/patronusai/flows.co b/nemoguardrails/library/patronusai/flows.co
@@ -13,3 +13,11 @@ flow patronus lynx check output hallucination
     else
       bot inform answer unknown
     abort
+
+flow patronus api check output
+  $patronus_response = await PatronusApiCheckOutputAction
+  global $evaluation_passed
+  $evaluation_passed = $patronus_response["pass"]
+
+  if not $evaluation_passed
+    bot inform answer unknown
diff --git a/nemoguardrails/library/patronusai/flows.v1.co b/nemoguardrails/library/patronusai/flows.v1.co
@@ -13,3 +13,10 @@ define flow patronus lynx check output hallucination
     else
       bot inform answer unknown
     stop
+
+define flow patronus api check output
+  $patronus_response = execute PatronusApiCheckOutputAction
+  $evaluation_passed = $patronus_response["pass"]
+
+  if not $evaluation_passed
+    bot inform answer unknown
diff --git a/nemoguardrails/rails/llm/config.py b/nemoguardrails/rails/llm/config.py
@@ -18,6 +18,7 @@
 import logging
 import os
 import warnings
+from enum import Enum
 from typing import Any, Dict, List, Optional, Set, Tuple, Union
 
 import yaml
@@ -392,6 +393,54 @@ class AutoAlignRailConfig(BaseModel):
     )
 
 
+class PatronusEvaluationSuccessStrategy(str, Enum):
+    """
+    Strategy for determining whether a Patronus Evaluation API
+    request should pass, especially when multiple evaluators
+    are called in a single request.
+    ALL_PASS requires all evaluators to pass for success.
+    ANY_PASS requires only one evaluator to pass for success.
+    """
+
+    ALL_PASS = "all_pass"
+    ANY_PASS = "any_pass"
+
+
+class PatronusEvaluateApiParams(BaseModel):
+    """Config to parameterize the Patronus Evaluate API call"""
+
+    success_strategy: Optional[PatronusEvaluationSuccessStrategy] = Field(
+        default=PatronusEvaluationSuccessStrategy.ALL_PASS,
+        description="Strategy to determine whether the Patronus Evaluate API Guardrail passes or not.",
+    )
+    params: Dict[str, Any] = Field(
+        default_factory=dict,
+        description="Parameters to the Patronus Evaluate API",
+    )
+
+
+class PatronusEvaluateConfig(BaseModel):
+    """Config for the Patronus Evaluate API call"""
+
+    evaluate_config: PatronusEvaluateApiParams = Field(
+        default_factory=PatronusEvaluateApiParams,
+        description="Configuration passed to the Patronus Evaluate API",
+    )
+
+
+class PatronusRailConfig(BaseModel):
+    """Configuration data for the Patronus Evaluate API"""
+
+    input: Optional[PatronusEvaluateConfig] = Field(
+        default_factory=PatronusEvaluateConfig,
+        description="Patronus Evaluate API configuration for an Input Guardrail",
+    )
+    output: Optional[PatronusEvaluateConfig] = Field(
+        default_factory=PatronusEvaluateConfig,
+        description="Patronus Evaluate API configuration for an Output Guardrail",
+    )
+
+
 class RailsConfigData(BaseModel):
     """Configuration data for specific rails that are supported out-of-the-box."""
 
@@ -405,6 +454,11 @@ class RailsConfigData(BaseModel):
         description="Configuration data for the AutoAlign guardrails API.",
     )
 
+    patronus: Optional[PatronusRailConfig] = Field(
+        default_factory=PatronusRailConfig,
+        description="Configuration data for the Patronus Evaluate API.",
+    )
+
     sensitive_data_detection: Optional[SensitiveDataDetection] = Field(
         default_factory=SensitiveDataDetection,
         description="Configuration for detecting sensitive data.",
diff --git a/tests/test_patronus_evaluate_api.py b/tests/test_patronus_evaluate_api.py
