Skip to content

Commit

Permalink
docs(call_hooks.md): add async_moderation_hook to docs
Browse files Browse the repository at this point in the history
  • Loading branch information
krrishdholakia committed Feb 17, 2024
1 parent 0cd2dee commit e3fab50
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion docs/my-website/docs/proxy/call_hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ curl --location 'http://0.0.0.0:8000/chat/completions' \
```


## *NEW* async_moderation_hook
## [BETA] *NEW* async_moderation_hook

Run a moderation check in parallel to the actual LLM API call.

Expand All @@ -89,6 +89,12 @@ In your Custom Handler add a new `async_moderation_hook` function
- If your `async_moderation_hook` raises an Exception, we will return that to the user.


:::info

We might need to update the function schema in the future, to support multiple endpoints (e.g. accept a call_type). Please keep that in mind, while trying this feature

:::

See a complete example with our [Llama Guard content moderation hook](https://github.com/BerriAI/litellm/blob/main/enterprise/hooks/llama_guard.py)

```python
Expand Down

0 comments on commit e3fab50

Please sign in to comment.