-
Notifications
You must be signed in to change notification settings - Fork 555
Feat: Support Models with Reasoning Traces #996
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…variable is not defined
Documentation preview |
|
@mikemckiernan , if you have any feedback about additional documentation changes (including examples). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good, though I'd appreciate a second reviewer as well. I expect we won't have big issues since you've done plenty of "adapt to this LLM" before.
|
|
||
|
|
||
| @pytest.mark.asyncio | ||
| async def test_1(rails_config): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| async def test_1(rails_config): | |
| async def test_reasoning_traces_removed_from_responses(rails_config): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thank you Traian!
Description
Add support for reasoning models that output reasoning traces in the response (e.g. DeepSeek-R1).
Current implementation strips out the reasoning traces in the output parser for each executed Guardrails task, but different output parsers can be added to use the information in the reasoning trace as well (e.g. implement a custom rail that analyzes the reasoning traces).
Sample model config for DeepSeek-R1 is:
Using other platforms for serving the model, such as NVIDIA AI Endpoints, is supported as long as the model name contains
deepseekin the name.MR includes:
deepseekandgoogle-genaias LLM providersRelated Issue(s)
Checklist