Skip to content

Commit 97b4cc4

Browse files
committed
Add rails.output.apply_to_reasoning_traces
Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>
1 parent 89935d3 commit 97b4cc4

File tree

1 file changed

+29
-6
lines changed

1 file changed

+29
-6
lines changed

docs/user-guides/configuration-guide.md

Lines changed: 29 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -142,16 +142,39 @@ models:
142142
model: nvidia/llama-3.1-nemotron-ultra-253b-v1
143143
reasoning_config:
144144
remove_reasoning_traces: True
145+
146+
rails:
147+
output:
148+
apply_to_reasoning_traces: False
145149
```
146150

147-
The `reasoning_config` field for a model specifies the required configuration for a reasoning model that returns reasoning traces.
148-
By removing the traces, the guardrails runtime processes only the actual responses from the LLM.
151+
```{list-table}
152+
:header-rows: 1
153+
154+
* - Field
155+
- Description
156+
- Default Value
157+
158+
* - `reasoning_config.remove_reasoning_traces`
159+
- When set to `True`, reasoning traces are omitted from internal tasks.
160+
- `True`
161+
162+
* - `reasoning_config.start_token`
163+
- Specifies the start token for the reasoning trace.
164+
- `<think>`
149165
150-
You can specify the following parameters for a reasoning model:
166+
* - `reasoning_config.end_token`
167+
- Specifies the end token for the reasoning trace.
168+
- `</think>`
151169
152-
- `remove_reasoning_traces`: if the reasoning traces should be ignored (default `True`).
153-
- `start_token`: the start token for the reasoning process (default `<think>`).
154-
- `end_token`: the end token for the reasoning process (default `</think>`).
170+
* - `rails.output.apply_to_reasoning_traces`
171+
- When set to `True`, output rails are applied to the reasoning traces and the model response.
172+
By default, output rails are applied to the text of the model response only.
173+
- `False`
174+
```
175+
176+
The `reasoning_config` field for a model specifies the required configuration for a reasoning model that returns reasoning traces.
177+
By removing the traces, the guardrails runtime processes only the actual responses from the LLM.
155178

156179
Even if `remove_reasoning_traces` is set to `True`, end users can still receive the thinking traces from the Nemotron models by requesting the detailed thinking, as shown in the following example:
157180

0 commit comments

Comments
 (0)