You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/user-guides/configuration-guide.md
+29-6Lines changed: 29 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -142,16 +142,39 @@ models:
142
142
model: nvidia/llama-3.1-nemotron-ultra-253b-v1
143
143
reasoning_config:
144
144
remove_reasoning_traces: True
145
+
146
+
rails:
147
+
output:
148
+
apply_to_reasoning_traces: False
145
149
```
146
150
147
-
The `reasoning_config` field for a model specifies the required configuration for a reasoning model that returns reasoning traces.
148
-
By removing the traces, the guardrails runtime processes only the actual responses from the LLM.
151
+
```{list-table}
152
+
:header-rows: 1
153
+
154
+
* - Field
155
+
- Description
156
+
- Default Value
157
+
158
+
* - `reasoning_config.remove_reasoning_traces`
159
+
- When set to `True`, reasoning traces are omitted from internal tasks.
160
+
- `True`
161
+
162
+
* - `reasoning_config.start_token`
163
+
- Specifies the start token for the reasoning trace.
164
+
- `<think>`
149
165
150
-
You can specify the following parameters for a reasoning model:
166
+
* - `reasoning_config.end_token`
167
+
- Specifies the end token for the reasoning trace.
168
+
- `</think>`
151
169
152
-
-`remove_reasoning_traces`: if the reasoning traces should be ignored (default `True`).
153
-
-`start_token`: the start token for the reasoning process (default `<think>`).
154
-
-`end_token`: the end token for the reasoning process (default `</think>`).
170
+
* - `rails.output.apply_to_reasoning_traces`
171
+
- When set to `True`, output rails are applied to the reasoning traces and the model response.
172
+
By default, output rails are applied to the text of the model response only.
173
+
- `False`
174
+
```
175
+
176
+
The `reasoning_config` field for a model specifies the required configuration for a reasoning model that returns reasoning traces.
177
+
By removing the traces, the guardrails runtime processes only the actual responses from the LLM.
155
178
156
179
Even if `remove_reasoning_traces` is set to `True`, end users can still receive the thinking traces from the Nemotron models by requesting the detailed thinking, as shown in the following example:
0 commit comments