You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: docs/source/en/generation_strategies.md
+12-1
Original file line number
Diff line number
Diff line change
@@ -225,10 +225,21 @@ array([True, True])
225
225
## Decoding strategies
226
226
227
227
Certain combinations of the `generate()` parameters, and ultimately `generation_config`, can be used to enable specific
228
-
decoding strategies. If you are new to this concept, we recommend reading [this blog post that illustrates how common decoding strategies work](https://huggingface.co/blog/how-to-generate).
228
+
decoding strategies. If you are new to this concept, we recommend reading
229
+
[this blog post that illustrates how common decoding strategies work](https://huggingface.co/blog/how-to-generate).
229
230
230
231
Here, we'll show some of the parameters that control the decoding strategies and illustrate how you can use them.
231
232
233
+
<Tip>
234
+
235
+
Selecting a given decoding strategy is not the only way you can influence the outcome of `generate()` with your model.
236
+
The decoding strategies act based (mostly) on the logits, the distribution of probabilities for the next token, and
237
+
thus selecting a good logits manipulation strategy can go a long way! In other words, manipulating the logits is another
238
+
dimension you can act upon, in addition to selecting a decoding strategy. Popular logits manipulation strategies include
239
+
`top_p`, `min_p`, and `repetition_penalty` -- you can check the full list in the [`GenerationConfig`] class.
240
+
241
+
</Tip>
242
+
232
243
### Greedy Search
233
244
234
245
[`generate`] uses greedy search decoding by default so you don't have to pass any parameters to enable it. This means the parameters `num_beams` is set to 1 and `do_sample=False`.
0 commit comments