Skip to content

Commit b7fb393

Browse files
authored
Docs: alert for the possibility of manipulating logits (#32467)
* logits * words
1 parent b640103 commit b7fb393

File tree

1 file changed

+12
-1
lines changed

1 file changed

+12
-1
lines changed

docs/source/en/generation_strategies.md

+12-1
Original file line numberDiff line numberDiff line change
@@ -225,10 +225,21 @@ array([True, True])
225225
## Decoding strategies
226226

227227
Certain combinations of the `generate()` parameters, and ultimately `generation_config`, can be used to enable specific
228-
decoding strategies. If you are new to this concept, we recommend reading [this blog post that illustrates how common decoding strategies work](https://huggingface.co/blog/how-to-generate).
228+
decoding strategies. If you are new to this concept, we recommend reading
229+
[this blog post that illustrates how common decoding strategies work](https://huggingface.co/blog/how-to-generate).
229230

230231
Here, we'll show some of the parameters that control the decoding strategies and illustrate how you can use them.
231232

233+
<Tip>
234+
235+
Selecting a given decoding strategy is not the only way you can influence the outcome of `generate()` with your model.
236+
The decoding strategies act based (mostly) on the logits, the distribution of probabilities for the next token, and
237+
thus selecting a good logits manipulation strategy can go a long way! In other words, manipulating the logits is another
238+
dimension you can act upon, in addition to selecting a decoding strategy. Popular logits manipulation strategies include
239+
`top_p`, `min_p`, and `repetition_penalty` -- you can check the full list in the [`GenerationConfig`] class.
240+
241+
</Tip>
242+
232243
### Greedy Search
233244

234245
[`generate`] uses greedy search decoding by default so you don't have to pass any parameters to enable it. This means the parameters `num_beams` is set to 1 and `do_sample=False`.

0 commit comments

Comments
 (0)