Skip to content

Commit

Permalink
Add beam search visualizer to the doc
Browse files Browse the repository at this point in the history
  • Loading branch information
aymeric-roucher committed Mar 27, 2024
1 parent 7628b3a commit 5735722
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion docs/source/en/generation_strategies.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ to stop generation whenever the full generation exceeds some amount of time. To
- `num_beams`: by specifying a number of beams higher than 1, you are effectively switching from greedy search to
beam search. This strategy evaluates several hypotheses at each time step and eventually chooses the hypothesis that
has the overall highest probability for the entire sequence. This has the advantage of identifying high-probability
sequences that start with a lower probability initial tokens and would've been ignored by the greedy search.
sequences that start with a lower probability initial tokens and would've been ignored by the greedy search. Visualize how it works [here](https://huggingface.co/spaces/m-ric/beam_search_visualizer).
- `do_sample`: if set to `True`, this parameter enables decoding strategies such as multinomial sampling, beam-search
multinomial sampling, Top-K sampling and Top-p sampling. All these strategies select the next token from the probability
distribution over the entire vocabulary with various strategy-specific adjustments.
Expand Down Expand Up @@ -254,6 +254,13 @@ Unlike greedy search, beam-search decoding keeps several hypotheses at each time
the hypothesis that has the overall highest probability for the entire sequence. This has the advantage of identifying high-probability
sequences that start with lower probability initial tokens and would've been ignored by the greedy search.

<a herf="https://huggingface.co/spaces/m-ric/beam_search_visualizer" class="flex flex-col justify-center">
<img style="max-width: 90%; margin: auto;" src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/beam_search.png"/>
<figcaption class="text-center">Beam-search exploring decoding paths from a short sentence.</figcaption>
</a>

You can visualize how beam-search decoding works in [this interactive demo](https://huggingface.co/spaces/m-ric/beam_search_visualizer): type your input sentence, and play with the parameters to see how the decoding beams change.

To enable this decoding strategy, specify the `num_beams` (aka number of hypotheses to keep track of) that is greater than 1.

```python
Expand Down

0 comments on commit 5735722

Please sign in to comment.