Skip to content

Commit

Permalink
readme : fix the description of the Tail free sampling (TFS) method (#…
Browse files Browse the repository at this point in the history
  • Loading branch information
WeirdConstructor authored Jul 28, 2023
1 parent 65cdf34 commit d91f3f0
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions examples/main/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,9 +202,9 @@ Example usage: `--top-p 0.95`

- `--tfs N`: Enable tail free sampling with parameter z (default: 1.0, 1.0 = disabled).

Tail free sampling (TFS) is a text generation technique that aims to reduce the impact of less likely tokens, which may be less relevant, less coherent, or nonsensical, on the output. The method adjusts the logits (token probabilities) by raising them to the power of the parameter z. A higher value of z (e.g., 2.0) will further suppress less likely tokens from the tail of the distribution, while a value of 1.0 disables the effect of TFS. By setting the parameter z, you can control how much the probabilities of less likely tokens are reduced.
Tail free sampling (TFS) is a text generation technique that aims to reduce the impact of less likely tokens, which may be less relevant, less coherent, or nonsensical, on the output. Similar to Top-P it tries to determine the bulk of the most likely tokens dynamically. But TFS filters out logits based on the second derivative of their probabilities. Adding tokens is stopped after the sum of the second derivatives reaches the parameter z. In short: TFS looks how quickly the probabilities of the tokens decrease and cuts off the tail of unlikely tokens using the parameter z. Typical values for z are in the range of 0.9 to 0.95. A value of 1.0 would include all tokens, and thus disables the effect of TFS.

Example usage: `--tfs 2.0`
Example usage: `--tfs 0.95`

### Locally Typical Sampling

Expand Down

0 comments on commit d91f3f0

Please sign in to comment.