Add limit token filter docs #8159
Conversation
Signed-off-by: Anton Rubin <anton.rubin@eliatra.com>
Signed-off-by: Anton Rubin <anton.rubin@eliatra.com>
AntonEliatra
requested review from
kolchfa-aws,
Naarcha-AWS,
vagimeli,
AMoo-Miki,
natebower,
dlvenable,
stephen-crawford and
epugh
as code owners
|
Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged. Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer. When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review. |
|
@udabhas Will you review this PR for technical accuracy? Thank you. |
vagimeli
added
3 - Tech review
Signed-off-by: Anton Rubin <anton.rubin@eliatra.com>
Signed-off-by: AntonEliatra <anton.rubin@eliatra.com>
@udabhas @varun-lodaya This is a month old. We need tech review approval to move it forward in the documentation process. Please review this week or provide a peer who can review it. Thank you. |
Signed-off-by: Anton Rubin <anton.rubin@eliatra.com>
Signed-off-by: Fanit Kolchina <kolchfa@amazon.com>
kolchfa-aws
added
5 - Editorial review
| :--- | :--- | :--- | :--- | ||
| `max_token_count` | Optional | Integer | The maximum number of tokens to be generated. Default is `1`. | ||
| `consume_all_tokens` | Optional | Boolean | (Expect level setting) Use all tokens from tokenizer, even if result exceeds `max_token_count`. When this parameter is set, the output still only contains the number of tokens specified in `max_token_count`. However, all tokens generated by the tokenizer are processed. Default is `false`. | ||
|
|
There was a problem hiding this comment.
Line 19: What does "(Expect level setting)" mean?
There was a problem hiding this comment.
Expert 😄 Thanks for the catch.
There was a problem hiding this comment.
YW, but should be "Expert-level setting" in that case 😄
Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com>
* adding limit token filter docs Signed-off-by: Anton Rubin <anton.rubin@eliatra.com> * removing parameter which is not working #8153 Signed-off-by: Anton Rubin <anton.rubin@eliatra.com> * adding consume_all_tokens Signed-off-by: Anton Rubin <anton.rubin@eliatra.com> * Update limit.md Signed-off-by: AntonEliatra <anton.rubin@eliatra.com> * updating parameter table Signed-off-by: Anton Rubin <anton.rubin@eliatra.com> * Doc review Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> * Apply suggestions from code review Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _analyzers/token-filters/index.md Co-authored-by: Nathan Bower <nbower@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Anton Rubin <anton.rubin@eliatra.com> Signed-off-by: AntonEliatra <anton.rubin@eliatra.com> Signed-off-by: Fanit Kolchina <kolchfa@amazon.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina <kolchfa@amazon.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower <nbower@amazon.com> (cherry picked from commit 1026a84) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Description
Add limit token filter docs
Issues Resolved
Closes #8153
Version
all
Checklist
For more information on following Developer Certificate of Origin and signing off your commits, please check here.