Current behavior:

We first randomly choose max number of tokens that we want to generate, and then try to find a response for the prompt in the dataset. If the response is longer than max tokens, we choose a random response (either from the dataset or preset).

Better behavior:

First find possible responses in the dataset for the prompt. We always want to use one of these responses. If all the possibilities are longer than max number of tokens, we can trim one of them.

If IgnoreEOS is true, and the response is shorter than max, we need to fill the rest with random tokens.