When selecting, ensure the last token is the option encoded

[Mihaiii]

This prevents the case when we would have a token that represents a text that only ends with the option (ex: both " True" and "True" are in vocab).

Fixes #67

This feels like the right solution, but I'm only trying to learn the codebase. The codebase seem to assume that options can only be 1 token long (for example, the call to the LLM has max_tokens=1) - any plans to support more tokens?

[slundberg]

Thanks! Will take a look. The code does not assume a single token, but it does make single token generation calls to branch the decision tree that is implied by a select statement.

[Mihaiii]

Closing as the issue can't be replicated in the latest release.