Update README to include chat templating

[cpfiffer]

The existing README has underwhelming or incorrect results (Example is underwhelming #1347) due to lack of templating for instruct models.

This adds special tokens for each instruct model call, as well as provide comments on how to obtain/produce special tokens.

[torymur]

Seems this part also needs to be adjusted, but maybe we can show only the difference:

from enum import Enum

class Food(str, Enum):
      pizza = "Pizza"
      pasta = "Pasta"
      salad = "Salad"
      dessert = "Dessert"

...

generator = outlines.generate.choice(model, Food)

[cpfiffer]

IMO every example in a README should be completely copy-able with no slice-and-dice on the user's part. This is of course personal preference, so up to y'all.

[yvan-sraka]

If one day we generate documentation with mdbook, it offers a feature for hiding code lines from the user. I realized there isn't a similar feature in GitHub Flavored Markdown (yet?). The closest thing I'm aware of is collapsed sections...

[torymur]

@yvan-sraka yeah, sadly collapse sections doesn't work in the code snippets. Would be nice if visually repetitive code would be hidden by collapsing, but then it would copy everything including the hidden code, but without tags. This way "copy-paste and it works" magic would not be lost and "what you see is what you get" predictability will also be served.

In mdbook hiding code lines nicely works for running doc tests for example, but on "copying the code" side it still copies just what's visible, which might not work as it is without hidden code parts. Kind of on the same side of things there is also html comments, but also completely not helpful in copying the code snippets.

@cpfiffer fair point!

But this enum example still needs to be updated, considering that we're just showing different ways of multiple choice, maybe we can just extend original section with Enum in the first place and list as an alternative, wdyt?:

import outlines
from enum import Enum

model_name = "HuggingFaceTB/SmolLM2-360M-Instruct"
model = outlines.models.transformers(model_name)

# You must apply the chat template tokens to the prompt!
# See below for an example.
prompt = """
<|im_start|>system
You extract information from text.
<|im_end|>

<|im_start|>user
What food does the following text describe?

Text: I really really really want pizza.
<|im_end|>
<|im_start|>assistant
"""

class Food(str, Enum):
      pizza = "Pizza"
      pasta = "Pasta"
      salad = "Salad"
      dessert = "Dessert"

generator = outlines.generate.choice(model, Food)

# You can also pass these choices simply as list:
# generator = outlines.generate.choice(model, ["Pizza", "Pasta", "Salad", "Dessert"])

answer = generator(prompt)
# Likely answer: Pizza

[cpfiffer]

Awesome, I like the updated example. I'll add it. In general it sounds like we'll need to just overhaul the docs, which I suspect is probably more fruitful after the 1.0 release.

[cpfiffer]

I opted into collapsing these into one section as @torymur suggested. The second code block in this section is not copy-pastable, but I think the arrangement of code blocks makes it clear that the Enum variant is an extension of the first block.

[yvan-sraka]

Rendered

[cpfiffer]

Great, appreciate the comments!

[yvan-sraka]

This overall LGTM. I like that we provide a lot of examples in the README.md and understand why the Positive/Negative one felt counterintuitive... That said, I found most of the examples a bit abstract/useless and would prefer more concrete, real-world use cases!

The example that probably fits this category best is character generation, though it would be even more interesting if one of the character generation fields was an unconstrained (or less constrained) string, something like a description field (eventually with a limit, e.g., less than 500 characters of lore)! Otherwise, I think it could be nice to hide internet culture jokes in example because, hey, we’re cool kids! Ofc, these comments are for future improvement, the PR is already great and ready to be merged as is!

[rlouf]

We can leave the warning but this section should be in the documentation and we can add a link from here.

[rlouf]

I would go as far as having this warning before the first example instead of having a separate section.

[cpfiffer]

Agreed. I'm not sure what the best place is for it. We could add it to the prompting reference page or a new page. Not sure which is better. It wouldn't be too hard to cook up a new page for the chat templating issue, as we can provide lots of little bits of context.

[torymur]

My 5 cents here is that since chat templating issue kind of grows all over the examples and at the same time has nothing to do with structure-generating showcases it could be properly stated in the description before all the sections, and considering how many nuances could be there, probably, deserves its own page with a reference from the description.

[cpfiffer]

I moved the chat template tips to the top of the README, and added a specific page in the docs for it.

[torymur]

LGTM!