Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chat template rendering extensions to match transformers #1486

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

CISC
Copy link
Contributor

@CISC CISC commented May 26, 2024

I've been using the jinja2 tojson filter in a couple of chat templates to render function calling instructions, which is working very well, however sometimes this will introduce escaped unicode characters, which is undesirable.

This PR simply changes the default parameters of tojson replaces tojson so that the JSON is rendered in unicode without escaped HTML characters.

Examples of models using tojson:

I've also removed the leftover (and unnecessary) loader parameter since I was touching this code area anyway.

Update:

Further changes to chat templates are being made in transformers and since they are dependant on the changes already made here I will simply append them to this PR:

  • Added loopcontrols jinja2 extension (adds break and continue)
  • Added strftime_now to render the current time

See huggingface/transformers#32684

@CISC
Copy link
Contributor Author

CISC commented Jun 20, 2024

Since the merging of a similar patch to transformers and the formalizing of tools (see references in #1336) there are starting to pop up a lot more models using tojson in their chat templates.

@CISC
Copy link
Contributor Author

CISC commented Aug 1, 2024

@abetlen Llama 3.1 is the latest model to use tojson, would be nice to have matching behaviour as transformers.

@CISC CISC changed the title Render chat template tojson filter as unicode Chat template rendering extensions to match transformers Aug 15, 2024
@CISC
Copy link
Contributor Author

CISC commented Aug 15, 2024

@abetlen Even more transformers changes incoming, see updated OP.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant