0.1.16

What's Changed

• Allow generating arbitrary (schemaless) JSON by @hudson-ai in #892
• [Bug] Fix Llama3ChatTemplate by @jjkjkj in #884
• Use python's re module instead of pyformlang to build regex ASTs by @hudson-ai in #854
• [Feature] Add Support for Phi-3-small by @ambisinister in #935
• [Feature] Constant elements in JSON schemas by @wjn0 in #886
• [Bug] Fix for LlamaCpp tokeniser prepending spaces by @riedgar-ms in #903
• New Benchmarking system by @nopdive in #843
• Mistral chat template integrated by @Harsha-Nori in #919
• [Bug] Passing through kwargs to TransformersTokenizer by @riedgar-ms in #922
• [Feature] Add extra support to JSON string schema by @riedgar-ms in #927
• Update protobuf definitions for remote list_append fix by @nking-1 in #937
• Downgrade protobuf required version using protoc 25.3 by @nking-1 in #946
• [Bug] Fix regex for urls in image function by @kklemon in #928
• Dedent fixes for Python 3.12 by @Harsha-Nori in #959
• Refector TransformersTokenizer and change fallback behavior for byte_tokens by @hudson-ai in #973
• Fix Transformer models with bos_token null by @Mihaiii in #990

New Contributors

• @nopdive made their first contribution in #843
• @jjkjkj made their first contribution in #884
• @wjn0 made their first contribution in #886
• @kklemon made their first contribution in #928
• @ambisinister made their first contribution in #935
• @Mihaiii made their first contribution in #990

Full Changelog: 0.1.15...0.1.16