Skip to content

Python: v0.2.4 - Update to latest tokenizer crates
Compare
Choose a tag to compare
@benbrandt benbrandt released this 15 Nov 15:21
· 519 commits to main since this release
python-v0.2.4
58169b1
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

What's Changed

  • Update to v0.4.5 of text-splitter to support tokenizers@0.15.0
  • Update tokenizers and tiktoken-rs to latest version
Assets 2
Loading
0 Join discussion