Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QoL: Default to using fast tokenizer for Llama models #3625

Merged
merged 2 commits into from
Sep 18, 2023

Conversation

arnavgarg1
Copy link
Contributor

Quality of life improvement - we used to set use_fast to False since the tokenizer was not natively written so it would take 2-4 minutes to load, however in newer transformers versions there is first class support so we no longer have to do this.

@github-actions
Copy link

Unit Test Results

  4 files  ±0    4 suites  ±0   31m 57s ⏱️ + 1m 59s
31 tests ±0  26 ✔️ ±0    5 💤 ±0  0 ±0 
62 runs  ±0  52 ✔️ ±0  10 💤 ±0  0 ±0 

Results for commit 14b3534. ± Comparison against base commit 42723e3.

Copy link
Contributor

@justinxzhao justinxzhao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for following up on this! I was wondering about this and why setting use_fast=True didn't seem faster.

@arnavgarg1 arnavgarg1 merged commit 69c2c0b into master Sep 18, 2023
@arnavgarg1 arnavgarg1 deleted the llama-tokenizer branch September 18, 2023 17:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants