Skip to content

Add sentencepiece to transformers optional dependencies #1796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
majiayu000 wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add sentencepiece to transformers optional dependencies #1796

majiayu000 wants to merge 1 commit into from

Conversation

@majiayu000
Copy link

@majiayu000 majiayu000 commented Jan 5, 2026

Summary

  • Add sentencepiece to the transformers optional dependencies in pyproject.toml
  • This fixes Undeclared dependency on sentencepiece #1475 where users installing outlines[transformers] encounter ModuleNotFoundError: No module named 'sentencepiece' when using models that rely on LlamaTokenizer

Changes

  • Modified pyproject.toml to include sentencepiece in the transformers optional dependency group

Test plan

  • Install outlines[transformers] and verify sentencepiece is included
  • Import LlamaTokenizer from transformers without additional installs

@majiayu000
Add sentencepiece to transformers optional dependencies
4e42d16 
When using Outlines with transformers for models that use LlamaTokenizer,
the sentencepiece library is required but was not declared as a dependency.
This caused runtime errors when users installed outlines[transformers].

Fixes dottxt-ai#1475

Signed-off-by: majiayu000 <1835304752@qq.com>
@RobinPicard
Copy link
Contributor

RobinPicard commented Jan 5, 2026

Hi @majiayu000! Thanks for opening a PR! I think we should also update the uv.lock such that it corresponds to the dependencies listed in the pyproject.toml.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Undeclared dependency on sentencepiece

2 participants

@majiayu000 @RobinPicard