🚨🚨🚨 Fully remove Tensorflow and Jax support library-wide #40760
Merged
+3,792
−181,269
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Cyrilvallez
force-pushed
the
the-great-cleaning
branch
from
475c567 to
9190046
Compare
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Cyrilvallez
force-pushed
the
the-great-cleaning
branch
from
f71cedb to
1ecdc29
Compare
Cyrilvallez
force-pushed
the
the-great-cleaning
branch
from
8706485 to
b00b07e
Compare
This was referenced Sep 15, 2025
Cyrilvallez
force-pushed
the
the-great-cleaning
branch
from
7120b37 to
4069e64
Compare
Cyrilvallez
changed the title
This was referenced Sep 17, 2025
ArthurZucker
approved these changes
Sep 18, 2025
src/transformers/models/albert/convert_albert_original_tf_checkpoint_to_pytorch.py
Show resolved
Hide resolved
src/transformers/models/bert/convert_bert_original_tf_checkpoint_to_pytorch.py
Show resolved
Hide resolved
| @@ -181,7 +174,7 @@ def _sanitize_parameters( | |||
| preprocess_params["prefix"] = prefix | |||
| if prefix: | |||
| prefix_inputs = self.tokenizer( | |||
| prefix, padding=False, add_special_tokens=add_special_tokens, return_tensors=self.framework | |||
| prefix, padding=False, add_special_tokens=add_special_tokens, return_tensors="pt" | |||
There was a problem hiding this comment.
i understand why you wanted it to return pt by default/ We could also have a "bool" return_tensors=True to return pt tensors
There was a problem hiding this comment.
In all the pipelines, self.framework would be set to pt for torch models during init. So I simply removed self.framework and made it explicit!
There was a problem hiding this comment.
no no I mean because you have now to manually say "pt" I understand why you told me you wanna default to returning tensors in tokenizer haha
Cyrilvallez
force-pushed
the
the-great-cleaning
branch
from
4069e64 to
9a41b8d
Compare
vijayabhaskar-ev
pushed a commit
to vijayabhaskar-ev/transformers
that referenced
this pull request
Oct 2, 2025
…#40760) * setup * start the purge * continue the purge * more and more * more * continue the quest: remove loading tf/jax checkpoints * style * fix configs * oups forgot conflict * continue * still grinding * always more * in tje zone * never stop * should fix doc * fic * fix * fix * fix tests * still tests * fix non-deterministic * style * remove last rebase issues * onnx configs * still on the grind * always more references * nearly the end * could it really be the end? * small fix * add converters back * post rebase * latest qwen * add back all converters * explicitly add functions in converters * re-add
yuchenxie4645
pushed a commit
to yuchenxie4645/transformers
that referenced
this pull request
Oct 4, 2025
…#40760) * setup * start the purge * continue the purge * more and more * more * continue the quest: remove loading tf/jax checkpoints * style * fix configs * oups forgot conflict * continue * still grinding * always more * in tje zone * never stop * should fix doc * fic * fix * fix * fix tests * still tests * fix non-deterministic * style * remove last rebase issues * onnx configs * still on the grind * always more references * nearly the end * could it really be the end? * small fix * add converters back * post rebase * latest qwen * add back all converters * explicitly add functions in converters * re-add
Open
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Apart from obvious tf/jax support, I believe the following should be the only potential breaking changes to torch-only code:
frameworkargument anymoreframeworkargument anymoreIt may break current torch code if users do
framework="pt"explicitly, but it's a necessary change. It makes no sense to keep those arguments, as the only framework working for those objects is nowtorch. Would be weird to keep it only for BC, as we are breaking the support anyway.Note: I did not remove traces of tensorflow/jax in docs
.md(markdown) files for now, as this PR is already enormous. It's a very tedious task, and moreover a lot of doc is written in another alphabet that I cannot read at all. Will be done in a subsequent PR, hopefully with the help of AI (should be a perfect fit for that)