Default to fast image processors for all models #41388
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| if common_kwargs: | ||
| for kwarg in output_kwargs.values(): | ||
| kwarg.update(common_kwargs) | ||
|
|
There was a problem hiding this comment.
I'm sure there's a good reason but I'm missing it, why is this moved up?
There was a problem hiding this comment.
yep, it is a fix from #41381 :)
There was a problem hiding this comment.
aah, which fixes #40931, got it
There was a problem hiding this comment.
Yes mb, switch to a new branch without checking out main first 🥴
| def __init__(self, **kwargs): | ||
| super().__init__(**kwargs) | ||
| if not self.is_fast: | ||
| logger.warning_once( | ||
| f"Using a slow image processor (`{self.__class__.__name__}`). " | ||
| "As we are transitioning to fast (PyTorch-native) processors, consider using `AutoImageProcessor` or the model-specific fast image processor class " | ||
| "to instantiate a fast image processor." | ||
| ) |
There was a problem hiding this comment.
SGTM!
Related, since we're touching on the topic of "loading old models from the hub with new utils" this is related to the "from_pretrained conversion" @Cyrilvallez is working on, if we have modifications to apply to some old image processors, they should be in the from_pretrained as well to "convert" the processor in the same sense.
| class VideoLlavaProcessor(ProcessorMixin): | ||
| r""" | ||
| Constructs a VideoLlava processor which wraps a VideoLlava image processor and a Llava tokenizer into a single processor. | ||
| Constructs a VideoLlava processor which wraps a AutoImageProcessor and a Llava tokenizer into a single processor. |
There was a problem hiding this comment.
nit: imo we need not change the name when it is not referenced. Instead we only change the "[VideoLlavaImageProcessor] " one line below
There was a problem hiding this comment.
Yes you're right, not very useful to have AutoImageProcessor as in the docstring. I'll change these back. I'm also working on getting auto_docstring to work on processors which should do all that automatically (check which subprocessors are in auto for this model) ;)
There was a problem hiding this comment.
I'm also working on getting auto_docstring to work on processors which should do all that automaticall
nice, very needed
| if common_kwargs: | ||
| for kwarg in output_kwargs.values(): | ||
| kwarg.update(common_kwargs) | ||
|
|
There was a problem hiding this comment.
yep, it is a fix from #41381 :)
Good point for the lanczos sampling, I might add an exception for these, as the diffs are not close enough imo |
|
[For maintainers] Suggested jobs to run (before merge) run-slow: align, auto, blip, blip_2, bridgetower, chameleon, chinese_clip, clip, clipseg, emu3, flava, fuyu, grounding_dino, idefics, idefics2, idefics3 |
4 participants
What does this PR do?
Following the trial testing with Qwen_VL image processors, this extends defaulting to fast image processors even for checkpoints saved with a slow one to all models.
Also made sure that all processors use AutoImageProcessor to instantiate their image_processor_class.
On that point, defining default subclass in processors feels a bit redundant, as we basically already have that in auto classes. It would be nice to get rid of this for v5, wdyt @molbap @zucchini-nlp @ArthurZucker ?
I'll open a PR for that too.