Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add TimmBackbone model #22619

Merged
merged 33 commits into from
Jun 6, 2023

Conversation

amyeroberts
Copy link
Collaborator

@amyeroberts amyeroberts commented Apr 6, 2023

What does this PR do?

Adds a new model TimmBackbone to use for loading timm weights for use in the AutoBackbone API.

Example usage:

from transformers import AutoBackbone

# Loads a transformers model 
backbone = AutoBackbone.from_pretrained("microsoft/resnet-18")

# Loads a timm checkpoint
backbone = AutoBackbone.from_pretrained("resnet18", use_timm_backbone=True)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline,
    Pull Request section?
  • Was this discussed/approved via a Github issue or the forum? Please add a link
    to it if that's the case.
  • Did you make sure to update the documentation with your changes? Here are the
    documentation guidelines, and
    here are tips on formatting docstrings.
  • Did you write any new necessary tests?

@HuggingFaceDocBuilderDev
Copy link

HuggingFaceDocBuilderDev commented Apr 6, 2023

The documentation is not available anymore as the PR was closed or merged.

@huggingface huggingface deleted a comment from github-actions bot May 9, 2023
@amyeroberts amyeroberts force-pushed the add-timm-backbone-model branch 2 times, most recently from 37cd88d to 345672d Compare May 11, 2023 12:00
@amyeroberts amyeroberts changed the title Add timm backbone model Add TimmBackbone model May 11, 2023
@amyeroberts amyeroberts marked this pull request as ready for review May 11, 2023 18:05
@@ -445,6 +445,7 @@ def job_name(self):
"pip install -e .[dev]",
"pip install git+https://github.com/huggingface/accelerate",
"pip install --upgrade pytest pytest-sugar",
"pip install natten",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needed for Dinat and Nat doc tests

@amyeroberts amyeroberts requested review from sgugger and ydshieh May 11, 2023 18:06
Copy link
Collaborator

@sgugger sgugger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot for your PR! Just a couple of nits here and there but it looks great!

src/transformers/models/auto/auto_factory.py Outdated Show resolved Hide resolved
src/transformers/models/auto/auto_factory.py Outdated Show resolved Hide resolved

def __init__(self, config, **kwargs):
requires_backends(self, "timm")
import timm
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be cleaner to do this at the top of the file under an is_timm_available() but no strong opinions.

@ydshieh ydshieh self-assigned this May 12, 2023
@amyeroberts amyeroberts force-pushed the add-timm-backbone-model branch from 0745535 to 8316f23 Compare June 2, 2023 11:43
amyeroberts and others added 2 commits June 2, 2023 12:58
Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
@amyeroberts
Copy link
Collaborator Author

@ydshieh Merging now. I'll address any comments or requests for changes in a follow up PR.

@amyeroberts amyeroberts merged commit a717e03 into huggingface:main Jun 6, 2023
@amyeroberts amyeroberts deleted the add-timm-backbone-model branch June 6, 2023 16:11
novice03 pushed a commit to novice03/transformers that referenced this pull request Jun 23, 2023
* Add test_backbone for convnext

* Add TimmBackbone model

* Add check for backbone type

* Tidying up - config checks

* Update convnextv2

* Tidy up

* Fix indices & clearer comment

* Exceptions for config checks

* Correclty update config for tests

* Safer imports

* Safer safer imports

* Fix where decorators go

* Update import logic and backbone tests

* More import fixes

* Fixup

* Only import all_models if torch available

* Fix kwarg updates in from_pretrained & main rebase

* Tidy up

* Add tests for AutoBackbone

* Tidy up

* Fix import error

* Fix up

* Install nattan in doc_test_job

* Revert back to setting self._out_xxx directly

* Bug fix - out_indices mapping from out_features

* Fix tests

* Dont accept output_loading_info for Timm models

* Set out_xxx and don't remap

* Use smaller checkpoint for test

* Don't remap timm indices - check out_indices based on stage names

* Skip test as it's n/a

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Cleaner imports / spelling is hard

---------

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants