-
Notifications
You must be signed in to change notification settings - Fork 321
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rethink usage pattern for pretrained models #200
Comments
yeah, agree... although this is basically just the same as load_from_checkpoint no? sounds like we're looking for checkpoint nicknames instead? doesn't it read better as:
|
Right, I think the distinction here is that So, yes! We are looking for something that can point to a nickname/identifier for a pretrained model. I think 'pretrained_on' is a limiting name, as a model could be pretrained on the same dataset twice w/ different settings, and then would be ambiguous to load if using that function name. Thats why I suggest something a little more open, such as This is just my opinion... I could be convinced otherwise haha 😄 . Let's have others weigh in to come to consensus. CC: @PyTorchLightning/core-contributors |
oh i see. it's an id not a dataset. for instance we can have many backbones with different datasets as well
|
Yes, they are trained on a defined dataset, in this case, the dataset name serves just as Look-up-table to a specific path on PL side... |
@williamFalcon @Borda @nateraw I included this pattern in the latest AE, VAE commits to bolts. Few points that I realized:
In this example stl10 weights have a different configuration for the encoder of the VAE. But, at the same time the internal method has a
I have added all of this + tests for the AE and VAE classes I have updated for bolts. |
🚀 Feature
Switch to using
SomeModel.from_pretrained('pretrained-model-name')
for pretrained models.Motivation
Seems we are following torchvision's pattern of having a 'pretrained' argument in the init of our models to initialize a pretrained model. In my opinion, this is extremely confusing as it makes the other init args + kwargs ambiguous/useless.
Pitch
add
.from_pretrained
classmethod to models and initialize an instance of the class based off of that. Pretrained models should incorporate any hparams needed to fill out init, I guess.Alternatives
Additional context
The text was updated successfully, but these errors were encountered: