Don't download model weights and load imagenet weights if using models for inference

[pjbull]

We currently use load_from_checkpoint in our ModelManager to initialize models when doing inference. This can cause the models to download the pretrained imagenet weights from the internet even thought we don't need those. To address this, we need a parameter that we pass in to the __init__ of the model to indicate we are doing inference/loading from a checkpoint, and then we need to pass this paramter to load_from_checkpoint in the ModelManager.

We should check across all of our models for this behavior, but this is how it works for the time_distributed model:

• Here is where we need to pass a parameter indicating that we're doing inference:
  

  zamba/zamba/models/model_manager.py

  Lines 98 to 101 in 7fb2a0f

  	if labels is None:
  	# predict; load from checkpoint uses associated hparams
  	logger.info("Loading from checkpoint.")
  	return model_class.load_from_checkpoint(checkpoint_path=checkpoint)

• Here is where we need to use that param to skip intializing from timm:
  

  zamba/zamba/models/efficientnet_models.py

  Lines 23 to 27 in 7fb2a0f

  	if finetune_from is None:
  	efficientnet = timm.create_model("efficientnetv2_rw_m", pretrained=True)
  	efficientnet.classifier = nn.Identity()
  	else:
  	efficientnet = self.load_from_checkpoint(finetune_from).base.module

[ejm714]

This also arises if we are training a model that has labels which are a subset of the zamba labels, which means we "resume training" instead of replacing the head. This stems from the fact that finetune_from is still None in this case; we should instead do model_class(finetune_from={official_ckpt}) rather than load from checkpoint

zamba/zamba/models/model_manager.py

Lines 139 to 147 in 7986c41

	elif is_subset:
	logger.info(
	"Provided species fully overlap with Zamba species. Resuming training from latest checkpoint."
	)
	# update in case we want to resume with different scheduler
	if scheduler_config != "default":
	hparams.update(scheduler_config.dict())
	model = model_class.load_from_checkpoint(checkpoint_path=checkpoint, **hparams)

In addition: we may want super().load_from_checkpoint instead here to avoid re-passing through the init with the timm weight download:

zamba/zamba/models/efficientnet_models.py

Line 27 in 7fb2a0f

efficientnet = self.load_from_checkpoint(finetune_from).base.module

[papapizzachess]

The code has changed a lot. Has this bug been resolved? I'm trying to work on this if it's not resolved.

[ejm714]

@papapizzachess yes this bug still exists and the code sections in the issue description are still correct.