Fashion MNIST using improve unified interface

[rajeeja]

Demo example for new community models.
This is a first draft that uses IMPROVE for initialize parameters and candle for checkpointing.

[wilke]

Why are these parameters for inference?

[adpartin]

@rajeeja I agree with @wilke, it's weird that these parameters are defined in inference script. Generally, inference is done without knowledge of training settings (e.g., train batch size, learning rate optimizer, etc.). Is there a reason why these are defined here?

[rajeeja]

@adpartin @wilke

1. we need to load the test data in batches -> Hence - batch_size (see line 50)
2. I'm getting the model from the ckpt method, we can get it via other means also. The ckpt object needs optimizer to instantiate -> Hence - optimizer (see line 64)

[wilke]

Sorry, but it is unclear why you must load the data for inference in batches. Is this in any way faster than a simple

for v in File
  label=infer(v)

[wilke]

There should be a better way of loading the model. We want the model from the input directory. No optimizer is needed. If this is a problem, I suggest writing a load_model_weights function as a wrapper.

[wilke]

@rajeeja @adpartin If we use this as an example, we have to make it clean. This is a great hack but not a sustainable solution. Please come up with a better solution or hide it in a function call. These are constants in this case.

[wilke]

Loading test data for inference? I would expect to load the model weights.

[rajeeja]

See line 72, how are you going to get images to test?
outputs = model(images)

[wilke]

I don't understand, or is this a problem of naming conventions? We are doing label prediction in this script no testing. Do you have a specific use case in mind?

[wilke]

@rajeeja Any thoughts?

[wilke]

Same here

[rajeeja]

same as above :) we can get the model weights in some other fashion, do you know how else to get the model weights to perform inferece? - where optimizer or learning rate is not needed, it is a minor thing and can be ignored, IMO. The overall logic is to get the model weights and infer on what was done in the training step.

[wilke]

I agree, get the model weights and infer on any input data in the input data directory. The model weights should be located there as well. The outputs of training are model weights and learning metrics.

[wilke]

Which parameter is for loading model weights?

[rajeeja]

I'm loading it from the ckpt files, so no specific parameter.

this can be done by using a specific directory to save the model weights and load from there, something along the lines of test_ml_data_dir

[wilke]

If this is for data preprocessing, please move it to the preprocessing script.

[rajeeja]

trains on 60k images, but there are 10k images we need to infer on, those we have to get here. This is load not the entire preprocessing.

[wilke]

I don't see where you export the final weights.

[rajeeja]

ckpt, does that for you, if you run it once, you will see it in the directory.
see ckpt_epoch line 94 and documentation here: https://candle-lib.readthedocs.io/en/latest/api_ckpt_pytorch_utils/_autosummary/candle.ckpt_pytorch_utils.CandleCkptPyTorch.html

[wilke]

The final weights should be in the top-level output directory.

[adpartin]

@rajeeja @wilke The code structure doesn't follow the IMPROVE structure, so it's hard to go through the code and fix everything. https://jdacs4c-improve.github.io/docs/content/unified_interface.html

[rajeeja]

@rajeeja @wilke The code structure doesn't follow the IMPROVE structure, so it's hard to go through the code and fix everything. https://jdacs4c-improve.github.io/docs/content/unified_interface.html

That link is not helpful. The data is different. This model doesn't use genetics or drug data, so, the structure won't be exactly the same. The .py files are very simple and easy to understand. Also the notebook is very standard fashion-mnist.