example request: load weight example for pruned model

[tabVersion]

What would you like to be added:

an example showing how to load weights for a pruned model

Why is this needed:

the design is really confusing

I use Pruner.export_model to save model weight and masks but I don't know how to assemble them for inference.

[tabVersion]

I tried the following way but got RuntimeError

dummy_input = torch.randn([1, 4, 3, 16, 16])
m_speedup = ModelSpeedup(device_model, dummy_input, f'all_edge_sparsity_0.05-mask.pth', torch.device('cpu'))
m_speedup.speedup_model()
model = m_speedup
model.load_state_dict(torch.load('models.all_edge_sparsity_0.05-prune.pth'))

[2021-04-13 17:21:51] INFO (nni.compression.pytorch.speedup.compressor/MainThread) start to speed up the model
[2021-04-13 17:21:51] INFO (nni.compression.pytorch.speedup.compressor/MainThread) fix the mask conflict of the interdependent layers
[2021-04-13 17:21:51] INFO (nni.compression.pytorch.utils.mask_conflict/MainThread) {'models.0.model.0.c1.0': 1, 'models.0.model.0.c2.0': 1, 'models.0.model.1.c1.0': 1, 'models.0.model.1.c2.0': 1, 'models.0.model.2.c1.0': 1, 'models.0.model.2.c2.0': 1, 'models.0.model.3.c1.0': 1, 'models.0.model.3.c2.0': 1, 'models.0.model.4.c1.0': 1, 'models.0.model.4.c2.0': 1, 'models.1.model.0.c1.0': 1, 'models.1.model.0.c2.0': 1, 'models.1.model.1.c1.0': 1, 'models.1.model.1.c2.0': 1, 'models.1.model.2.c1.0': 1, 'models.1.model.2.c2.0': 1, 'models.1.model.3.c1.0': 1, 'models.1.model.3.c2.0': 1, 'models.1.model.4.c1.0': 1, 'models.1.model.4.c2.0': 1, 'models.2.model.0.c1.0': 1, 'models.2.model.0.c2.0': 1, 'models.2.model.1.c1.0': 1, 'models.2.model.1.c2.0': 1, 'models.2.model.2.c1.0': 1, 'models.2.model.2.c2.0': 1, 'models.2.model.3.c1.0': 1, 'models.2.model.3.c2.0': 1, 'models.2.model.4.c1.0': 1, 'models.2.model.4.c2.0': 1, 'models.3.model.0.c1.0': 1, 'models.3.model.0.c2.0': 1, 'models.3.model.1.c1.0': 1, 'models.3.model.1.c2.0': 1, 'models.3.model.2.c1.0': 1, 'models.3.model.2.c2.0': 1, 'models.3.model.3.c1.0': 1, 'models.3.model.3.c2.0': 1, 'models.3.model.4.c1.0': 1, 'models.3.model.4.c2.0': 1}
[2021-04-13 17:21:51] INFO (nni.compression.pytorch.utils.mask_conflict/MainThread) dim0 sparsity: 0.019531
[2021-04-13 17:21:51] INFO (nni.compression.pytorch.utils.mask_conflict/MainThread) dim1 sparsity: 0.000000
[2021-04-13 17:21:51] INFO (nni.compression.pytorch.utils.mask_conflict/MainThread) detected conv prune dim: 0
[2021-04-13 17:21:52] INFO (nni.compression.pytorch.speedup.compressor/MainThread) infer module masks...
Traceback (most recent call last):
  File "train.py", line 171, in <module>
    m_speedup.speedup_model()
  File "/home/_/anaconda3/lib/python3.8/site-packages/nni/compression/pytorch/speedup/compressor.py", line 183, in speedup_model
    self.infer_modules_masks()
  File "/home/_/anaconda3/lib/python3.8/site-packages/nni/compression/pytorch/speedup/compressor.py", line 140, in infer_modules_masks
    self.infer_module_mask(module_name, None, mask=mask)
  File "/home/_/anaconda3/lib/python3.8/site-packages/nni/compression/pytorch/speedup/compressor.py", line 124, in infer_module_mask
    self.infer_module_mask(_module_name, module_name, in_shape=output_cmask)
  File "/home/_/anaconda3/lib/python3.8/site-packages/nni/compression/pytorch/speedup/compressor.py", line 124, in infer_module_mask
    self.infer_module_mask(_module_name, module_name, in_shape=output_cmask)
  File "/home/_/anaconda3/lib/python3.8/site-packages/nni/compression/pytorch/speedup/compressor.py", line 124, in infer_module_mask
    self.infer_module_mask(_module_name, module_name, in_shape=output_cmask)
  [Previous line repeated 4 more times]
  File "/home/_/anaconda3/lib/python3.8/site-packages/nni/compression/pytorch/speedup/compressor.py", line 90, in infer_module_mask
    raise RuntimeError(
RuntimeError: Has not supported infering output shape from input shape for module/function: `prim::TupleUnpack`, .prim::TupleUnpack.130

[J-shang]

The first way is using apply_compression_results https://nni.readthedocs.io/en/stable/Compression/Tutorial.html#export-compression-result

Or just

model.load_state_dict('model_weight.pth')

The speedup error can refer this: #3645