export_torchscript_with_instances gives Module 'ResNet' has no attribute 'stages_and_names' error

[matthewchung74]

Instructions To Reproduce the 🐛 Bug:

1. Full runnable code or full changes you made:

code is here in colab
https://colab.research.google.com/drive/1Z8hWbW7hgTdVUu7kr5qQPwAddtRW251C?usp=sharing

it is the same code as the example colab except with parts of this file https://github.com/facebookresearch/detectron2/blob/master/tests/test_export_torchscript.py#L37 pasted in.

from detectron2.structures import Boxes
from torch import Tensor
from detectron2.export.torchscript import export_torchscript_with_instances

def _test_rcnn_model(config_path):
    model = model_zoo.get(config_path, trained=True)
    model.eval()

    fields = {
        "proposal_boxes": Boxes,
        "objectness_logits": Tensor,
        "pred_boxes": Boxes,
        "scores": Tensor,
        "pred_classes": Tensor,
        "pred_masks": Tensor,
    }
    script_model = export_torchscript_with_instances(model, fields)

    inputs = [{"image": get_sample_coco_image()}]
    with torch.no_grad():
        instance = model.inference(inputs, do_postprocess=False)[0]
        scripted_instance = script_model.inference(inputs, do_postprocess=False)[0]
    assert_instances_allclose(instance, scripted_instance)

_test_rcnn_model("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")

RuntimeError: 
Module 'ResNet' has no attribute 'stages_and_names' (This attribute exists on the Python module, but we failed to convert Python type: 'list' to a TorchScript type.):
  File "/usr/local/lib/python3.7/dist-packages/detectron2/modeling/backbone/resnet.py", line 435
        if "stem" in self._out_features:
            outputs["stem"] = x
        for stage, name in self.stages_and_names:
                           ~~~~~~~~~~~~~~~~~~~~~ <--- HERE
            x = stage(x)
            if name in self._out_features:

Expected behavior:

I would have expected an outputted torchscript file.

Environment:

colab

[ppwwyyxx]

As https://detectron2.readthedocs.io/en/latest/tutorials/deployment.html#deployment-with-tracing-or-scripting says the feature requires pytorch >= 1.8

[matthewchung74]

thanks @ppwwyyxx , the colab instructions for running Dectron2 use pytorch 1.7

!pip install torch==1.7.1 torchvision==0.8.2

do you know if that is just outdated? or do I need to train on 1.7 and have a different notebook for inference running 1.8?

[ppwwyyxx]

It is outdated.

You can train on 1.8 as well.

Using detectron2 with pytorch 1.8 requires building detectron2 from source for now.

[matthewchung74]

awesome. thank you.

[matthewchung74]

@ppwwyyxx sorry to bother but have one more question :

when looking at the outputs["instances"] of

predictor = DefaultPredictor(cfg)
outputs = predictor(I'm)

vs instance https://github.com/facebookresearch/detectron2/blob/master/tests/test_export_torchscript.py#L61

inputs = [{"image": image}]
with torch.no_grad():
    instance = model.inference(inputs, do_postprocess=False)[0]
    scripted_instance = script_model.inference(inputs, do_postprocess=False)[0]

I am not able to use Visualizer for the inference output. where it works fine for outputs["instances"]. Is that because this comment https://github.com/facebookresearch/detectron2/blob/master/tools/deploy/export_model.py#L77 .

If so, is there a workaround where I can use the visualizer on torchscript inference another way?

[ppwwyyxx]

Since do_postprocess=False is used in torchscript,

detectron2/detectron2/modeling/meta_arch/rcnn.py

Lines 218 to 219 in 321735b

	assert not torch.jit.is_scripting(), "Scripting is not supported for postprocess."
	return GeneralizedRCNN._postprocess(results, batched_inputs, images.image_sizes)

will have to be called after inference.

Also note that for now the scripted model cannot be reloaded later (#2582). --export-method tracing would work better for that purpose

[matthewchung74]

hi @ppwwyyxx when I tried using GeneralizedRCNN._postprocess I got an error NotImplementedError: Instances object is not iterable! which is thrown here.

detectron2/detectron2/modeling/meta_arch/rcnn.py

Line 239 in 3c0509e

for results_per_image, input_per_image, image_size in zip(

I was wondering if you have any advice or an example of turning the inference output such that it can be postprocessed?

i have my example colab here https://colab.research.google.com/drive/1Z8hWbW7hgTdVUu7kr5qQPwAddtRW251C?usp=sharing

(also, I saw your video on dectron2 on youtube, very cool)

[ppwwyyxx]

try _postprocess([instance], ...)