Whether the mmd3d support visualize 3d bbox

[ustbzjf1]

As I have trained the fcos on nuscenes dataset, I want to visualize the prediction for analyzing. I found that mmd3d seems only support visualize the 2d bbox.

[wHao-Wu]

Hi, @ustbzjf1

You can set the --show to True when testing. Then the prediction results will be visualized based on Open3d online or based on MeshLab offline.

[ustbzjf1]

Yeah, I have set the --show when testing, but then I found that a series bug happened in mmdet/core/visualization/image.py.
I don't find the code visualize 3d bbox with Open3d.

[wHao-Wu]

The codes to visualize 3d bbox are in here and here.

[wHao-Wu]

Thanks for your report. I found that FCOS based on monocular data is inherited from the single-stage detector from mmdet. So FCOS will use the show_results from mmdet instead of mmdet3d, which cause incorrect visualization in 3D detection. We will fix the problem as soon as possible.

[ustbzjf1]

Yes, I have tried define show_results function in fcos_mono3d.py, but it raise another bug, now I am trying to make SingleStageMono3DDetector inherit from SingleStage3DDetector

[Wuziyi616]

@ustbzjf1 Hi, I am currently working on a PR to add visualization functions to Mono-3D dataset and models. But it may take a while because I also encountered some errors. May be merged to MMDet3D in next week. It would be great if you can create a PR if you succeed in doing visualization.

[ustbzjf1]

OK，I will try to solve this bug in the weekend, if I solved this problem, I will update the comment.

[Wuziyi616]

Hi, the errors are solved now. You can refer to this branch of my repo if you want to use this feature in advance. The merge of this PR may not be very soon.

See some visualization examples here #542.

[ustbzjf1]

Yeah, I modify the code accordding to the branch of mono_vis_demo, but the visualize result is show as bellow:

[Wuziyi616]

@ustbzjf1 It's true that there are many false positives here. Maybe you can try setting a score_thr to filter low confidence boxes? Note that if you filter too much boxes, the AP will be lower.

[ustbzjf1]

If it's convenient to add your wechat, I found that after modify the code as branch mono_vis_demo. the show_score_thr is not used in the show_results function.

[Wuziyi616]

@ustbzjf1 Can give me your email and I will send my wechat account there? I don't want to show it public here after all :)
Just send me an email to wuzy17@mails.tsinghua.edu.cn

[ustbzjf1]

@Wuziyi616 I have sent you the email. My email address is ustbzjf@gmail.com. You can sent me your wechat, I will add you. Thanks

[DMing11]

Hi @Wuziyi616 , I got the result as below when run demo/mono_det_demo.py in the master branch

I have tried setting several --score-thr to filter low confidence boxes, but don't got the same results as you do #542
Are you using NMS to filter detection results? Thanks