How to get probability values from the Detectron model

[bokelai1989]

Hello Harsh,

First of all, I want to say thank you very much for sharing your work about this background removal! One of my current projects is quite related to it. I have read both your articles and your code repo and they are quite helpful to me! Thanks a lot! :)

I am using Detectron2 to do instance segmentation (human face + hair) on images and achieved 91% precision. My next step is to do background removal on those segmentations. I understand from your article that a trimap is needed, and you mentioned, " However, with a few tweaks, we can get the probability values from the Detectron model also.", I am wondering whether you can share those codes as well? I tried to track down the raw code of detectron2 and found it hard to get it out. Thanks a lot!!

I did look at your detectron_seg.py script, but I see the outputs (using out.pred_mask as below code shows) look like still just binary instead of pixel-level probability for the segmentation.

` def get_seg_output(self,image:np.array):
out = self.model(image)['instances']

    outputs = [(out.pred_masks[i],out.pred_classes[i]) for i in  range(len(out.pred_classes)) if out.pred_classes[i]==0]
    
    return outputs`

[harsh2912]

You need to look inside the detectron library and grab the input before it is applying the threshold. I did that a year back, so I don't exactly remember where you will find that. But there certainly is a way.

…

On Tue, 13 Jul, 2021, 7:33 am bokelai1989, ***@***.***> wrote: Hello Harsh, First of all, I want to say thank you very much for sharing your work about this background removal. One of my current projects is quite related to it. I have read both your articles and your code repo and they are quite helpful to me! Thanks a lot! I am using Detectron2 to do instance segmentation (human face + hair) on images and achieved 91% precision. My next step is to do background removal on those segmentations. I understand from your article <https://heartbeat.fritz.ai/changing-backgrounds-with-image-segmentation-deep-learning-code-implementation-d848648ac477> that a trimap is needed, and you mentioned, " *However, with a few tweaks, we can get the probability values from the Detectron model also*.", *I am wondering whether you can share those codes as well?* I tried to track down the raw code of detectron2 and found it hard to get it out. Thanks a lot!! I did look at your detectron_seg.py script, but I see the outputs (using out.pred_mask as below code shows) look like still just *binary* instead of *pixel-level probability* for the segmentation. ` def get_seg_output(self,image:np.array): out = self.model(image)['instances'] outputs = [(out.pred_masks[i],out.pred_classes[i]) for i in range(len(out.pred_classes)) if out.pred_classes[i]==0] return outputs` — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AENHXF2XQT7KQS4EABRYH7LTXONG7ANCNFSM5AH7NODQ> .

[bokelai1989]

Hello Harsh, thanks for your fast reply! I will try to track it down further and see then.

[harsh2912]

You're welcome!

…

On Wed, 14 Jul, 2021, 5:52 am bokelai1989, ***@***.***> wrote: Hello Harsh, thanks for your fast reply! I will try to track it down further and see then. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AENHXF4UTNPC2PEB6T4WOQLTXTKE5ANCNFSM5AH7NODQ> .