SAVi++ conditional background

[Divadi]

Hello, thank you for the interesting work. I had a question regarding Figure 7 in the appendix:

It is mentioned that thresholding does not have an affect on the conditional model. However, I am slightly confused by this point.

Since the task is to reconstruct the depth map for the entire image, there must be some slot(s) that predict depth for the background (ground, guardrail, etc). We see this be the case for SAVi++ unconditional. However, there seems to be no slots at all constructing the ground, etc for SAVi++ conditional - why might this be? Even with first-frame conditioning, again, I believed there should be some slot predicting the depth for BG regions.

Or at the very least, when doing argmax for the alpha mask over the slots for the ground region, the ground region should be assigned to some slot, right?

Thank you in advance, and I apologize if I misunderstood something.

[tkipf]

Thank you for your question! Apologies for the confusion -- our wording in the figure caption was slightly ambiguous. For the conditional model, we only visualize segmentations for conditioned slots, i.e. unconditioned slots (which would capture the background) are hidden by default. We noticed that for the conditioned slots the threshold does not have an effect in those visualizations since they typically capture smaller regions in the images.