Question with the REFUGE2 dataset

[mingsiliu557]

Dear author,
Congrats to ur excellent work accepted by miccai 2024. But I meet some problem when i ran ur code for REFUGE2:

1. I wonder why need to map the pixel in the mask into the range of (-1, 1).

2. I noticed that in ur ldm/data/refuge2.py, only extract the cup label, ignore the disk label. And u mentioned only support for binary segmentation:
   def getitem(self, i):
   # read segmentation and images
   example = dict((k, self.labels[k][i]) for k in self.labels)
   segmentation = Image.open(example["file_path_"].replace("image", "mask")).convert("RGB")
   image = Image.open(example["file_path_"]).convert("RGB") # same name, different postfix

   if self.size is not None:
       segmentation = segmentation.resize((self.size, self.size), resample=PIL.Image.NEAREST)
       image = image.resize((self.size, self.size), resample=PIL.Image.BILINEAR)
   
   if self.mode == "train":
       segmentation, image = self._utilize_transformation(segmentation, image, self.transform)
   
   # only support binary segmentation now:
   segmentation = np.array(segmentation).astype(np.float32)
   cup_label = (segmentation[:, :, 1] == 255.)         # extract `cup` segmentation
   segmentation = np.zeros((self.size, self.size, 3))
   segmentation[cup_label] = 1.
   if self.mode == "test":
       example["segmentation"] = segmentation   
   else:
       example["segmentation"] = ((segmentation * 2) - 1)   # range: binary -1 and 1
   
   image = np.array(image).astype(np.float32) / 255.
   image = (image * 2.) - 1.                            # range from -1 to 1, np.float32
   example["image"] = image
   example["class_id"] = np.array([-1])  # doesn't matter for binary seg
   
   assert np.max(segmentation) <= 1. and np.min(segmentation) >= -1.
   assert np.max(image) <= 1. and np.min(image) >= -1.
   return example
   

[lin-tianyu]

Hi Mingsi,

Thanks for your interest in our work. For the questions you've mentioned:

1.  If you are familiar with DDPM or Stable Diffusion, you may notice that diffusion models are designed to learn a data distribution by gradual denoising a normally distributed variable, which has a mean of 0 and a standard variation of 1 (most of the values are under the range of (-1, 1) ). Therefore, we map the mask into the range of (-1, 1) to fulfill this condition. In addition, we have tried using mask input with the range of (0, 1), but it doesn't work.

2. (1) Yes, we only extract the cup label of the REFUGE-2 dataset. This setting follows previous work such as MedSegDiff.

   (2) The SDSeg version accepted by MICCAI 2024 indeed only supports binary segmentation for now. In fact, we have come out with a way to expand SDSeg for multi-class segmentation, but it has not got SOTA yet, so we haven't made it published. If you have any ideas, feel free to contact me :)