Add Benchmark CT-Dataset KiTS19 #36
Conversation
| question = "<img> Please segment the kidney and tumor in the CT images." | ||
|
|
||
| batcher_input._add_text_prompt("user", question) | ||
| if prompt: |
There was a problem hiding this comment.
As you have suggested, we perhaps should always do the conversion within class, so that BatcherInput only takes Numpy Array as input for both images and segmentation masks. @corentin-ryr
There was a problem hiding this comment.
Yes you're right it would simplify the usage of the tool. For now let's keep it this way and we will revisit it soon.
| answer = torch.LongTensor(answer) | ||
| one_hot_answer = one_hot(answer, num_classes= self.num_classes).movedim(-1, 1) | ||
|
|
||
| return one_hot_answer |
There was a problem hiding this comment.
The calculation of DICE score has two modes: one-hot and index.
My idea is: we default on the one-hot encoding, so for different datasets, probably pre-processing is needed. The return will always be a hot-encoded LongTensor. @corentin-ryr
There was a problem hiding this comment.
Another solution would be to only have single-class masks (binary masks) and allow the batcher to return multiple masks in case there are multiple labels in the dataset.
We can either have :
- complex question asking to label multiple things and the batcher has to return multiple masks
- split the questions to have 2 different questions: one to label the kidney and another to label the tumor. This way it simplifies the metric computation and it will be easier to stratify (compute overall accuracy and accuracy on just kidney/tumor).
Created KiTS Segmentation Task
To-do: