Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Link to dataset #4

Open
moeinheidari7829 opened this issue Dec 20, 2024 · 5 comments
Open

Link to dataset #4

moeinheidari7829 opened this issue Dec 20, 2024 · 5 comments

Comments

@moeinheidari7829
Copy link

Hi there,

Thank you so much for open-sourcing your amazing work. I was wondering if it would be possible for you to share a link to one of the datasets used in the paper (such as BRATS) formatted in the same way that your dataloader reads the data? I would greatly appreciate it.

Thank you in advance!

@bilalkabas
Copy link
Collaborator

bilalkabas commented Dec 30, 2024

Hi,

Thank you for your interest in our research. We released a dataset preparation script for BraTS (2021) dataset. This script can be modified for other datasets as well. It takes the original dataset and creates the one that our dataloader expects.

Following links can be used to download original datasets:

@moeinheidari7829
Copy link
Author

Hi,

Thank you for your interest in our research. We released a dataset preparation script for BraTS (2021) dataset. This script can be modified for other datasets as well. It takes the original dataset and creates the one that our dataloader expects.

Following links can be used to download original datasets:

Hi, thank you for responding to my questions and for sharing the code to organize the data. I have downloaded and preprocessed the data as outlined in the repository and trained the model using the same configurations as yours.
However, I am encountering inconsistent and poor results, as shown in the attached image, along with NaN values for PSNR and SSIM. Have you experienced a similar issue? I am running the code without any modifications to reproduce the result

epoch_49
epoch_49

@bilalkabas
Copy link
Collaborator

bilalkabas commented Jan 5, 2025

Hi,

We performed our experiments using BraTS 2015 dataset. I have shared a link to the BraTS 2021 along with the preprocessing script as it is easier to access and start experimentation straightaway. After performing debugging, I have detected 2 issues with the preprocessing script that I shared with you: (1) an overwrite issue causing each split to only have a single volume (a total of 155 slices), (2) empty slices at the beginning and end of the volumes. These are the reasons for the poor performance and NaN values.

I have updated the preprocessing script accordingly. Now, it expects the directory where you unzipped the original BraTS 2021 dataset. It creates train, val, test splits and then the formatted dataset.

Please not that this will not reproduce the exact results in the paper as it is the 2021 version. However, it is going to produce valid and similar results.

Thank you for your time.

@moeinheidari7829
Copy link
Author

Hi,

We performed our experiments using BraTS 2015 dataset. I have shared a link to the BraTS 2021 along with the preprocessing script as it is easier to access and start experimentation straightaway. After performing debugging, I have detected 2 issues with the preprocessing script that I shared with you: (1) an overwrite issue causing each split to only have a single volume (a total of 155 slices), (2) empty slices at the beginning and end of the volumes. These are the reasons for the poor performance and NaN values.

I have updated the preprocessing script accordingly. Now, it expects the directory where you unzipped the original BraTS 2021 dataset. It creates train, val, test splits and then the formatted dataset.

Please not that this will not reproduce the exact results in the paper as it is the 2021 version. However, it is going to produce valid and similar results.

Thank you for your time.

Hi,

Thank you very much for taking the time to answer my questions—I really appreciate your help!

I tested the code with the new preprocessing steps and trained the model using the same configurations. However, the results are still not satisfactory, as shown in the attached image (epoch_49_validation). Additionally, after reviewing the logs, it appears that the model is producing the same PSNR/SSIM values across different epochs, indicating a potential issue. I have also uploaded some of the logs for your reference.

Could you please help me troubleshoot this so that we can reproduce the results from the paper? Or at the very least, would you be able to kindly provide the preprocessing code for one of the datasets used in the paper? This would help us better understand the necessary steps and ensure that we are following the correct approach.

Looking forward to your guidance, and thanks again for your time and support!

Best regards,
Moein
epoch_49
output_0.5-3805905.txt

@bilalkabas
Copy link
Collaborator

Hi,

Additionally, after reviewing the logs, it appears that the model is producing the same PSNR/SSIM values across different epochs, indicating a potential issue.

The validation metrics are computed at every 10 epochs which can be configured in config.yaml > check_val_every_n_epoch: 10. Thus, val metrics are updated at every 10 epochs in logs.

However, the results are still not satisfactory, as shown in the attached image.

BraTS translations may seem poor due to blurry source-target ground truth image pairs compared to IXI.

BraTS2021 validation sample:
image

IXI validation sample:
image

We got 27.04 dB PSNR in validation and 26.89 dB PSNR in test set with the BraTS2021 dataset for the T1->T2 translation task. We have reported 27.42 dB PSNR in test set for the BraTS2015 dataset in our preprint.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants