This is the official PyTorch implementation of our paper:
Improving Unsupervised Video Object Segmentation via Fake Flow Generation, arXiv 2024
Suhwan Cho, Minhyeok Lee, Jungho Lee, DongHyeong Kim, Seunghoon Lee, Sungmin Woo, Sangyoun Lee
Link: [arXiv]
You can also find other related papers at awesome-video-object-segmentation.
In unsupervised VOS, the scarcity of training data has been a significant bottleneck in achieving high segmentation accuracy. Inspired by observations on two-stream approaches, we introduce a novel data generation method based on the depth-to-flow conversion process. With our fake flow generation protocol, large-scale image-flow pairs can be leveraged during network training. To facilitate future research, we also prepare the DUTSv2 dataset, which is an extended version of DUTS, comprising pairs of the original images and the simulated flow maps.
1. Download DUTS and YouTube-VOS for network training.
2. Download DAVIS for network training and testing.
3. Download FBMS for network testing.
4. Download YouTube-Objects for network testing.
5. Download Long-Videos for network testing.
6. Save optical flow maps of YouTube-VOS 2018, DAVIS, FBMS, YouTube-Objects, and Long-Videos using RAFT.
7. For convenience, I also provide the pre-processed DUTSv2, YouTube-VOS, DAVIS, FBMS, YouTube-Objects, and Long-Videos.
8. Replace dataset paths in "run.py" file with your dataset paths.
1. Move to "run.py" file.
2. Define model version (ver): 'mitb0' or 'mitb1' or 'mitb2'
3. Check training settings.
4. Run FakeFlow training!!
python run.py --train
1. Move to "run.py" file.
2. Define model version (ver): 'mitb0' or 'mitb1' or 'mitb2'
3. Select a pre-trained model that accords with the defined model version.
4. Run FakeFlow testing!!
python run.py --test
pre-trained model (mitb0)
pre-trained model (mitb1)
pre-trained model (mitb2)
pre-computed results
Code and models are only available for non-commercial research purposes.
If you have any questions, please feel free to contact me :)
E-mail: suhwanx@gmail.com