FOUND

Pytorch implementation of the unsupervised object localization method FOUND. More details can be found in the paper:

Unsupervised Object Localization: Observing the Background to Discover Objects

by Oriane Siméoni, Chloé Sekkat, Gilles Puy, Antonin Vobecky, Eloi Zablocki and Patrick Pérez
CVPR 2023

| project page | arXiv | gradio |

If you use our FOUND code or framework in your research, please consider citing:

@inproceedings{simeoni2023found,
  author    = {Siméoni, Oriane and Sekkat, Chloé and Puy, Gilles and Vobecky, Antonin and Zablocki, Éloi and Pérez, Patrick},
  title     = {Unsupervised Object Localization: Observing the Background to Discover Objects},
  booktitle = {{IEEE} Conference on Computer Vision and Pattern Recognition, {CVPR}},
  year      = {2023},
}

Updates

• [Apr. 2023] Release of training code available here
• [Mar. 2023] Gradio space is available (link)
• [Feb. 2023] FOUND is accepted to CVPR23 !
• [Dec. 2022] First release

Overview

• Presentation
• Installation
• Using FOUND
• Evaluation: Saliency object detection
• Evaluation: Unsupervised object discovery
• Training of FOUND
• Acknowledgments

Installation of FOUND

Environment installation

This code was implemented and tested with python 3.7, PyTorch 1.8.1 and CUDA 11.1. Please install PyTorch. In order to install the additionnal dependencies, please launch the following command:

# Create conda environment
conda create -n found python=3.7
conda activate found

# Example of pytorch installation
pip install torch===1.8.1 torchvision==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html

# Install dependencies
pip install -r requirements.txt

Please install also DINO paper. The framework can be installed using the following commands:

git clone https://github.com/facebookresearch/dino.git
cd dino; 
touch __init__.py
echo -e "import sys\nfrom os.path import dirname, join\nsys.path.insert(0, join(dirname(__file__), '.'))" >> __init__.py; cd ../;

Usage of FOUND

We provide here the different command lines in order to repeat all results provided in our paper.

Application to one image

Using the following command it is possible to apply and visualize our method on one single image.

python main_visualize.py --img-path /datasets_local/VOC2007/JPEGImages/000030.jpg

Saliency object detection

We evaluate our method FOUND for the saliency detection on the datasets

• DUT-OMRON: --dataset-eval DUT-OMRON
• DUTS-TEST: --dataset-eval DUTS-TEST
• ECSSD: --dataset-eval ECSSD.

Please download those datasets and provide the dataset directory using the argument --dataset-dir. The parameter --evaluation-mode allow you to choose between single and multi setup and --apply-bilateral can be added to apply the bilateral solver. Please find here examples on the dataset ECSSD.

export DATASET_DIR=data/ # Put here your directory

# ECSSD, single mode, with bilateral solver post-processing
python main_found_evaluate.py --eval-type saliency --dataset-eval ECSSD --evaluation-mode single --apply-bilateral --dataset-dir $DATASET_DIR
# same but multi mode
python main_found_evaluate.py --eval-type saliency --dataset-eval ECSSD --evaluation-mode multi --apply-bilateral --dataset-dir $DATASET_DIR

Unsupervised object discovery

In order to evaluate on the unsupervised object discovery task, we follow the framework used in our previous work LOST. The task is implemented for the following datasets, please download the benckmarks and put them in the folder data/.

• VOC07: --dataset-eval VOC07
• VOC12: --dataset-eval VOC12
• COCO20k: --dataset-eval COCO20k

export DATASET_DIR=data/ # Put here your directory

# VOC07
python main_found_evaluate.py --eval-type uod --dataset-eval VOC07 --evaluation-mode single --dataset-dir $DATASET_DIR
# VOC12
python main_found_evaluate.py --eval-type uod --dataset-eval VOC12 --evaluation-mode single --dataset-dir $DATASET_DIR
# COCO20k
python main_found_evaluate.py --eval-type uod --dataset-eval COCO20k --evaluation-mode single --dataset-dir $DATASET_DIR

Training of FOUND

In order to train a FOUND model, please start by installing the framework. If already installed, please run again

# Create conda environment
conda activate found

# Install dependencies
pip install -r requirements.txt

The training is performed on the dataset DUTS-TR that should be put in the directory data.

Then the training can be launched using the following command. Visualizations and training curves can be observed using tensorboard.

export DATASET_DIR=data/ # Root directory of all datasets, both training and evaluation

python main_found_train.py --dataset-dir $DATASET_DIR

Once the training done, you can launch the evaluation using the scripts evaluate_saliency.sh and evaluate_uod.sh with the commands:

export MODEL="outputs/FOUND-DUTS-TR-vit_small8/decoder_weights_niter500.pt"

# Evaluation of saliency detection
source evaluate_saliency.sh $MODEL $DATASET_DIR single
source evaluate_saliency.sh $MODEL $DATASET_DIR multi

# Evaluation of unsupervised object discovery
source evaluate_uod.sh $MODEL $DATASET_DIR

Acknowledgments

This repository was build on the great works SelfMask, TokenCut and our previous work LOST. Please, consider acknowledging these projects.