A High-Resolution Dataset for Instance Detection with Multi-View Instance Capture

NeurIPS (Datasets and Benchmarks) 2023

Authors: Qianqian Shen^*, Yunhan Zhao^*, Nahyun Kwon, Jeeeun Kim, Yanan Li, Shu Kong

If you find our model/method/dataset useful, please cite our work (NeurIPS version on arxiv):

@article{shen2024high,
  title={A High-Resolution Dataset for Instance Detection with Multi-View Object Capture},
  author={Shen, Qianqian and Zhao, Yunhan and Kwon, Nahyun and Kim, Jeeeun and Li, Yanan and Kong, Shu},
  journal={Advances in Neural Information Processing Systems},
  volume={36},
  year={2024}
}

The InsDet datase is a high-resolution real-world dataset for Instance Detection with Multi-view Instance Capture.
We provide an InsDet-mini for demo and visualization, and the full dataset InsDet-FULL.

Dataset

The full dataset contains 100 objects with multi-view profile images in 24 rotation positions (per 15°), 160 testing scene images with high-resolution, and 200 pure background images. The mini version contains 5 objects, 10 testing scene images, and 10 pure background images.

Details

The Objects contains:

• 000_aveda_shampoo
  • images: raw RGB images (e.g., "images/001.jpg")
  • masks: segmentation masks generated by GrabCut Annotation Toolbox (e.g., "masks/001.png")

• $\vdots$

• 099_mug_blue

Tip: The first three digits specify the instance id.

The Scenes contains:

• easy
  • leisure_zone
    • raw RGB images with 6144×8192 pixels (e.g. “office001/rgb_000.jpg”)
    • bounding box annotation for objects in test scenes generated by labelImg toolbox and using PascalVOC format (e.g. “office_001/rgb_000.xml”)
  • meeting_room
  • office_002
  • pantry_room_002
  • sink
• hard
  • office_001
  • pantry_room_001

Tip: Each bounding box is specified by [xmin, ymin, xmax, ymax].

The Background contains 200 pure background images that do not include any instances from Objects folder.

Code

The project is built on detectron2, segment-anything, and DINOv2.

Demo

The Jupyter notebooks files demonstrate our non-learned method using SAM and DINOv2. We choose light pretrained models of SAM (vit_l) and DINOv2 (dinov2_vits14) for efficiency.