DATASETS.md

Video Dataset Preparation

Your are expected to prepare data for Kinetics-400 dataset, UCF101, HMDB51, Something-something-v1 and Something-something-v2 datasets.

Official Downloader

• Kinetics-400: Download the videos via the official scripts. We download the copy from the Nonlocal Network.
• UCF101: Download the videos via the official website
• HMDB51: Download the videos via the official website
• Something-Something V1: Download the video frames via the official website
• Something-Something V2: Download the videos via the official website

Dataset Processing

This repo supports raw frame format of videos. Therefore, you can extract frames from raw videos. Also, we support a fast VideoLoader, which can be found at Fast PyAVDecode.

Resize original videos (Optional)

# Use FFmpeg to rescale the videos to short edge =256 pixels. 
python video_resize.py ROOT_PATH OUT_PATH --level 2 -se 256

Extract raw frames

Extract frames for training and validation, use --lib to choose opencv or ffmpeg

# For Kinetics400
python video2image.py ROOT_PATH OUT_PATH --level 2  --lib ffmpeg -fps 30

# For Sth-Sth v2
python video2image.py ROOT_PATH OUT_PATH --level 1 -se 256 --lib opencv --prefix %06d.jpg

# For UCF101
python video2image.py ROOT_PATH OUT_PATH --level 2 --lib opencv --prefix image_%04d.jpg

# For HMDB51
python video2image.py ROOT_PATH OUT_PATH --level 2 --lib opencv --prefix image_%06d.jpg

Prepare annotations

Prepare label list, [video_name, #frames, label] for each row, and save them in datalist folder.

# For sthv1/v2
python gen_label.py IMAGE_ROOT sthv1

# kinetics400/600/700
python gen_label.py IMAGE_ROOT kinetics400 --phase train  --level 1
python gen_label.py VIDEO_ROOT kinetics400 --phase train  --level 2 --source video

# For ucf101/hmdb51
python gen_label.py IMAGE_ROOT ucf101  --split 1
python gen_label.py VIDEO_ROOT ucf101  --split 1 --source video
python gen_label.py FLOW_ROOT ucf101  --split 1 --source flow