This repository contains instructions needed to recreate the benchmark detailed in the paper Learning Human Action Recognition Representations Without Real Humans. Specifically, we discuss how to download the two pretraining datasets and six downstream evaluation datasets.
Fig 1: Schematic of Benchmark
We run the HAT framework (https://github.com/princetonvisualai/HAT) to remove humans from the Kinetics dataset. The Kinetics-400 dataset can be downloaded here. We run HAT on a subset of Kinetics that is 150 classes as indicated in splits/kinetics/kinetics_labels.txt to created No-Human Kinetics, a dataset of 150 classes consisting of videos with humans removed from each frame through inpainting. We use No-Human Kinetics to pretraining action recognition models.
See this google doc for detailed instructions on running HAT.
We use the same Synthetic dataset of 150 classes from SynAPT: https://github.com/mintjohnkim/SynAPT. Please follow instructions from that github.
| Downstream Dataset | Download Instructions |
|---|---|
| UCF101 | https://www.crcv.ucf.edu/data/UCF101.php |
| HMDB51 | https://serre-lab.clps.brown.edu/resource/hmdb-a-large-human-motion-database/ |
| Mini-SSV2 | https://developer.qualcomm.com/software/ai-datasets/something-something |
| Diving48 | http://www.svcl.ucsd.edu/projects/resound/dataset.html |
| IkeaFA | https://tengdahan.github.io/ikea.html |
| UAV-Human | https://github.com/sutdcv/UAV-Human |
Note: Mini-SSV2 is a subset of the Something-Something V2 dataset found in the link above. After downloading, you can access the subset used for train and validation in splits/mini_ssv2.
Note 2: IkeaFA and UAV should be resized to 224 x 224 to reduce data loader latency during training. Refer to splits/resize_ikea_and_uav.py.
splits/folder contains train/val splits used to evaluate the models. Files are stored as txt or csv.