This is an unofficial Tensorflow implementation of the DDNet β Double-feature Double-motion Network
The authors of this repository believe, that this implementation is more readable and easier to understand than the original one.
Results on SHREC dataset (using 3D skeletons)
Methods | Parameters | 14 Gestures | 28 Gestures | Speed on GPU |
---|---|---|---|---|
Dynamic hand [19] (CVPRW16) | - | 88.2% | 81.9% | - |
Key-frame CNN [4] (3DOR17) | 7.92 M | 82.9% | 71.9% | - |
3 Cent [21] (STAG17) | - | 77.9% | - | - |
CNN+LSTM [38] (PR18) | 8-9 M | 89.8% | 86.3% | 238 FPS |
Parallel CNN [5] (RFIAP18) | 13.83 M | 91.3% | 84.4% | - |
STA-Res-TCN [6] (Gesture18) | 5-6 M | 93.6% | 90.7% | 303 FPS |
MFA-Net [23] (Sensor19) | - | 91.3% | 86.6% | 361 FPS |
DD-Net (filters=64, w/o global fast&slow motion) | 1.70 M | 55.2% | 41.6% | - |
DD-Net (filters=64, w/o global slow motion) | 1.76 M | 92.7% | 90.2% | - |
DD-Net (filters=64, w/o global fast motion) | 1.76 M | 93.3% | 90.5% | - |
DD-Net (filters=64) | 1.82 M | 94.6% | 91.9% | 2,200 FPS |
DD-Net (filters=32) | 0.50 M | 93.5% | 90.4% | 3,100 FPS |
DD-Net (filters=16) | 0.15 M | 91.8% | 90.0% | 3,500 FPS |
JHMDB raw data download link: http://jhmdb.is.tue.mpg.de/challenge/JHMDB/datasets
SHREC raw data download link: https://shapenet.cs.stanford.edu/shrec17/
This repository contains sources directory with implementation.
sources/DDNet.py
- contains the actual model class
sources/create_dataset.py
- script for creating a binary files with skeletons and labels (with pickle) from a
dataset that has a structure like SHREC. Note that you may need to
modify it if the structure of your dataset is different.
+---gesture_1
|ββ +---finger_1
|ββ|ββ +---subject_1
|ββ|ββ|ββ +---essai_1
|ββ|ββ|ββ | ββ
|ββ|ββ|ββ | ββdepth_0.png
|ββ|ββ|ββ | ββdepth_1.png
|ββ|ββ|ββ | ββ...
|ββ|ββ|ββ | ββdepth_N-1.png
|ββ|ββ|ββ | ββgeneral_informations.txt
|ββ|ββ|ββ | ββskeletons_image.txt
|ββ|ββ|ββ | ββskeletons_world.txt
|ββ|ββ|ββ |
|ββ|ββ|ββ \---essai_2
|ββ|ββ|ββ ...
|ββ|ββ|ββ \---essai_5
|ββ|ββ \---subject_2
|ββ|ββ ...
|ββ|ββ \---subject_20
|ββ \---finger_2
...
\---gesture_14
train_gestures.txt
test_gestures.txt
souces/utils.py
- contains an important Config class with configurations for model training. There are also all
utilisation functions for the model training.
sources/train.py
- script for training a model on a preprocessed dataset like
SHREC. Note that can modify this script to work with your own
dataset.
Please cite the original paper.
@inproceedings{yang2019ddnet,
title={Make Skeleton-based Action Recognition Model Smaller, Faster and Better},
author={Fan Yang, Sakriani Sakti, Yang Wu, and Satoshi Nakamura},
booktitle={ACM International Conference on Multimedia in Asia},
year={2019}
}