Fall-Detection-Network

Introduction

This repo provides demo Pytorch implementation of the paper A Video-based Fall Detection Network by Spatio-temporal Joint-point Model on Edge Devices. We extract and filtrate the time-series joint-point features to capture the spatial and temporal dynamics, which are subsequently fed to the tensorized spatio-temporal joint-point model to predict falls.

Requirements

The requirements of extracting joint-point features are as follows:

Pytorch >= 1.1.0
torchvision >= 0.3.0
visdom
nibabel
pandas
tqdm
matplotlib
opencv-python
ntpath

The implementation of LSTM tensor compression is under the Keras 2.2.0.

Download the models manually: duc_se.pth (Google Drive | Baidu pan), yolov3-spp.weights(Google Drive | Baidu pan). Place them into ./models/sppe and ./models/yolo respectively.

Training

First step is preparing the training data, the fall detection dataset can be accessed by Multiple cameras fall dataset and UR Fall Detection Dataset.

Second step is getting the frames by video clip at different FPS. (Recommend FPS higher than 8)

Then run the extracting codes for training.

python demo_ectract.py

After getting the time-series joint-point features, we can feed them to the tensorized LSTM, which can been seen in TT_LSTM/MULT_TT.py. By running the following code, we can attain the model file by optimizing the training parameters.

python MULT_TT.py