Video Anomaly Detection - powered by Azure MLOps

The automation of detecting anomalous events in videos is a challenging problem that currently attracts a lot of attention by researchers, but also has broad applications across industry verticals.

The approach involves training deep neural networks to develop an in-depth understanding of the physical and causal rules in the observed scenes. The model effectively learns to predict future frames in the video in a self-supervised fashion.

By calculating the error in this prediction, it is then possible to detect if something unusual, an anomalous event, occurred, if there is a large prediction error.

The approach can be used both in a supervised and unsupervised fashion, thus enabling the detection of pre-defined anomalies, but also of anomalous events that have never occurred in the past.

Post on LinkedIn (includes video demonstration)

Learning Goals

You will learn:

1. How to adapt an existing neural network architecture to your use-case.
2. How to prepare video data for deep learning.
3. How to perform hyperparameter tuning with HyperDrive to improve the performance of you model.
4. How to deploy a deep neural network as a webservice for video processing.
5. How to post-process the output of a Keras model for secondary tasks (here, anomaly detection)
6. How to define a build pipeline for DevOps.

Pre-requisites

Skills

1. Some familiarity with concepts and frameworks for neural networks:
   • Framework: Keras
   • Concepts: convolutional, recurrent, and pooling layers.
2. Knowledge of basic data science and machine learning concepts. Here and here you'll find short introductory material.
3. Moderate skills in coding with Python and machine learning using Python. A good place to start is here.

Software Dependencies

• Various python modules. We recommend working with a conda environement (see environment.yml) - Documentation
• VS code https://code.visualstudio.com/
• X2Go https://wiki.x2go.org/doku.php

We found that a useful development environment is to have a VM with a GPU and connect to it using X2Go.

Hardware Dependencies

A computer with a GPU, Standard NC6 sufficient, faster learning with NC6_v2/3 or ND6. compare VM sizes

Dataset

UCSD Anomaly Detection Dataset

The dataset consists of individual files for video frames. Please use this script to create video files.

Agenda

Getting Started

1. Data Preparation
2. Model Development
3. Hyperparameter Tuning
4. Anomaly Detection
5. Deployment

Advanced Topics

1. Transfer learning - How to quickly retrain the model on new data.
2. AML Pipelines - Use AML pipelines to scale your solution.
3. MLOps - How to quickly scale your solution with the MLOps extension for DevOps.

References / Resources

• Research Article: Deep predictive coding networks for video prediction and unsupervised learning by Lotter, W., Kreiman, G. and Cox, D., 2016.

   @article{lotter2016deep,
   title={Deep predictive coding networks for video prediction and unsupervised learning},
   author={Lotter, William and Kreiman, Gabriel and Cox, David},
   journal={arXiv preprint arXiv:1605.08104},
   year={2016}
   }
  

• Original Prednet implentation is on github.com. Note, that the original implementation will only work in Python 2, but not in Python 3.

• Interesting blog post on Self-Supervised Video Anomaly Detection by Steve Shimozaki