GPU based optical flow extraction from videos

Forked from https://github.com/feichtenhofer/gpu_flow by Antonino Furnari

News

2020-01-09

The semantics of the dilation parameter have changed to allow finer grained configuration. Previously optical flow was computed between frames I_{st} and I_{s(t+d)} where s is the stride and d the dilation. The code now computes flow between I_{st} and I_{st+d}--this makes the stride and dilation parameters completely independent which is more intuitive. If you wish to continue using the old code then use the docker image tagged with v1. All subsequent images and the latest tag will adopt the new behaviour described above.

Usage

We support running via docker and singularity.

Docker

• Ensure you're running nvidia-docker as this software is GPU accelerated. If using docker 19.03 or above then you can use the native docker nvidia GPU support.
• Pull the docker image:

  $ docker pull willprice/furnari-flow

• Dump out frames from the video you wish to compute flow for:

  $ mkdir my_video; ffmpeg -i my_video.mp4 -qscale 3 my_video/img_%06d.jpg

• Compute the flow using furnari-flow:

  $ mkdir my_video_flow
  $ docker run \
      --runtime=nvidia \
      --rm \
      --mount "type=bind,source=$PWD/my_video,target=/input" \
      --mount "type=bind,source=$PWD/my_video_flow,target=/output" \
      --mount "type=bind,source=$HOME/.nv,target=/cache/nv" \
      willprice/furnari-flow \
      img_%06d.jpg
  $ ls my_video_flow
  u v
  $ ls my_video_flow/u
  img_0000001.jpg
  img_0000002.jpg
  ...

Details

The software assumes that all video frames have been extracted in a directory. Files should be named according to some pattern, e.g., img_%07d.jpg. The software will put flow files in the same directory using a provided filename pattern, e.g., flow_%s_%07d.jpg, where the %s will be subsituted with "x" for the x flows and "y" for the y flows. For example, if DIR is a directory containing 4 images:

DIR:

• img_0000001.jpg
• img_0000002.jpg
• img_0000003.jpg
• img_0000004.jpg

the command compute_flow DIR img_%07d.jpg flow_%s_%07d.jpg will read the images in order and compute optical flows. The content of DIR will be as follows after the execution of the command:

DIR:

• img_0000001.jpg
• img_0000002.jpg
• img_0000003.jpg
• img_0000004.jpg
• flow_x_0000001.jpg
• flow_x_0000002.jpg
• flow_x_0000003.jpg
• flow_x_0000004.jpg
• flow_y_0000001.jpg
• flow_y_0000002.jpg
• flow_y_0000003.jpg
• flow_y_0000004.jpg

where flow_x_{n}.jpg is the x flow computed between img_{n}.jpg and img_{n+1}.jpg (if no dilation is used - see help).

Build

You only need to build this software if you intend on tweaking the source, otherwise you should just use the pre-built docker images.

Dependencies:

• OpenCV 2.4
• cmake

Installation

First, build opencv with gpu support. To do so, download opencv 2.4.x sources from https://opencv.org/releases.html. Unzip the downloaded archive, then enter the opencv folder and issue the following commands:

• mkdir build
• cd build
• cmake -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF .. (inspect the Dockerfile for further flags that might be of use)
• make -j $(nproc)

Then clone the current repository and enter the compute_flow_video folder. Type:

• export OpenCV_DIR=path_to_opencv_build_directory
• mkdir build
• cd build
• cmake -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF ..
• make -j $(nproc)