A program that provides a preview of the Azure-Kinect's skeleton data and saves the coordinates to file
@author: Matthew Millard @date : 25 November 2021
-
Connect the Kinect: a. To power supply it comes with b. Via USB cable to the laptop
-
Place the Kinect at least 70 cm away from you. Point the Kinect towards you
-
Open a terminal in the folder: /home/orb/Research/Kinect/code/Kinect-Projects/kinect_mocap_studio/build
-
Run ./kinect_mocap_studio --help
You should see
USAGE:
./kinect_mocap_studio [-s <double>] [-f <int>] [-c <string>] [-d
<string>] [-o <string>] [-w <bool>] [-i <string>]
[--] [--version] [-h]
Where:
-s <double>, --smoothing <double>
Amount of temporal smoothing in the skeleton tracker (0-1)
-f <int>, --fps <int>
Frames per second: 5, 15, 30
-c <string>, --color_mode <string>
Color resolution: OFF, 720P, 1080P, 1440P, 1536P, 2160P, 3072P
-d <string>, --depth_mode <string>
Depth mode: OFF, NFOV_2X2BINNED, NFOV_UNBINNED, WFOV_2X2BINNED,
WFOV_UNBINNED, PASSIVE_IR
-o <string>, --outfile <string>
Name of output file excluding the file extension
-w <bool>, --write <bool>
Write sensor data to *.mkv file
-i <string>, --infile <string>
Input sensor file of type *.mkv to process
--, --ignore_rest
Ignores the rest of the labeled arguments following this flag.
--version
Displays version information and exits.
-h, --help
Displays usage information and exits.
kinect_mocap_studio is a command-line tool to recordvideo and skeletal
data from the Azure-Kinect
- Run the kinect: ./kinect_mocap_studio -f 15 -c 720P -d WFOV_UNBINNED -o testData -w 1
You should see a view of the depth camera with the skeleton tracking overlaid on people within the frame (in orange-yellow). The following output should also appear in the terminal:
Basic Navigation:
Rotate: Rotate the camera by moving the mouse while holding mouse left button Pan: Translate the scene by holding Ctrl key and drag the scene with mouse left button Zoom in/out: Move closer/farther away from the scene center by scrolling the mouse scroll wheel Select Center: Center the scene based on a detected joint by right clicking the joint with mouse
Key Shortcuts
ESC: quit h: help b: body visualization mode k: 3d window layout
depth_mode :WFOV_UNBINNED color_resolution :720P frames_per_second :15 temporal smoothing :0 output file name :testData_WFOV_UNBINNED_720P_15fps.json video file name :testData_WFOV_UNBINNED_720P_15fps.mkv
The *.mkv file contains the raw depth camera and RGB images. The *.json file contains the output of the skeleton tracker.
- To stop the recording, make sure the viewer has the main focus and press the 'ESC' button.
If you run the kinect camera again using exactly the same command, and thus exactly the same output file name, the program will add an integer to the end of the name so that you don't overwrite the old results. For example:
./kinect_mocap_studio -f 15 -c 720P -d WFOV_UNBINNED -o testData -w 1
will produce files with the names
testData_WFOV_UNBINNED_720P_15fps_0.json testData_WFOV_UNBINNED_720P_15fps_0.mkv
If you would like to reprocess data that you have already collected, perhaps with a different smoothing parameter, you can do so using the -i flag:
./kinect_mocap_studio -f 15 -c 720P -d WFOV_UNBINNED -s 0.5 -i testData_WFOV_UNBINNED_720P_15fps_0.mkv
Upon calling this function a view of the recorded depth data will appear and the skeleton tracking will be overlaid in yellow-orange. However, this time the movements of the skeleton tracking will be smoothed by a factor of 0.5. These results will be written to a *.json file with the same name as the *.mkv file. If necessary a number will be appended to the file name to prevent filename collisions. In this case:
testData_WFOV_UNBINNED_720P_15fps_0_0.json
See the Kinect online documentation for further details of how the smoothing algorithm works.
The Kinect libraries are supported to run on Ubuntu 18.04. While it is possible to install it on Ubuntu 20.04 this takes some extra time and patience. The following steps should help.
- To install the Azure-Kinect tools on Ubuntu 20.04 begin by following the instructions here, which are copied below:
These are the steps I followed to install k4a-tools, libk4a and libk4abt on Ubuntu 20.04. The general steps are as outlined in https://docs.microsoft.com/en-us/azure/Kinect-dk/sensor-sdk-download, with a couple of hacks to make things work on 20.04.
- use of 18.04 repo, even though OS is 20.04
- installed lower versions of tools and libraries (as latest versions of sensor and body tracker don't seem to be compatible on 20.04)
$ curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add - $ sudo apt-add-repository https://packages.microsoft.com/ubuntu/18.04/prod $ curl -sSL https://packages.microsoft.com/config/ubuntu/18.04/prod.list | sudo tee /etc/apt/sources.list.d/microsoft-prod.list $ curl -sSL https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add - $ sudo apt-get update $ sudo apt install libk4a1.3-dev $ sudo apt install libk4abt1.0-dev $ sudo apt install k4a-tools=1.3.0
- Verify sensor library by launching camera viewer
I have to run k4aviewer using sudo.
- Clone and build Azure Kinect Samples.
- Run
simple_3d_viewerto verify body tracker works.
I ran into all kinds of problems here.
Still cranking through: https://feedback.azure.com/forums/920053-azure-kinect-dk/suggestions/40368301-support-for-ubuntu-20-04-lts
https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/usage.md
-
Clone repo: https://github.com/microsoft/Azure-Kinect-Samples
-
Clone submodules: git submodule update --init --recursive
-
As instructed here try building: https://github.com/microsoft/Azure-Kinect-Samples
I had to (libk4abt was already installed):
apt install libxi-dev
Then I tried:
mkdir build cd build cmake .. -GNinja ninja
At this point git will complain as you are probably missing a bunch of libraries. Simply google the name of the offending library/package, and figure out what you need to install. This worked without a hitch.
-
Next simple_3d_viewer failed to build upon calling ninja:
[6/33] Building C object body-tracking-samples/extern/glfw/src/src/CMakeFiles/glfw.dir/linux_joystick.c.o
../body-tracking-samples/extern/glfw/src/src/linux_joystick.c: In function ‘_glfwInitJoysticksLinux’:
../body-tracking-samples/extern/glfw/src/src/linux_joystick.c:224:46: warning: ‘%s’ directive output may be truncated writing up to 255 bytes into a region of size 19 [-Wformat-truncation=]
224 | snprintf(path, sizeof(path), "%s/%s", dirname, entry->d_name);
| ^~
../body-tracking-samples/extern/glfw/src/src/linux_joystick.c:224:13: note: ‘snprintf’ output 2 or more bytes (assuming 257) into a destination of size 20
224 | snprintf(path, sizeof(path), "%s/%s", dirname, entry->d_name);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[33/33] Linking CXX executable bin/simple_3d_viewer
FAILED: bin/simple_3d_viewer
: && /usr/bin/c++ -Wl,--as-needed,-rpath-link=/home/mjhmilla/dev/AzureKinect/Azure-Kinect-Samples/build/bin body-tracking-samples/simple_3d_viewer/CMakeFiles/simple_3d_viewer.dir/main.cpp.o -o bin/simple_3d_viewer -Wl,-rpath,"$ORIGIN" -lk4a -lk4abt body-tracking-samples/sample_helper_libs/window_controller_3d/libwindow_controller_3d.a body-tracking-samples/extern/glfw/src/src/libglfw3.a /usr/lib/x86_64-linux-gnu/librt.so -lm -ldl /usr/lib/x86_64-linux-gnu/libX11.so -lpthread /usr/lib/x86_64-linux-gnu/libXrandr.so /usr/lib/x86_64-linux-gnu/libXinerama.so /usr/lib/x86_64-linux-gnu/libXcursor.so && :
/usr/bin/ld: body-tracking-samples/simple_3d_viewer/CMakeFiles/simple_3d_viewer.dir/main.cpp.o: in function PlayFile(InputSettings)': main.cpp:(.text+0xd42): undefined reference to k4a_playback_open'
/usr/bin/ld: main.cpp:(.text+0xd89): undefined reference to k4a_playback_get_calibration' /usr/bin/ld: main.cpp:(.text+0xefd): undefined reference to k4a_playback_get_next_capture'
/usr/bin/ld: main.cpp:(.text+0x10df): undefined reference to `k4a_playback_close'
collect2: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
The fix for this comes from this github post: microsoft/Azure-Kinect-Sensor-SDK#1296
And it was:
"This could be resolved by adding k4arecord in target_link_libraries in CMakeLists.txt of simple_3d_viewer"
Quite literally (2nd last line added manually):
add_executable(simple_3d_viewer main.cpp)
target_include_directories(simple_3d_viewer PRIVATE ../sample_helper_includes)
target_link_libraries(simple_3d_viewer PRIVATE k4a k4abt window_controller_3d::window_controller_3d glfw::glfw k4arecord )
To learn more, I recommend carefully going through the kinect_mocap_studio.cc code. This code was created using the examples that appear in the Azure-Kinect-Sensor-SDK and the Azure-Kinect-Samples. Note that not all of these samples are configured to be built in the original download. However, with a little bit of CMake work most of the samples can be built and run. There is a lot of additional functionality that appears in these examples that you can take advantage of in your own programs.