Click here to Skip to main content
15,868,340 members
Articles / Programming Languages / C#
Tip/Trick

Kinect Version 2 Depth Frame to .mat File Exporter Tool

Rate me:
Please Sign up or sign in to vote.
5.00/5 (5 votes)
16 Sep 2014CPOL4 min read 111.8K   4.7K   5   42
Tool for extracting depth frames from Kinect v2 to .mat files, with point cloud generator script. Ready to use!

Introduction

The new Kinect v2 is awesome, however for people who are not coding experts, it can be hard to get the data of the Kinect into a workable setting, like MatLab. This tool is meant as a solution to solve the problem of getting depth data from the Kinect SDK into MatLab. And no external libraries are needed! (except for the ones needed for Windows Kinect and Windows)

Furthermore, a class is also provided, which can be used to export any ushort (or uint16) array to a loadable .mat file.

Background

I tried a few libraries (including csmatio and matio) for extracting the depth frames to .mat files. None of them seemed to work and therefore I decided to make my own .mat file writer. It's not meant to be an example of good coding, but rather a usable tool.

Main Tool

The main tool is called "KinectMLConnect", which is both found as source code and .exe, ready to be built in VS (tested in VS 2013), or run directly in Windows.
(The .exe is located in: "KinectMLConnect\KinectMLConnect\KinectMLConnect\bin\Release".)
The tool simply listens for an active sensor (or for the Kinect studio sensor emulator), grabs the stream and exports each frame as a .mat file.

The interface is quite simple and self explanatory, and is shown here:

Image 1

There are two options, extracting IR frames or depth frames. Due to a heavy amount of bit reordering in the program, I have chosen not to make it available to get both at the same time. This would probably cause the program to freeze and/or break.

By default, the frames are exported to "\%My documents dir%\Kinect to Matlab\TYPEframeXX.mat". The tool will overwrite previous frames, so make sure to move them to another folder! TYPE refers to Depth or IR.

Intrinsic parameters of the depth camera are saved to the file "Intrinsic parameters.txt" in the Kinect to Matlab folder. NOTE: Some problems have been reported on this, however, I have not been able to recreate the problems, so it is hard for me to debug them.

The frame timings are saved in a .mat file called "FrameTimings.mat" in the Kinect to Matlab folder. This file is set to a fixed size of 10000 elements giving roughly 6 hours of recording. If there is overflow, the world will end.

REMARK: The matrix extracted from this code is rotated 90 degrees, hence in matlab use:

rot90(depthmat,-1) 

And it should be right like this:

Image 2

MATWriter Class

Included is also a class file, for the MATWriter class, which is the one used for the actual export of the frames. Its constructor (and only callable code) is given here:

C#
public MATWriter(string name, string filepath, ushort[] data, int height, int width)

Hopefully it is very clear, that it is kept very simple, so that it is easy to use.

A new input string, 'name', is added to the function. This sets the name of the variable in MatLab. Since the .mat file writing is troublesome, this variable will be concatenated to 8 characters. Hence if you write "rainfall2008" only "rainfall" will be shown as the variable name in MatLab.

It cannot be used directly to export the RGB channels from the kinect, but with some Ninja tricks, it could probably be used for this purpose. (e.g. run matwriter for each channel in the RGB image)

DepthToXYZ

To make it even more simple, I've added a matlab script, which should be placed in the "\%My documents dir%\Kinect to Matlab" folder. From here, it will run all the frames through and calculate the cartesian coordinates of each point and save the coordinates in the variables, 'wx', 'wy' and 'wz' (with 'w' signifying 'world').
At the end of the file, there is a small code snippet to display a frame with scatter3 plotting, like this:

Image 3

Points of Interest

If anyone would like to edit the MATwriter or dig into how a .mat file is created, see this pdf from MathWorks.

Also, I would like to refer to Vangos Pterneas, whose code I was inspired by.

History

  • 10/19/2014: Updated main tool, included an option to choose between two types of extracted data, IR or Depth images.
  • 10/16/2014: Updated main tool to extract the timestamps, and save them in a .mat file, called FrameTimings.mat. Also, the MatWriter class is changed, so the name of the variable can be set as an input.
  • 9/17/2014: Updated main tool to extract the instrinsic parameters of the Kinect v2 camera, and save it to "Intrinsic parameters.txt". In addition, a small MatLab script is added, which converts the depth frames to Cartesian coordinates and displays the point cloud.
  • 9/16/2014: First upload, tool can export .mat files from Kinect v2 stream.

Feel free to contact me with any problems or questions.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)


Written By
Student
Denmark Denmark
I'm a student studying Electronics and IT, and currently doing my final year of the masters degree in Vision, Graphics and Interactive Systems, specializing in computer vision.

Comments and Discussions

 
GeneralRe: Works perfect for Depth and IR!! RGB? Pin
Member 115031376-Mar-15 0:31
Member 115031376-Mar-15 0:31 
GeneralRe: Works perfect for Depth and IR!! RGB? Pin
SergentMT9-Mar-15 2:26
SergentMT9-Mar-15 2:26 
GeneralRe: Works perfect for Depth and IR!! RGB? Pin
Member 1234592523-Feb-16 1:35
Member 1234592523-Feb-16 1:35 
GeneralRe: Works perfect for Depth and IR!! RGB? Pin
SergentMT24-Feb-16 22:59
SergentMT24-Feb-16 22:59 
QuestionFile Exporter Tool for Kinect 1.8 ( for old kinect) Pin
Member 1130641523-Jan-15 3:46
Member 1130641523-Jan-15 3:46 
AnswerRe: File Exporter Tool for Kinect 1.8 ( for old kinect) Pin
SergentMT13-Feb-15 11:46
SergentMT13-Feb-15 11:46 
Questionproblem with VS app Pin
Member 1130183811-Dec-14 4:53
Member 1130183811-Dec-14 4:53 
AnswerRe: problem with VS app Pin
SergentMT11-Dec-14 8:11
SergentMT11-Dec-14 8:11 
Hi Adam
Ah okay, I actually experienced some of the same, when I ran it on a mid-range CPU.
I think actually it's because of the heavy bitswapping in the code, it's unfortunate, but I'm afraid that there's not so much to do about it (at least not for now), because the data comming from the Kinect has the wrong endianness for the .mat format (to my knowledge!) - so it needs to be swapped, which is a pretty heavy task, when the CPU has to do it for 25 images a second. Smile | :)

I just shut the program brute force, and my data was there. I'm sorry for the inconinience, but I can't fix it at the moment, sorry Smile | :)

Good luck with it.

BR

Mikkel
QuestionIt Works!!! Pin
Member 1130021010-Dec-14 2:42
Member 1130021010-Dec-14 2:42 
AnswerRe: It Works!!! Pin
SergentMT11-Dec-14 7:56
SergentMT11-Dec-14 7:56 
GeneralRe: It Works!!! Pin
Member 113585997-Jan-15 2:44
Member 113585997-Jan-15 2:44 
GeneralRe: It Works!!! Pin
SergentMT13-Feb-15 11:44
SergentMT13-Feb-15 11:44 
GeneralMy vote of 5 Pin
jeanmichel615-Oct-14 13:14
jeanmichel615-Oct-14 13:14 
QuestionAwesome! Pin
jeanmichel615-Oct-14 12:23
jeanmichel615-Oct-14 12:23 
AnswerRe: Awesome! Pin
SergentMT15-Oct-14 20:34
SergentMT15-Oct-14 20:34 
GeneralRe: Awesome! Pin
jeanmichel616-Oct-14 3:42
jeanmichel616-Oct-14 3:42 
GeneralRe: Awesome! Pin
SergentMT16-Oct-14 5:22
SergentMT16-Oct-14 5:22 
SuggestionRe: Awesome! Pin
jeanmichel616-Oct-14 12:24
jeanmichel616-Oct-14 12:24 
GeneralRe: Awesome! Pin
SergentMT17-Oct-14 3:52
SergentMT17-Oct-14 3:52 
GeneralRe: Awesome! Pin
jeanmichel617-Oct-14 5:57
jeanmichel617-Oct-14 5:57 
GeneralRe: Awesome! Pin
SergentMT19-Oct-14 3:45
SergentMT19-Oct-14 3:45 
GeneralHi again! Pin
jeanmichel630-Apr-15 12:50
jeanmichel630-Apr-15 12:50 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.