Click here to Skip to main content
Click here to Skip to main content
 
Add your own
alternative version

Smoothing Kinect Depth Frames in Real-Time

, 24 Jan 2012
Removing noise from the Kinect Depth Frames in real-time using pixel filters and weighted moving average techniques.
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using Microsoft.Research.Kinect.Nui;

namespace KinectDepthSmoothing
{
    public partial class MainWindow : Window
    {
        private BitmapSource CreateSmoothImageFromDepthArray(ImageFrame image)
        {
            int width = image.Image.Width;
            int height = image.Image.Height;

            // We first want to create a simple array where each index represents a single pixel of depth information.
            // This will make it easier to work with the data to filter and average it for smoothing.
            short[] depthArray = CreateDepthArray(image);

            // Users can decide from the UI if this feature is applied
            if (useFiltering)
                depthArray = CreateFilteredDepthArray(depthArray, width, height);

            // Users can decide from the UI if this feature is applied
            if (useAverage)
                depthArray = CreateAverageDepthArray(depthArray);

            // After we have processed the data, we can transform it into color channels for final rendering.
            byte[] colorBytes = CreateColorBytesFromDepthArray(depthArray, width, height);


            return BitmapSource.Create(width, height, 96, 96, PixelFormats.Bgr32, null, colorBytes, width * PixelFormats.Bgr32.BitsPerPixel / 8);
        }

        private byte[] CreateColorBytesFromDepthArray(short[] depthArray, int width, int height)
        {
            // We multiply the product of width and height by 4 because each byte
            // will represent a different color channel per pixel in the final iamge.
            byte[] colorFrame = new byte[width * height * 4];

            // Process each row in parallel
            Parallel.For(0, 240, depthArrayRowIndex =>
            {
                // Process each pixel in the row
                for (int depthArrayColumnIndex = 0; depthArrayColumnIndex < 320; depthArrayColumnIndex++)
                {
                    var distanceIndex = depthArrayColumnIndex + (depthArrayRowIndex * 320);

                    // Because the colorFrame we are creating has four times as many bytes representing
                    // a pixel in the final image, we set the index to for times of the depth index.
                    var index = distanceIndex * 4;

                    // Map the distance to an intesity that can be represented in RGB
                    var intensity = CalculateIntensityFromDistance(depthArray[distanceIndex]);

                    // Apply the intensity to the color channels
                    colorFrame[index + BlueIndex] = intensity;
                    colorFrame[index + GreenIndex] = intensity;
                    colorFrame[index + RedIndex] = intensity;
                }
                
            });

            return colorFrame;
        }
    }
}

By viewing downloads associated with this article you agree to the Terms of Service and the article's licence.

If a file you wish to view isn't highlighted, and is a text file (not binary), please let us know and we'll add colourisation support for it.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Karl Sanford
Software Developer Open Systems Technologies
United States United States
First learned to program in 1997 on my TI-83 and have been doing it ever since, with a foray into networking and infrastructure.
 
Mostly a C# junky (Win\Web Forms, WP7.5/8, WPF and MVC), though I have experience with many other technologies and products.
 
I have also been trying to learn and apply more in the area of AI; focusing on computer vision, natural language processing, and classification.
 
In my spare time, I love to tinker with electronics and various useless DIY projects.
 
My brain is a shark... if it stops moving, it will die. I'm always looking to learn more.
Follow on   Twitter   LinkedIn

| Advertise | Privacy | Mobile
Web04 | 2.8.140821.2 | Last Updated 24 Jan 2012
Article Copyright 2012 by Karl Sanford
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid