Detect and Track Objects in Live Webcam Video based on Color and Size by C#

• Download source - 229.14 KB

Introduction

You can select a color in real time and it tracks that color object and gives you the position. I use aforge library for that. I also used .NET Framework 4.0. It is a C# desktop application, it can take up to 25 frames per second. You can change color size any time you want, the color of drawing point will also change.

Background

I saw a very interesting project in CodeProject name making of lego pit camera. With the help of this project, I thought a realtime tracker could be made where color and objects size could also be changed in real time. and can draw movement of that objects in a bitmap with that color. I use some part of their code, although I use separate color filter for more accuracy.

Using the Code

The steps of work are very simple:

1. Take videoframe from webcam
2. Use filtering by given color (here eucladianfiltering used)
3. Make greyscale
4. Find objects by given size
5. Find biggest object
6. Draw object position in bitmap

First, how my software works. It can start tracking by default color black and color range of 120 but you can change in real time to color change press that a color dialog will come, then select color, remember to press OK and see the solidcolor box that your required color is present or not.

To select the range and define minimum size of the object, see the next image, there are 3 numeric updown list for that. objectsize calculated in pixel now about the views:

There are 4 views, 3 of them are Aforge video source control and another one is a picturebox, the first one shows normal video, the second one shows all detected objects, the third box shows only the biggest object and the fourth one just draws a ’0' sign in biggest object location. See the below image for a clearer view:

I also added a richtextbox which shows the pixel of biggestrectangle position. Now about the code for connection, I use normal aforge connection code.

For that, you need to download aforge DLL, and also add a very good videocontrol for showing video you can add it in your toolbox. Just click freespace on toolbox chooseitems-> browse the DLL -> add the control.

using AForge;
using AForge.Video.DirectShow;

then initialize:

  videoDevices = new FilterInfoCollection(FilterCategory.VideoInputDevice);
               if (videoDevices.Count == 0)
                    throw new ApplicationException();
               foreach (FilterInfo device in videoDevices)
                {
VideoCaptureDevice videoSource = new VideoCaptureDevice(device.MonikerString);
 videoSource.DesiredFrameSize = new Size(320, 240);
 videoSource.DesiredFrameRate = 15;
videoSourcePlayer1.VideoSource = videoSource;
 videoSourcePlayer1.Start();
               }

For videoplayercontrol, add a event in its misc named:

 private void videoSourcePlayer1_NewFrame(object sender, ref Bitmap image)
        {//work with image
}

For understanding how filtering works, I will explain using a picture of a jellyfish. Here, I take a red center color for understanding. In my provided software, the user can choose his request color and size.

In that function, I apply my color detection code by writing Euclidean filtering and then use blobcounter to extract data. Here, I am explaining it.

So see EuclideanColorFiltering work, first see:

We are going to apply a color filter. It is very simple code to use euclideanfiltering:

// create filter
EuclideanColorFiltering filter = new EuclideanColorFiltering( );
// set center colol and radius
filter.CenterColor = Color.FromArgb( 215, 30, 30 );
filter.Radius = 100;
// apply the filter
filter.ApplyInPlace( image );

Now see the effect:

Well to understand how it works, look closely at the code:

filter.CenterColor = Color.FromArgb( 215, 30, 30 );
filter.Radius = 100;

The first line selects the select color value. You all know color has a value 0 to 255.
By filter.CenterColor = Color.FromArgb( 215, 30, 30 ); I specified my center color will be a red effected color because here value of red is 215, green and blue is 30, and filter.Radius = 100 means that all color value near 100 in my specified color.
Now my filter filters pixels, which color is inside/outside of RGB sphere with specified center and radius – it keeps pixels with colors inside/outside of the specified sphere and fills the rest with specified color.

Now for detect objects, I use bitmap data and use lockbits method to understand clearly this method see here. Then, we make it greyscale algorithom, then unlock it.

BitmapData objectsData = image.LockBits(new Rectangle
	(0, 0, image.Width, image.Height),ImageLockMode.ReadOnly, image.PixelFormat);
            // grayscaling
            UnmanagedImage grayImage = 
		grayscaleFilter.Apply(new UnmanagedImage(objectsData));
            // unlock image
            image.UnlockBits(objectsData);

Now for the object, we use blobcounter. It is a very strong class that aforge provided.

  blobCounter.MinWidth = 5;
  blobCounter.MinHeight = 5;
  blobCounter.FilterBlobs = true;
  blobCounter.ProcessImage(grayImage);
  Rectangle[] rects = blobCounter.GetObjectRectangles();
  foreach(Rectangle recs in rects)
  if (rects.Length > 0)
   {
      foreach (Rectangle objectRect in rects)
   {

      Graphics g = Graphics.FromImage(image);

    using (Pen pen = new Pen(Color.FromArgb(160, 255, 160), 5))
                    {
                        g.DrawRectangle(pen, objectRect);
                    }

                    g.Dispose();
                }
            }

blobCounter.MinWidth and blobCounter.MinHeight define the smallest size of the object in pixel, and blobCounter.GetObjectRectangles() returns all the objects rectangle position, and using graphics class, I draw rectangle over the images.

Now, if you want to take only the biggest object, there is a method for that:

   blobCounter.MinWidth = 5;
   blobCounter.MinHeight = 5;
   blobCounter.FilterBlobs = true;
   blobCounter.ObjectsOrder = ObjectsOrder.Size;
  blobCounter.ProcessImage(grayImage);
  Rectangle[] rects = blobCounter.GetObjectRectangles();
  foreach(Rectangle recs in rects)
    if (rects.Length > 0)
            {
                   Rectangle objectRect = rects[0];
                    Graphics g = Graphics.FromImage(image);
                    using (Pen pen = new Pen(Color.FromArgb(160, 255, 160), 5))
                    {
                        g.DrawRectangle(pen, objectRect);
                    }
                    g.Dispose();
                           }

Now image is like that:

But if you want to extract, you can use the following code:

Bitmap bmp = new Bitmap(rects[0].Width, rects[0].Height);
Graphics g = Graphics.FromImage(bmp);
g.DrawImage(c, 0, 0, rects[0], GraphicsUnit.Pixel);

So you will get your object like that:

For drawing the bitmap, I had to use threading. The thread was called in videosurecplayer event where I called the thread.

 void p(object r)
   {
       try
       {//create image
       Bitmap b = new Bitmap(pictureBox1.Image);
       Rectangle a = (Rectangle)r;
       Pen pen1 = new Pen(Color.FromArgb(160, 255, 160), 3);
       Graphics g2 = Graphics.FromImage(b);
       pen1 = new Pen(color, 3);
       SolidBrush b5 = new SolidBrush(color);
        Font f = new Font(Font, FontStyle.Bold);
       g2.DrawString("o", f, b5, a.Location);//draw the position
       g2.Dispose();
       pictureBox1.Image = (System.Drawing.Image)b;
       this.Invoke((MethodInvoker)delegate
           {
               richTextBox1.Text = a.Location.ToString() + 
				"\n" + richTextBox1.Text + "\n"; ;
           });
       }
       catch (Exception faa)
       {
           Thread.CurrentThread.Abort();
       }
      Thread.CurrentThread.Abort();
   }

I have to use invoke to write in textbox because of cross threading. To send rectangle object, I use parameterized thread. The thread 'aa' called the function "p()" to draw at bitmap and write in textbox.

 if (rects.Length > 0)
            {
                Rectangle objectRect = rects[0];

                // draw rectangle around derected object
                Graphics g = Graphics.FromImage(image);

                using (Pen pen = new Pen(Color.FromArgb(160, 255, 160), 5))
                {
                    g.DrawRectangle(pen, objectRect);
                }
              g.Dispose();
                int objectX = objectRect.X + objectRect.Width / 2 - image.Width / 2;
                int objectY = image.Height / 2 - (objectRect.Y + objectRect.Height / 2);
                ParameterizedThreadStart t = new ParameterizedThreadStart(p);
               Thread aa = new Thread(t);
               aa.Start(rects[0]);               
            }  

Points of Interest

Parameterized threading, cross threading has to implement here, also there is a need to apply invoke method for write using good threading, we can increase the framerate. About filtering Euclidian filtering is more accurate than HSLfiltering or Colorfiltering, yrbrcr filtering.

Thanks to Andrew Kirillov for his help.

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)