Click here to Skip to main content
Click here to Skip to main content

How to use video and sound information to detect intruder

By , 19 Sep 2011
Rate this:
Please Sign up or sign in to vote.
intruderdetection2.png

intruderdetection1.png

Introduction

This article shows you how to use video and sound information to detect an intruder. Image data and sound data are continually collected from the environment respectively through webcam and microphone. Once the conditions of surrounding environment is changing, the corresponding alarm will be raised immediately.

The package contains:

  • DirectShowLib: Manipulate the webcam, grab images. I have wrapped functions related to IntruderDetection in class CManipulateWebcam. For more information about DirectShowLib, please see directshow.net.
  • ManipulateMicrophone: Manipulate the microphone, grab sound, play sound. Functions related to IntruderDetection are wrapped in class CManipulateMicrophone. For more information about ManipulateMicrophone, please see the part "Using the code" of the article: Sending and playing microphone audio over network.
  • WavStream: Read .wav file, save data into .wav file. The class CManipulateMicrophone uses WavStream.dll to save sound data into .wav file. The following code is useful:
    WaveStreamWriter wavwrite = WaveStreamWriter
    	(FileName, SamplingRate, Channels, BitPerSample);
    // SoundBuffer is an array which contains the sound data you want to save
    wavwrite.Write(SoundBuffer, BufferLength);
  • IntruderDetection: The main application needs (DirectShowLib.dll, ManipuateMicrophone.dll, WavStream.dll).
  • DirectShowLib, ManipulateMicrophone and WavStream are independent projects. You can use DirectShowLib in your application to interact with webcam. Use ManipulateMicrophone to interact with microphone. Use WavStream to interact with .wav file. Of course, use IntruderDetection to detect intruder.

How Does the IntruderDetection Work

  • Video detection: Check the box "Video Detection" to open video detection. If the box "Video Alarm" is checked, the alarm will be activated. Once there is something, abnormal information (0-3 second) will be automatically recorded in the folder: IntruderDetection>>bin>>Debug>>image>>(yyyy-MM-dd-H), filename (mm-ss-ff).jpg
  • Sound detection: Check the box "Sound Detection" to open sound detection. If the box "Sound Alarm" is checked, the alarm will be activated. Check the box "Sound Echo" to play the sound from microphone. In the same way, abnormal information (0-3 second) will be automatically recorded in the folder:
    IntruderDetection>>bin>>Debug>>sound>>(yyyy-MM-dd-H), filename (mm-ss-ff).wav

Background knowledge of IntruderDetection

Video detection

For video, define the distance of two image a(i,j) (ra,ga,ba) and  b(i,j)(rb,gb,bb) as

D1=((|ra-rb|+|ga-gb|+|ba-bb|)/3)  i=1,2...; j=1,2,... or 

D2=((|ra-rb|2+|ga-gb|2+|ba-bb|2)/3)  i=1,2...; j=1,2,...  or

D3=max((|ra-rb|+|ga-gb|+|ba-bb|)/3)  i=1,2...; j=1,2,...

If the D1,D2 or D3 is bigger than a threshold value, we believe something is happening.  For example:

private bool CompareImage(Bitmap a, Bitmap b)
{
    int width = b.Width;
    int height = b.Height;
    BitmapData data_a = a.LockBits(new Rectangle(0, 0, width, height), 
			ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
    BitmapData data_b = b.LockBits(new Rectangle(0, 0, width, height), 
			ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
    unsafe
    {
        byte* pa = (byte*)data_a.Scan0;
        byte* pb = (byte*)data_b.Scan0;
        int offset = data_b.Stride - width * 3;
        double Ra, Ga, Ba, Rb, Gb, Bb;
        double temp1 = 0, temp2 = 0;
        double max = 0;
        for (int y = 0; y < height; y++)
        {
            for (int x = 0; x < width; x++)
            {
                Ra = (double)pa[2];
                Ga = (double)pa[1];
                Ba = (double)pa[0];
                Rb = (double)pb[2];
                Gb = (double)pb[1];
                Bb = (double)pb[0];
                temp1 = (Math.Abs(Ra - Rb) + Math.Abs(Ga - Gb) + Math.Abs(Ba - Bb)) / 3;
                temp2 += temp1;
                if (temp1 > max)
                {
                   max = temp1;
                }
                pa += 3;
                pb += 3;
            }
                pa += offset;
                pb += offset;
        }
        temp2 = temp2 / (height * width);
        a.UnlockBits(data_a);
        b.UnlockBits(data_b);
        if (max > 200 || temp2 > 5)
        {
            return true;
        }
        else
        {
            return false;
        }
    }
} 

In the default case, pointer is not allowed in C#. You need to open the Properties of IntruderDetection, click Build, check the box "Allow unsafe code". You may adjust the threshold values to adapt your machine.

Sound Detection

Calculate the sound energy every 25 millisecond. If the energy is bigger than a threshold value, something is happening.

Conclusion

This article shows how to use video and sound information to detect intruder (dig into IntruderDetection). In this process, some useful intermediate technology is used, including how to manipulate webcam (dig into DirectShowLib), how to manipulate microphone (dig into ManipulateMicrophone), how to interact with .wav file (dig into WavStream). Have fun!

History

  • 14th September, 2011: Initial version of the article

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Calinyara

China China
E-mail:mr.dengjie@gmail.com
 
http://www.linkedin.com/in/ddqre
 
http://www.facebook.com/people/Jie-Deng/100001707144205

Comments and Discussions

 
GeneralMy vote of 5 Pinmember Gun Gun Febrianza4-Jun-13 8:46 
QuestionMinimize form on click Pinmembertr4pz24-Jan-13 21:33 
Questionhum PinmemberMember 77393663-Nov-11 4:00 
GeneralMy vote of 5 PinmemberRene Pilon20-Sep-11 12:55 
GeneralRe: My vote of 5 PinmemberRene Pilon20-Sep-11 13:00 
GeneralRe: My vote of 5 PinmemberCalinyara20-Sep-11 16:01 
QuestionDoor Bell or Pets Pinmemberirpug19-Sep-11 12:51 
AnswerRe: Door Bell or Pets PinmemberCalinyara19-Sep-11 20:56 
GeneralMy vote of 4 PinmemberSergio Andrés Gutiérrez Rojas16-Sep-11 9:46 
GeneralRe: My vote of 4 PinmemberCalinyara16-Sep-11 19:42 
Thanks for your suggestion.
Calinyara

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140415.2 | Last Updated 20 Sep 2011
Article Copyright 2011 by Calinyara
Everything else Copyright © CodeProject, 1999-2014
Terms of Use
Layout: fixed | fluid