differenciating moving objects in a motion detection system.

If you are serious about solving the problem of distinguishing between human, animal and vehicle, I would suggest you take a step back and consider adding additional sensors:

Infrared / thermal sensor -- do some research into infrared sensors and see if there is one that can give you temperature readings on objects at the distance you want to detect them at. Humans will read around 99 +/- degrees farenheit. Smaller animals will generally have higher temperatures, larger animals generally have lower temperatures. Internal combustion engines run a lot hotter.

Add a second camera to get steroscopic data so that you can derive distance to object. Given the distance to object, and the size it appears in your video, you can derive actual size of object which would give you a good first estimate of animal / human / vehicle, except small children might be confused with large dogs and motorcycles might be confused with humans.

Add radar, vehicles should give you a good radar return, animals an humans won't.

Sound might also help, at least for detecting internal combustion driven vehicles.

An actual practical solution will be a lot more reliable with additional sensor inputs.

If on the other hand you are just pursuing this as a theoretical exercise (which I suspect since you talk about using Matlab). The first thing to concentrate on is your actual motion detection algorithm:

SafarTimura is right, you need to take the difference between successive frames, to detect motion. But it isn't as simple as that. There will be minor variations in most pixels from one frame to the next. So you'll first want to experiment with how you take the "difference" -- you might compare raw pixels and just look at the pixels whose values change by greater than some threshold (which you either tune by experimenting, or maybe you dynamically determine the threshold for each difference you take). Or maybe you'll want to do edge detection on each frame and then take the difference between the edge detection results. You might do your edge detection or differences in each color (red, green, blue) separately, or as a greyscale difference.

Write the code to take the diffences and generate a new frame based on the differences -- then look at the "difference" output and adjust your threshold until you are getting something that looks to you like the thing that is moving (if you can't recognize the shape then there it's pretty unlikely you'll be able to come up with an algorithm to do so).

Once you've got a good motion detection algorithm that is not just detecting motion but giving you something that you can recognize as a shape, then you can proceed to the problem of distinguishing what is moving.

Distinguishing what is moving is fundamentally hard. You can try to match based on shape, but that may be very difficult since the moving object could be rotated at any angle from you and at any distance, and your motion detection algorithm is going to be throwing away a lot of data.

If you have stereoscopic data and can estimate depth, you can derive rough estimate of size which will give you a quick way of classifying large and small things.

You may find that after you've detected the motion, you need to go back to the original frame data and do edge detection on your original frame in the region of the motion, to extract a better idea of the shape.

No matter what you do, to get a good algorithm you are going to have to have hundreds of thousands of example inputs to test against. Typically you'd take a huge set of example inputs and determine the right answer for each of them by hand. Then you'd divide the input into half, and use one half to develop your algorithms / train your pattern recongition against. Once you've trained your algorithm, you'd use the second half of your test set to test against and see how you did.

It's a fun problem, just realize it's huge and would take quite a bit of resources to arrive at a practical solution.

You would probably have to identify some basic shapes - a human, a four-legged anmal, and a car/truck/motorcycle/bicycle/airplane/missile/godzilla, and then come up with some sort of matching algorythm that can determine what shape best matches the moving object.

I did a little bit of the theory in University.

Basically you need to find the differences between two frames of the video. (this gives you the movement between the two frames)

Secondly once you have the differences you can then do your pattern matching to try to see what has moved. (I.E. a person or object, tip would be to start very general and then refine). Do this by comparing the shape of the change between two frames. You might need to expand some objects to be identified over more than one frame. Obviously there are various ways of pattern matching. http://en.wikipedia.org/wiki/Object_recognition[^]

If you can't figure out what it was then add it to a list of undefined object and continue to the next frame. (if this occurs alot go back and look to see if this should be matched to your patterns for matching)

so just to recap, find changes, pattern match on changes, then learn if there is a missing pattern.

Hopefully this helps clear things up a bit.