|


Introduction
I was busy trying to write a motion detection algorithm in C#. To determine motion, I compared change values from a previous image by calculating the absolute value of the pixel difference. For doing this, grayscale is easier and probably faster.
Background
If MIT thinks using grayscale for motion detection is a good idea, then I am definitely on the right track with my motion detection algorithm. For a reference, see the Cog project.
I got the code for the image loop from here because I originally attempted to use SetPixel which is painfully slow.
Using the code
If you use this code in a multithreaded environment, be aware that the parameter image will be locked until this method unlocks it. This will raise an exception when another code attempts to lock the bitmap, such as a customer OnPaint. To avoid this, you can always make a copy of the bitmap.
Here is the entire method: public Bitmap processImage(Bitmap image){
Bitmap returnMap = new Bitmap(image.Width, image.Height,
PixelFormat.Format32bppArgb);
BitmapData bitmapData1 = image.LockBits(new Rectangle(0, 0,
image.Width, image.Height),
ImageLockMode.ReadOnly,
PixelFormat.Format32bppArgb);
BitmapData bitmapData2 = returnMap.LockBits(new Rectangle(0, 0,
returnMap.Width, returnMap.Height),
ImageLockMode.ReadOnly,
PixelFormat.Format32bppArgb);
int a = 0;
unsafe {
byte* imagePointer1 = (byte*)bitmapData1.Scan0;
byte* imagePointer2 = (byte*)bitmapData2.Scan0;
for(int i = 0; i < bitmapData1.Height; i++) {
for(int j = 0; j < bitmapData1.Width; j++) {
a = (imagePointer1[0] + imagePointer1[1] +
imagePointer1[2])/3;
imagePointer2[0] = (byte)a;
imagePointer2[1] = (byte)a;
imagePointer2[2] = (byte)a;
imagePointer2[3] = imagePointer1[3];
imagePointer1 += 4;
imagePointer2 += 4;
}
imagePointer1 += bitmapData1.Stride -
(bitmapData1.Width * 4);
imagePointer2 += bitmapData1.Stride -
(bitmapData1.Width * 4);
}
}
returnMap.UnlockBits(bitmapData2);
image.UnlockBits(bitmapData1);
return returnMap;
}
The PixelFormat is crucial. This format determines the layout of the bitmap data. This is why the literal 4 is hard-coded (actually, it is hard-coded because I am lazy). If you use a different PixelFormat, you will need to determine the layout and offset for those formats. BitmapData bitmapData1 = image.LockBits(new Rectangle(0, 0,
image.Width, image.Height),
ImageLockMode.ReadOnly,
PixelFormat.Format32bppArgb);
BitmapData bitmapData2 = returnMap.LockBits(new Rectangle(0, 0,
returnMap.Width, returnMap.Height),
ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
This code locks the bitmap and returns its image data. Since I am converting the entire image to grayscale, I lock the entire bitmap. It is possible to lock a smaller area. byte* imagePointer1 = (byte*)bitmapData1.Scan0;
byte* imagePointer2 = (byte*)bitmapData2.Scan0;
Pointer arithmetic is unsafe in C# :(. Too bad because it is fast! Although, it is strange getting Access Violation errors in C#. If you change the code, test thoroughly using Mathematics to make sure you don't overwrite any protected memory. Scan0 is the pointer to the first byte of the bitmap's data. for(int i = 0; i < bitmapData1.Height; i++) {
for(int j = 0; j < bitmapData1.Width; j++) {
The astute will immediately recognize this will only work if the bitmaps are the same size. And fortunately, this always is since this method will create the returnMap bitmap based on the input bitmap. An incorrect loop will also cause an Access Violation. imagePointer2[0] = (byte)a;
imagePointer2[1] = (byte)a;
imagePointer2[2] = (byte)a;
imagePointer2[3] = imagePointer1[3];
See the comments in the code snippet above, to understand the layout of the bitmap data. a = (imagePointer1[0] + imagePointer1[1] + imagePointer1[2])/3;
Average the three color components in the original bitmap. Keep the alpha the same, otherwise your grayscale will also cause undesired blending. imagePointer1 += 4;
imagePointer2 += 4;
Move forward 4 bytes: imagePointer1 += bitmapData1.Stride - (bitmapData1.Width * 4);
imagePointer2 += bitmapData1.Stride - (bitmapData1.Width * 4);
This is definitely the most confusing part. See this for a picture of what is happening. The width of a bitmap is actually the composed width and an unused buffer to pad the bitmap to be a multiple of 4. Width + buffer = stride. It works, so good enough for me. returnMap.UnlockBits(bitmapData2);
image.UnlockBits(bitmapData1);
return returnMap;
UnlockBits unlocks the bitmap data, probably a good thing to do.
Points of Interest
A good place to find answers to hard problems: Google.
People much smarter than me: MIT Cog Project.
Possible Errors
I use the bitmap data from bitmapData1 for both bitmapData1 and bitmapData2. This could lead to unexpected errors if the data were some how different. However, in testing, everything works fine. A lot more prudent coder would verify a lot of things before putting this sort of code into anything critical.
If there are any more errors, I am quite sure that the overly critical among you will gladly point them out as glaring criticisms.
| You must Sign In to use this message board. |
|
| | Msgs 1 to 25 of 30 (Total in Forum: 30) (Refresh) | FirstPrevNext |
|
 |
|
|
hello i am making application to compare two images using pointer here is the code i am facing difficulty in setting indexes of pointer plz have a look on code:
public int[] Cmp_image(Bitmap img_test, Bitmap img_real) { BitmapData bmTestData = img_test.LockBits(new Rectangle(0, 0, img_test.Width, img_test.Height), ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
BitmapData bmRealData = img_real.LockBits(new Rectangle(0, 0, img_real.Width, img_real.Height), ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb); /// int[] check = new int[(bmRealData.Width * bmRealData.Height)];
try { hold = new int[(bmTestData.Width - bmRealData.Width) * (bmTestData.Height - bmRealData.Height) + 1];
} catch { MessageBox.Show("Real image must be smaller than Test Image"); }
unsafe { byte* ptrR = (byte*)bmRealData.Scan0; byte* ptrT = (byte*)bmTestData.Scan0;
int remainR = bmRealData.Stride - bmRealData.Width * 3; int remainT = bmTestData.Stride - bmTestData.Width * 3;
for (x = 0; x <= bmTestData.Width; x++) { for (y = 0; y <= bmTestData.Height; y++) {
hold[x * y] = 0; //set to zero
for (i = 0; i < bmRealData.Width; i++) { for (j = 0; j { // add image comparision code here hold[x * y] = Math.Abs(ptrR[0] - ptrT[0]) + Math.Abs(ptrR[1] - ptrT[1]) +Math.Abs(ptrR[2] - ptrT[2]);
} }
plz tell me about indexing of ptrRand ptrT for complete checking hope you understand
thanks in advance
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
I don't know your particular error, however, with pointer arithmetic you do need to advance your pointers by the appropriate size. In the case of a 32bit image you need to advance by 4 locations every time. Notice in my code the lines:
imagePointer1 += 4; imagePointer2 += 4;
Which your for loop neglects.
Need a C# Consultant? I'm available.
Happiness in intelligent people is the rarest thing I know. -- Ernest Hemingway
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
 |
|
|
I copied this article from my working code. Looking at the documentation, http://msdn2.microsoft.com/en-us/library/system.drawing.imaging.imagelockmode.aspx, I am left to assume that the lock is for concurrency sake. However, I think you are right. I am really glad you read my article enough to notice such a mistake.
Need a C# Consultant? I'm available.
Happiness in intelligent people is the rarest thing I know. -- Ernest Hemingway
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Simply cache the stride offset instead of calculate it every time.
So at the bottom of the inner for loop you should have:
//4 bytes per pixel imagePointer1 += strideOffset; imagePointer2 += strideOffset;
And you should define the strideOffset above the loops like so:
int strideOffset = bitmapData1.Stride - (bitmapData1.Width * 4);
I used the performance tool of Visual Studio 2005 and instrumented the test and found this works 3 times faster.
--Matt
|
| Sign In·View Thread·PermaLink | 4.80/5 (2 votes) |
|
|
|
 |
|
|
 |
|
|
I read the article you referred to about bitmap layouts and how stride works. If what the article is correct about the stride only being a buffer so that the bitmap width is a multiple of 4 then for a PixelFormat.Format32bppArgb bitmap with AlphaRGB you really shouldn't need to adjust the pointers for stride because any width will be a multiple of 4 with an AlphaRGB. Is this correct?
-- modified at 7:18 Thursday 21st September, 2006
Kenji
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
There is a diargram available online displaying the stride as a picture with an excellent documentation.
http://www.codersource.net/csharp_image_Processing.aspx
It is better safe to use the stride than to assume it to be a certain valye. There is no gaurantee of receiving a 24 bitmap and, in fact, there are many, excellent, game programming books that go into great detail regarding the differences if you want further reading.
On two occasions I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question. - Charles Babbage
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
 |
|
|
This is a good alternative using .NET 2.0. For the persons here who are using 1.0 or 1.1 that option does not exists.
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
I haven't benchmarked it, but using ColorMatrix should be much faster, I've used it in my own code to do simple greyscaling and other colour manipulations.
I can't find a nice chunk of my code to paste, so here's a link to a CodeProject article with some VB.NET. You should be able to port this to C# at about 1:1!
http://www.codeproject.com/vb/net/colormatrix.asp[^]
Regards, Ray
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Honestly I hadn't tested this before but upon test the following code yields around 8fps. Look at the code I pasted and let me know if I made a mistake but it looks like the unsafe version is still a lot faster: (converting grayscale only no motion detection) The unsafe version yields 24fps. Also I tried pulling the temp and cm variables out as static but saw no improvement.
float[][] temp = new float[][]{ new float[]{0.299f, 0.299f, 0.299f, 0f, 0f}, new float[]{0.587f, 0.587f, 0.587f, 0f, 0f}, new float[]{0.114f, 0.114f, 0.114f, 0f, 0f}, new float[]{0f, 0f, 0f, 1f, 0f}, new float[]{0f, 0f, 0f, 0f, 1f} }; ColorMatrix cm = new ColorMatrix(temp);
ImageAttributes ia = new ImageAttributes(); ia.SetColorMatrix(cm); Bitmap newMap = new Bitmap(image.Width, image.Height); Graphics g = Graphics.FromImage(newMap); g.DrawImage(image, new Rectangle(0,0,image.Width, image.Height), 0,0, image.Width, image.Height, GraphicsUnit.Pixel, ia); g.Dispose(); return newMap;
"Until the day of his death, no man can be sure of his courage" -- Jean Anouilh
|
| Sign In·View Thread·PermaLink | 2.00/5 (1 vote) |
|
|
|
 |
|
|
I'm quite supprised, but I did say I hadn't benchmarked it!
I guess the overhead of creating a new "output" image each time is quite high. In your 24fps code, do you do the same (create a new image from the colour source image) ?
Come to think of it, once you're in unsafe memory access mode, you're going to be very close to native machine speed for iterating through a block of memory (incrementing a pointer is going to be a very simple instruction in the underlying virtual machine and almost directly mappable to a single machine code instruction). Do you have reference C++ code?
What sizes are your images? I remember doing 25fps about 15 years back, but that was only 320x200 in 8 bit colour!
Have you exploited multi-core/-processor/-threading ?
I've made some very large gains in some of my apps lately by focusing on having a number of parallel threads (adjustable depending upon the machine hardware) do my work. E.g. if you have a dual-core, you may be able to process your images in two pieces and get 75%-85% speed increase.
Regards, Ray
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
The current images are 800x600 although after the grayscale conversion I will usually dump them to 32x32 or 64x64 which is still very accurate in the motion detection. Both sections of code create new images so that is not an issue. However, I assume new Image(existingImage) is very fast because memcpy is very fast.
Two processors could speed up the task but I don't know. My next step will be to see if I can blit to a directX buffer or OpenGl and apply a native video card op or matrix but retrieving could be slow. This would free the processor up for other tasks. (Actually if I rely on the video card I could blit both images and the XOR them for the difference)
Since I am downgrading the image to a smaller block (and not using anti-aliasing, long discussion another article, maybe) it would be possible to partition the image into blocks and assign each block to a "thread" and then wait for completion. It is definately an option, an appreciated one, but I think my threads for future use could be too important (pathing based on recognized motion vectors, still way off).
"Until the day of his death, no man can be sure of his courage" -- Jean Anouilh
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
I did some stuff with managed DirectX about six months ago whereby I transfered an image to video memory and later manipulated it in place (I needed a constantly modifying texture, every pixel was being changed in color/alpha value).
If the video card offers, exact term escapes me, fast-video-reading... I was getting something like:
~30fps on an Integrated Video card (in a Compaq laptop, old ATI Radion M) ~40fps on an Intel integrated video (Dell Gx270)
and ~350fps on my 7800GTX
Without the direct-reads, the Dell (with a Matrox G500) was giving me about 12fps.
All textures were ARGB-32 and 1000x1000!
I'll see if I can find the DX-CAP needed.
Regards, Ray
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Hello All,
Code Project has always been an inspiration for me, and I have learned a great deal from it. I do apologize for use of harsh words but that was the result of my disappointment which I experienced when I came across this posting.
Dirk Veldhuis, we sure are poor mortal beings and we all have our limits when it comes to thinking, knowledge and wisdom that is why this very portal exists that makes our jobs a little easier to work with, we should always be very careful to what we pass on so that we could save hundred of hours of man’s work all around the world. I am sure that all of you haven’t tested this code. Do me a favor, create a BMP with partly colored with RGB (255,0,0) and partly with RGB (85,85,85) which is (255+0+0)/3 and then turn it to grayscale using some image viewer such as Irfanview and see the difference.
Aparra, I m not sure what algorithm you have been using and how you came across to know the mean of RGB colors as the ‘defecto’ method but what Adobe has been following (http://www.fho-emden.de/~hoffmann/gray10012001.pdf) is the variations of:
0.30*R + 59*G + 0.11*B
Like, The_Myth said, life is short so we must make the best use of it, there are two type of people, ones who look at “what is said” and there are others who look at “why its said”, now one has to make a decision between the two to make best use of our life, Dear Abu Abdillah, I fully understand that the “blasting is not the spirit of this site” is not this site but lots of people “like me” use this site as a source of knowledge and I just felt like making the comment but I guess I should stick to “rate it” from now on because unfortunately I do not have much time to at my hand to answer such comments.
elynch, if you have searched the net and found this article most trivial then you haven’t searched hard enough or you need to polish your goggling skills. I would recommend you to visit the following link:
http://www.bobpowell.net/grayscale.htm
One other thing I will point out would be, not to use ‘unsafe code’ (pointers) as they ruin the spirit of .NET Framework (I hope there is nothing offensive in saying that).
Ayyaz-
|
| Sign In·View Thread·PermaLink | 5.00/5 (4 votes) |
|
|
|
 |
|
|
The method you provided .30*R + 59*G + 0.11*B would most certainly accomodate the difference in relative intensities of the different color spectrums, however, it costs too much. This method would require my compare in the motion detection to potentially compare three different values ( I don't think C# allows the cast from a 4byte array to an Int) and in doing the grayscale converion requires two extra multiplications. Since I am using an 800x600 sample in my motion detection that amounts to 2,400,000 extra ops per image.
As for unsafe code: I have only acheived 20fps in my motion detection algorithm I want more. Using SetPixel and GetPixel gives me about 1fpm.
"Until the day of his death, no man can be sure of his courage" -- Jean Anouilh
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
There is nothing in this world which is free - well yea looking at your limitations and your specific scenario I think you have done what was the best the option for you and i acknowledge that.
elynch wrote: ( I don't think C# allows the cast from a 4byte array to an Int)
Not sure about what you mean by 4 byte array but in C#, int is 4 Byte in size 
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
In c+++ the following is legal:
char* byteArray1 = new byte(256); //or is it square brackets? char* byteArray2 = new byte(256); int a = ((int)byteArray[4]); int b = ((int)byteArray[4]); if(a!=b){ //Fast comparison if only 1 difference is needed. See the assembly for strcpy for an even //better method }
Type safety prevents this in C#.
"Until the day of his death, no man can be sure of his courage" -- Jean Anouilh
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
Use C++/CLI and put the color conversion code in a #pragma unmanaged block and you'd see some performance gains.
Regards, Nish
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
When using unsafe code, type safety is not a matter. You can convert any kind of pointer to any other kind of pointer. So if you have a byte array you can do the following:
byte[] data = ...; // get data fixed (byte *pointer = data) { pointer += 128 // move to some position int *intPointer = (int *)pointer; //cast to int pointer
// now you can work with the int pointer and do your comparisons
// or convert it to a "real" int variable: int intValue = *intPointer; // Take care: assigning a (by-value) pointer to a variable causes the pointer value to be copied. So changing the data now will change the value of "*intPointer" but not the value of "intValue". But also, deleting the data[] now will leave the intValue intact while the intPointer becomes invalid.
Nice article by the way! I don't really care about grey scale conversion, just the unsafe stuff.
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
You just use those decimal values when computing the average, it will then be cast back to a byte so nothing should change in your motion detection routine, just the gray scale routine. But honestly, it shouldn't matter when detecting motion if your gray values are slightly "untrue" values.
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
 |
|
|
Well seeing that this is a place of learning, why don’t you explain why it’s so terrible or even better yet write an article and enlighten us poor mortal beings.
|
| Sign In·View Thread·PermaLink | 5.00/5 (6 votes) |
|
|
|
 |
|
|
This really is not certain correct, the color conversion is not a trivial task. The formula (r+g+b)/3 is "de facto" the standard of the conversion of RGB to grayscale.
Others way's to grayscale conversion:
Rec 601-1 (299*r+715*g+11*b)/1000 Rec 709 (213*r+715*g+72*b)/1000 Actually ITU standard: ITU (D65) (222*r+707*g+71*b)/1000
Remember, GDI+ can't manipulate "really" grayscale (8bits) images (PixelFormat.Format16bppGrayScale -> out of memory error)
Regards, Ángel.
|
| Sign In·View Thread·PermaLink | 4.00/5 (3 votes) |
|
|
|
 |
|
|
General News Question Answer Joke Rant Admin
|