|
Hi
I tested and the handles seem to be increasing, especially after playing rmvb files. I'm not so sure about the cause, and maybe it has something to do with the codecs, since the resources do not increse after playing some video types on my system. I'll check my code when I'm free.
For the second provlem, do you mean that you save I-frames from the cameras to video files and then period periodically save the single I-frame video to BMP file? I don't know much about your camera capturing framework, but since you can save I-frames to video files (this mean you have the frame data?), why don't you consider save the frame data to BMP files directly instead of reading it from video files? This seems to be a better idea, but it depends on your condition. Maybe you can describe your framework more specifically
旧日重来
|
|
|
|
|
Memory leak:
Thanks for looking into the problem with the memory/handles!
I have the problem with motion JPEG and MPEG-4 movies.
The second problem:
As it is now I have written code to connect to Network cameras through different protocols as RTP and TCP, and then stream MPEG-4 video from them.
My application has to be able to save the video in complete or to just save an I-frame with specified interval (in this case the P-frames becomes useless).
Furthermore, the application has to be able to convert the video frames to bitmaps. It's probably here my knowledge comes short: I don't know any other way to feed the graph but to extract a complete I-frame (with all the headers, so the frame by it self is playable) from the video stream and then feed the graph with the following source:
m_pGraph->AddSourceFilter(L"c:\\path_to_the_saved_iframe.mp4", L"Source", &m_pSrcFilter);
However, this means I have to teardown the graph and set it up again for each conversion, which takes lots of time and cpu!
There must be a better way to feed the graph! But I don't know how... I would like to be able to fill a buffer in the memory with the I-frame, specify the buffer as source filter, Run the graph and wait for the callback, where I take care of the decoded bitmap. And then when I need to convert next I-frame, just fill the buffer, and run the graph again...
But how can I add a source filter which is a memory buffer and not a physical file on disk?
About your suggestion to save the I-frame data to BMP directly, I'm not sure I understand this. The I-frame data is encoded as MPEG-4, so I don't know how to save this directly to BMP files. Therefore I use the filter graph to decode the MPEG-4 frame for me...
But if it can be done more directly, it would be nice!
Thanks for your effort!
|
|
|
|
|
I think I understand you now
As I am also a layman on DShow, I'm afraid I'm not able to give a perfect suggestion for you.
However, as I tested, this way may be faster, providing all the videos are in the same format (as you mentioned, they are MPEG4): After building graph and a new video comes, you can delete just the source filter from the graph (leaving the decoders and renderers), and add the new video using IGraphBuilder::RenderFile(), then the decoders and renderers used last time can be connected to the new source without creating them again. I tested in the GraphEdit, building a graph over again takes about 1 second or so, and using the existing decoders seems to be done at once. In this article[^] there is code showing how to enumerate a filter and delete it.
On the other hand, since you will only deal with MPEG4, maybe you can find a non-DShow MPEG4 decoder and use it directly without DShow? This may save much time since DShow is designed to play multiple media types so that it has to take much time on finding the right decoder.
旧日重来
|
|
|
|
|
Thank you very much for your help and your suggestion!
I don't know why, but in my code I don't gain very much.
This is what I do know, after I have set up the graph with AddSourceFilter:
<br />
hr = m_pControl->Run();
m_pEvent->WaitForCompletion(INFINITE, &evCode);
hr = m_pControl->Stop();
hr = m_pGraph->RemoveFilter(m_pSrcFilter);
<br />
int iii = 0;<br />
while(iii < ITERATIONS-1)<br />
{<br />
hr = m_pGraph->RenderFile(L"c:\\Tmp\\_convert.mp4", NULL);<br />
hr = m_pControl->Run();<br />
m_pEvent->WaitForCompletion(INFINITE, &evCode);<br />
hr = m_pControl->Stop();<br />
iii++;<br />
}<br />
So I use "m_pGraph->RemoveFilter" instead of enumerate and delete. Should be ok?
Here are the times it tok to make the frame grabbing with graph teardown and renderfile:
(For some reason, the first frame grabbing after removing source filter is real quick, why?)
Graph teardown:
Pic: 01, Time: 218, Interval: 218
Pic: 02, Time: 359, Interval: 141
Pic: 03, Time: 484, Interval: 125
Pic: 04, Time: 625, Interval: 141
Pic: 05, Time: 765, Interval: 140
Pic: 06, Time: 875, Interval: 110
Pic: 07, Time: 1015, Interval: 140
Pic: 08, Time: 1140, Interval: 125
Pic: 09, Time: 1281, Interval: 141
Pic: 10, Time: 1421, Interval: 140
AddSource, Remove Source and then Render file:
Pic: 11, Time: 1562, Interval: 141
Pic: 12, Time: 1593, Interval: 31
Pic: 13, Time: 1765, Interval: 172
Pic: 14, Time: 1890, Interval: 125
Pic: 15, Time: 2000, Interval: 110
Pic: 16, Time: 2125, Interval: 125
Pic: 17, Time: 2250, Interval: 110
Pic: 18, Time: 2359, Interval: 109
Pic: 19, Time: 2468, Interval: 109
Pic: 20, Time: 2578, Interval: 110
|
|
|
|
|
Well, actually I meant that the source filter shold be deleted every time when you save a frame. So the code may look like this:
hr = m_pControl->Run();
m_pEvent->WaitForCompletion(INFINITE, &evCode);
hr = m_pControl->Stop();
hr = m_pGraph->RemoveFilter(m_pSrcFilter);
<br />
int iii = 0;<br />
while(iii < ITERATIONS-1)<br />
{<br />
hr = m_pGraph->RenderFile(L"c:\\Tmp\\_convert.mp4", NULL);<br />
hr = m_pControl->Run();<br />
m_pEvent->WaitForCompletion(INFINITE, &evCode);<br />
hr = m_pControl->Stop();<br />
<br />
iii++;<br />
}
If you don't remove source filter every time, since there is a source filter in the graph, DShow actually build another whole filters for the new file to render. Even worse, the old filters were not removed, and the graph is now redering multiple files. And I think that's why the pic 12 took only 31ms, since before pic 12, the source filter was removed.
P.S.: You may use GraphEdit shipped with DShow to see the process visually.
旧日重来
|
|
|
|
|
Hello, Mingliang,
First of all, good job on implementing this. It really helps us a lot! Really appreciate your hard work.
I have two questions.
1. In the processFrame call back function, you say it will display a frame every .5 second. Is there a way to display every frame and without any delay in between?
2. For my project, I probably need some mouse interaction on the image. I see that you use staticText to display images. But there is no MouseMove and MouseUp event for StaticText. What can I do to improve this?
Thanks in advance for your helps!
Chi
TEST
|
|
|
|
|
Hi Shi,
Thanks for your interest.
For the first provlem, the project was designed to be an easy to use processing framework, so I did not consider much about the displaying (or previewing). If you do want display every frame, you may try to run the memcpy() and postmessage() stuffs in CVideoAnaDoc::ProcessFrame() everytime instead of waiting until the m_hUpdateEvent (and then the Timer and the event can be deleted). However, this is a lazy but not quite a good idea since it is low in performance. Using a DShow renderer is much better, but more complicated I guess.
For the second, you may set the "notify" property of the static control to "true" to enable the mouse events.
旧日重来
|
|
|
|
|
Hi everyone!
I wish to know how I can flip the frame at the previewbox. Is there any function that can do this? Please help!
|
|
|
|
|
Sorry for I didn't see your last message.
I hope you have resolved other problems you mentioned.
In CVideoAnaDoc::ProcessFrame() I have shown how to access a specific pexel, and once you can access a pixel, it's easy to move it to its position in the fliped image.
And here is some hint, you may move a line of pixels at a time, for pixels in the same line are stored continously in DIBits.
Also note that there may be padding BYTEs between lines, but this does not matter in your case
旧日重来
|
|
|
|
|
Thnx for your reply!In fact I didn't quite get how I should access lines. Can you show more specifically writing some code?
As far as the other issues are concerned, I still can't solve the problem with the memory. The program keeps allocating more and more memory! If you have any idea what might go wrong just let me know.I would fully appreciate it. I deallocate the memory every time I run each process by GlobalFree() command. I don't have any clue what might go wrong. Thnx again!
|
|
|
|
|
Well, as I noticed you may just want the image DISPALYED upside down, then you may just modify the StretchDIBits() in SPicBox::PaintDIB(). Modify the yDst and cyDst parameters to make a rectangle upside down.
And for your reference, the way to access lines. for the line with coordinate y, its first byte in DIBits is:
BYTE *pLine = pDIBits + (BMPHeight - y - 1) * nLineBytes;
while the number of bytes in a line (padded) is:
int nLineBytes = (BMPWidth * 24 + 31) / 32 * 4;
For the memory problem:
if (m_pImage->LoadImage(strFilename,&ProgCtrl)) {
m_pImage->AllocateDataMemory(); // Make sure the memory is freed before new allocate in AllocateDataMemory()
m_bDisplayImage = TRUE;
m_pVideoAnaView->m_PreviewBox.SetBih(&m_pImage->m_bmi.bmiHeader); // SetBih() could be called only once if the size of image does not change, and it's better not in CVideoAnaDoc::ProcessFrame() because of potential memory access conflict.
And in the WndProc of WM_USER_PREVIEW_FRAME2, the memory newed should be deleted.
And finally you may have to notice that it will be more convenience if you locate the codes that allocated the leaked memory, with VC++ debugger, than just guess the cause.
旧日重来
|
|
|
|
|
Thank you for your reply. However I still have a serious data access violation problem with integrating my processing code to your code. Though my software works perfectly with images when I try to implement it in your code then serious data access violation problems occur. What I try to do is save the frame into a file -using your code in ProcessFrame().Just after that I open the file using my LoadImage function and storing it to a look-alike BITMAPHEADER structure - p_mImage. So far so good.
When I pass this entity(frame) onto my processing function whose code is outside the ProcessFrame() body but inside VideoAnaDoc.cpp, then strange access violations occur.
As you mention in the ProcessFrame() body, someone has to be careful with access violations because the processing algorithm will run inside a thread.
How could I overcome the problem and make it work? The p_mImage entity is declared as global. Please I need your help on this! Thank you for your time! I really appreciate this!
|
|
|
|
|
I'm sorry for that
I did not see something wrong from your description, maybe you can paste some simplified version of your code to see. If you can find out at what point (the code and the target memory) that the access violation occur, it would be better.
Here is some advise that may help.
First check whether the image is saved correctly (say, can it be opened in an image viewer?), and whether the standalone version of the processing procedure work with the saved bmp.
You mentioned you "pass this entity(frame) onto my processing function", this may be the point. Did you call the processing method within ProcessFrame() and it would return before ProcessFrame() renturn?
Have you added the result preview? Comment those out in ProcessFrame() to try, since the preview is done in the main thread and may cause some problem.
旧日重来
|
|
|
|
|
My code inside the ProcessFrame() is this:
HRESULT CVideoAnaDoc::ProcessFrame(double SampleTime, BYTE *pBuffer, long nBufferLen)
{
// TODO: Put the frame processing code here
// Keep in mind that code here is executed within another thread,
// so do consider the data access problem among threads
// Here just do nothing but send a preview image and update progress view every 0.5 seconds
// The following code demonstrates how to save a snapshot to BMP file every 10 frames
if(0 == m_nCurFrame % 10)
{
CString strFilename;
strFilename.Format("C:\POULOS%d.bmp", m_nCurFrame / 1000);
FILE *pfSnap = fopen(strFilename, "wb");
fwrite(&m_Bfh, sizeof(m_Bfh), 1, pfSnap); // BITMAPFILEHEADER
fwrite(&m_Bih, sizeof(m_Bih), 1, pfSnap); // BITMAPINFOHEADER
fwrite(pBuffer, nBufferLen, 1, pfSnap); // DIBits
fclose(pfSnap);
if (m_pImage->LoadImage(strFilename,&ProgCtrl)) { //Loading image to m_pImage
m_pImage->AllocateDataMemory();
m_bDisplayImage = TRUE;
}
Smooth(m_pImage);
Segmentation(m_pImage);
}
m_pImage is a CBitmapImage type structure (briefly this is like a bitmap image with some extra features) and is declared in the "VideoAnaDoc.h" file and is linked to its class normally. That's why I save the frame and then load it with my function LoadImage(). I know that this is stupid code but due to the fact there is no correspondence between the two type I have to save and load the frame. Anyway, after Smooth() runs correctly, the segmentation() crashes with access violation.
In my program, Smooth(m_pImage) and Segmentation(m_pImage) run one after the other in the same way and they work. I really don't understand how Smooth(m_pImage) and Segmentation(m_pImage) can't! How could I write the code in such a way that ProcessFrame() , just saves and load the frame and then get out from its body,go to the Smooth(), run, finish, go to Segmentation(), run, finish and the go back to ProcessFrame() and repeat over and over again until no frame is available. I don't know if you get this but I think that running multiple threads inside a thread this might cause the problem.
Thank for your great help! I really really appreciate this!
|
|
|
|
|
Well, I think it should not cause data access violation dependng the code you paseted
Here is some advise that I can image:
1. You mentioned: "running multiple threads inside a thread". Did you mean that inside Smooth() or Segmentation() you create new threads (I hope you wrote chose codes or you have the source)? If yes, you'd better make sure the created threads ends before CVideoAnaDoc::ProcessFrame() returns (use something like WainforSingleObject()). If not, maybe the last frame is still being processed while new frame comes, and this would not happen if you just process a single image but happens when multiple images sequently.
2. There is one line m_bDisplayImage = TRUE; in your code. I suppose this means you want to display m_pImage (in the main window)? Since the display the image is within the main thread of the application (maybe somewhere else in CVideoAnaDoc but being member functinos for the same class does not mean they are in the same thread). If accessing m_pImage in the main thread (while displaying) and in the CVideoAnaDoc::ProcessFrame() thread (this is made by DShow, and the processing codes are also in this thread since you called them within CVideoAnaDoc::ProcessFrame()) at the same time, there may be some access violance. The safer way is to make a copy for displaying as I did (however if you don't like the sendmessage I used but want a varible used by both thread, you may consider techniques like Mutex or Semaphore).
旧日重来
|
|
|
|
|
Hi There!
The program rocks! I am in the procedure of adding my processing algorithm in the code. However I want to display my resulting processed image in a previewbox to the right of the existing previewbox. Any help?Thnx a lot!
|
|
|
|
|
I'm glad that you managed to make it work
Well, in my demonstration program, the preview window is not a very good way to "play" the video. My intent was to focus on providing some easy and efficient interface to add the processing code. As you have may noticed, in the demo, only one snapshot of 0.5 second (maybe many frames would be processd in 0.5 second) are displayed, and the preview is just suposed to be a sign that the framework is running.
However, if your purpose is just to see several snapshots (compared with showing each frame) of your processing result during processing, intigrating the preview of your result sounds a good idea Here is the way I display the previewing frame: In CVideoAnaDoc::ProcessFrame(), the image data is received from the DShow framework. I make a copy of it (for the receieved data was managed by DShow and should not used out of CVideoAnaDoc::ProcessFrame()), and send it to the main window through a WM_USER_PREVIEW_FRAME message. And then a WM_USER_UPDATE_PROGRESS is send with some extra informatino. Those messages are then processed in CVideoAnaView::WindowProc(). The image data are copied and managed (and displayed) in m_PreviewBox, and the copied data are deleted.
Before any frame can be displayed, the m_PreviewBox.SetBih() should be called to set the BITMAPINFOHEADER for the previewing BMP (since the recieved frame in CVideoAnaDoc::ProcessFrame() only contains DIBits, but not BITMAPINFOHEADER), this is done in CVideoAnaDoc::OnAnaStart() in my demo. For your convenience, you can use the same BITMAPINFOHEADER for the processing result as the original preview, providing you do not change the size and color depth of the frame image during processing.
旧日重来
|
|
|
|
|
Thnx again! I finally managed to display the resulting processed image to my secondary preview box.Quite cool!
1)I however want the code in ProcessFrame() not to save the image resized but save the original frame. How is this possible?
2)Secondary how can I flip the frame upside down in previewbox? Do you know?
3) I have a serious memory overflow problem with my program. The code inside ProcessFrame() is this:
HRESULT CVideoAnaDoc::ProcessFrame(double SampleTime, BYTE *pBuffer, long nBufferLen)
long unsigned int nDIBSize, nScanWidth;
// TODO: Put the frame processing code here
// Keep in mind that code here is executed within another thread,
// so do consider the data access problem among threads
// Here just do nothing but send a preview image and update progress view every 0.5 seconds
// The following code demonstrates how to save a snapshot to BMP file every 10 frames
if(0 == m_nCurFrame % 10)
{
CString strFilename;
strFilename.Format("C:\POULOS%d.bmp", m_nCurFrame / 1000);
FILE *pfSnap = fopen(strFilename, "wb");
fwrite(&m_Bfh, sizeof(m_Bfh), 1, pfSnap); // BITMAPFILEHEADER
fwrite(&m_Bih, sizeof(m_Bih), 1, pfSnap); // BITMAPINFOHEADER
fwrite(pBuffer, nBufferLen, 1, pfSnap); // DIBits
fclose(pfSnap);
if (m_pImage->LoadImage(strFilename,&ProgCtrl)) { //Loading image to m_pImage
m_pImage->AllocateDataMemory();
m_bDisplayImage = TRUE;
}
if(WAIT_OBJECT_0 == WaitForSingleObject(m_hUpdateEvent, 0) || m_nCurFrame == m_nTotalFrames - 1)
{
Segmentation(m_pImage); //MY PROCESS
nScanWidth = ((m_pImage->m_nWidth*PIXELSIZE+3)/4)*4;
nDIBSize = sizeof(BITMAPINFOHEADER)+nScanWidth*m_pImage->m_nHeight;
m_pVideoAnaView->m_PreviewBox.SetBih(&m_pImage->m_bmi.bmiHeader);
BYTE *pNewBuffer = new BYTE[nDIBSize];
memcpy(pNewBuffer, m_pImage->m_pDIBImage, nDIBSize); // Make a copy of DIBitsof processed frame
m_pVideoAnaView->m_PreviewBox2.SetBih(&m_Bih);
BYTE *pNewBuffer2 = new BYTE[nBufferLen];
memcpy(pNewBuffer2, pBuffer, nBufferLen); // Make a copy of DIBits of frame
m_pVideoAnaView->PostMessage(WM_USER_PREVIEW_FRAME2, (WPARAM)pNewBuffer2, nBufferLen);
m_pVideoAnaView->PostMessage(WM_USER_PREVIEW_FRAME, (WPARAM)pNewBuffer, nDIBSize);
m_pVideoAnaView->PostMessage(WM_USER_UPDATE_PROGRESS, (WPARAM)m_nCurFrame + 1);
}
//// The following code demonstrates how to get rgb values of a specified pixel
//// You can write a loop to examine all pixels
//// Keep in mind the pixel data is stored from bottom to top in pBuffer
//int x = 0;
//int y = 0;
//int nLineBytes = (m_Bih.biWidth * 24 + 31) / 32 * 4; // # of bytes per line
//BYTE *pLine = pBuffer + (m_Bih.biHeight - y - 1) * nLineBytes;
//BYTE *pPixel = pLine + 3 * x;
//BYTE B = *pPixel;
//BYTE G = *(pPixel + 1);
//BYTE R = *(pPixel + 2);
}
m_nCurFrame++; // m_nCurFrame indicates which frame is being processed
return S_OK;
}
After running the process for 100 frames the program occupies 800Mb RAM and keeps increasing!!!! I don't understand why this happens. Segmentation() function closes normally. I don't know if m_pImage is saved in memory and is not removed every time ProcessFrame() has finished a frame. Any hint will be more than welcome!Thanks again!
|
|
|
|
|
I am new in programming so I need to ask you :
1) What necessary libraries do I need to install in order to run your program? I have Windows Vista with Visual Studio 2005 and SDK installed. However, it seems to be a problem with DirectShow libraries, etc.
2) I want to combine your program with my code which processes images. I want to grab a frame from the video file, pass it to my processing code and when done, move to the next frame. Can you help a little bit on what I should be careful of before starting changing your code and my code? Please keep in mind I am new to programming, especially DirectShow. Thanx a lot!!
|
|
|
|
|
Thanks for your interest.
1). It seems that should work after you installed the new Platform SDK. Would you provide a few more details about the compiling problem so that I can guess the cause.
2). If not very strict (say, your application is just an experimental or research system), then put your image processing work in CVideoAnaDoc::ProcessFrame(), which is also demonstrated in my article. If this does not meet your demand, I'm afraid you should first understand the details on how my framework and doing some modification, and the modification just depends on your certain demand.
旧日重来
|
|
|
|
|
1)As far as the compiling is concerned I have installed .NET Framework SDK 2.0 . I am creating a new Empty C++ project (not Win32 Console Application), throw in the source files, header files and finally build the project. Then an fatal error occurs saying: "The "dshow.h" file does not exist".
I found the "dshow.h" file on the internet but then an error occurred: "The "ddraw.h" file does not exist". I found the "ddraw.h" file on the internet and another error occured with an Xlib.h file. This is like a chain reaction.
I don't know if I have to include some libraries in the project properties. However, I don't know which libraries and how to include.
2)As far as how to combine your code with mine, you are right .I need to go through your code thoroughly. basically I don't need your user interface part of your code, I just need the core-processing code. Which parts of your code should I have a look on it and which throw out?
Thank you very very much!!!
|
|
|
|
|
|
I try to use the code to grab Full HD resolution video frame. The tool is always crashed.
I found that if I remove NULL filter from Graph Manager, it works without any problem.
Adding null filter seems not the real problem. I found ConnectFilters() is the reason caused Graph Manager crashed.
I modified the code.
......
// PIN_DIRECTION PinDirThis;
PIN_DIRECTION PinDirThis = (PIN_DIRECTION)3; // <--- changed
if (PINDIR_OUTPUT == PinDirThis)
{
hr = pGraph->Connect(pOut, pIn);
if(!FAILED(hr))
{
pOut->Release();
break;
}
}
.....
After this, everything is all right even if adding a null filter.
I write another code to connect filters.
IPin *pSampleOutPin;
IPin *pInPin;
hr = m_pGrabberFilter->FindPin(L"Out",&pSampleOutPin);
hr = m_pNullRenderer->FindPin(L"In",&pInPin);
hr = m_pGraph->Connect(pSampleOutPin,pInPin);
pSampleOutPin->Release();
pInPin->Release();
I also find not all of video files can retrieve m_lTimeperFrame.
If m_lTimeperFrame is 0, you will get a 1/0 exception.
Thanks for your great code
|
|
|
|
|
Thank you very much for your advise.
I'm sorry I haven't test on enough types or videos. On the other hand, I didn't have so many video types to test
For the first problem, I write the "while" because some of the filters have multiple outputs, and without the "while" it may fail on some videos. It's strange that it fails on the Full HD resolution video. But, as you mentioned, it only crashes on connecting the SampleGrabber and NullRenderer, and since SampleGrabber only have one output, your code seems more suitable
For the second problem, if m_lTimeperFrame is not available, it becomes difficult to estimate the total number of frames Maybe I shall use the time information within the grabber callback to estimate that and update the progress bar...
Thanks again for your information, and I'll try to make my code more compatible.
旧日重来
|
|
|
|
|
Moim Hossain
Sr. Software Engineer
Onirban Orion Technologies.
|
|
|
|
|