Click here to Skip to main content
Click here to Skip to main content

DirectShow: Examples for Using SampleGrabber for Grabbing a Frame and Building a VU Meter

, 8 May 2009 CPOL
Rate this:
Please Sign up or sign in to vote.
DirectX.Capture class examples showing how to grab a frame from video and how to make a VU meter for audio
SampleGrabber video sample image

Introduction

This article is a follow-up of my previous articles:

Those articles describe how to do file saving for captured audio and video and to finetune a TV tuner. This article will explain how to use the SampleGrabber for audio and video. The first part shows how to grab a frame from a video stream, the second part shows the audio levels by making a VU meter!

Grabbing a Frame from a Video Stream

Background

DirectShow offers two basic methods to grab a frame or an image that is going to be rendered. The first method to grab a frame is the SampleGrabber method. Via SampleGrabber, a frame can be grabbed via a frame event or via GetCurrentBuffer. The second method involves use of VideoMixingRenderer or the BasicVideo interface by calling GetCurrentImage(). This method can be used with VMR or VMR9 for sure, and sometimes this might work for Video Renderer also.

The SampleGrabber code cannot be used if the capture device has a VP (Video Port) pin. Only video cards with a video capture device, such as Nivdia MX460 Vivo video card, will have such a pin. This is not a big concern because either SampleGrabber can be used via the capture pin or GetCurrentImage() can be used via the VMR that is connected with the VP pin to render the video.

In this example, the SampleGrabber method is used to grab a frame via a frame event. This method is used in most examples that are floating around on the Internet. Another advantage is that you have the choice to capture one frame or all frames. Most examples do not show what actions are really needed to get SampleGrabber working. This example shows what needs to be done.

The Code

First, I will give a description of the code changes that should be put in DirectX.Capture\Capture.cs. The function InitSampleGrabber adds the SampleGrabber filter to the graph and this function also initializes the media type it should be used for. This function should be called upon rendering Video for preview.

private bool InitSampleGrabber()
{
    // Get SampleGrabber
    this.sampGrabber = new SampleGrabber() as ISampleGrabber;

    if(this.sampGrabber == null)
    {
        return false;
    }

#if DSHOWNET
    this.baseGrabFlt    = (IBaseFilter)this.sampGrabber;
#else
    this.baseGrabFlt = sampGrabber as IBaseFilter;
#endif

    if(this.baseGrabFlt == null)
    {
        Marshal.ReleaseComObject(this.sampGrabber);
        this.sampGrabber = null;
    }
    AMMediaType media = new AMMediaType();

    media.majorType    = MediaType.Video;
    media.subType    = MediaSubType.RGB24;
    media.formatPtr = IntPtr.Zero;
    hr = sampGrabber.SetMediaType(media);
    if(hr < 0)
    {
        Marshal.ThrowExceptionForHR(hr);
    }
    hr = graphBuilder.AddFilter(baseGrabFlt, "SampleGrabber");
    if(hr < 0)
    {
        Marshal.ThrowExceptionForHR(hr);
    }
    hr = sampGrabber.SetBufferSamples(false);
    if( hr == 0 )
    {
        hr = sampGrabber.SetOneShot(false);
    }
    if( hr == 0 )
    {
        hr = sampGrabber.SetCallback(null, 0);
    }
    if( hr < 0 )
    {
        Marshal.ThrowExceptionForHR(hr);
    }
    return true;
}

It might be possible that the selected media type RGB24 is not usable. In such a case, modify the code. The following line of code shows how to get SampleGrabber in the graph upon rendering video:

#if DSHOWNET
hr = captureGraphBuilder.RenderStream(ref cat, ref med, videoDeviceFilter,
    this.baseGrabFlt, this.videoRendererFilter);
#else
hr = captureGraphBuilder.RenderStream(DsGuid.FromGuid(cat),
    DsGuid.FromGuid(med), videoDeviceFilter, this.baseGrabFlt,
    this.videoRendererFilter);
#endif

If GetCurrentBuffer would be used, then SetBufferSamples(true) should be called instead of SetBufferSamples(false). The function SetMediaSampleGrabber retrieves media-specific data and stores that data for later use. This function should be called upon initializing the preview window.

private int snapShotWidth = 0;
private int snapShotHeight = 0;
private int snapShotImageSize = 0;
private bool snapShotValid = false;

private void SetMediaSampleGrabber()
{
    this.snapShotValid = false;
    if((this.baseGrabFlt != null)&&(this.AllowSampleGrabber))
    {
        AMMediaType media = new AMMediaType();
        VideoInfoHeader videoInfoHeader;
        int hr;

        hr = sampGrabber.GetConnectedMediaType(media);
        if (hr < 0)
        {
            Marshal.ThrowExceptionForHR(hr);
        }

        if ((media.formatType != FormatType.VideoInfo) || (media.formatPtr ==
            IntPtr.Zero))
        {
            Throw new NotSupportedException(
                "Unknown Grabber Media Format");
        }

        videoInfoHeader = (VideoInfoHeader)Marshal.PtrToStructure(
            media.formatPtr, typeof(VideoInfoHeader));
        this.snapShotWidth = videoInfoHeader.BmiHeader.Width;
        this.snapShotHeight = videoInfoHeader.BmiHeader.Height;
        this.snapShotImageSize = videoInfoHeader.BmiHeader.ImageSize;
        Marshal.FreeCoTaskMem(media.formatPtr);
        media.formatPtr = IntPtr.Zero;
        this.snapShotValid = true;
    }

    if (!this.snapShotValid)
    {
        this.snapShotWidth = 0;
        this.snapShotHeight = 0;
        this.snapShotImageSize = 0;
    }
}

Keep in mind that if the media type changes, the number of bytes per pixel (stride) might change as well. The code for grabbing the frame might look like this:

/// <summary> Interface frame event </summary>
public delegate void HeFrame(System.Drawing.Bitmap BM);
/// <summary> Frame event </summary>
public event HeFrame FrameEvent2;
private    byte[] savedArray;
private    int    bufferedSize;

int ISampleGrabberCB.BufferCB(double SampleTime, IntPtr pBuffer,
    int BufferLen )
{
    this.bufferedSize = BufferLen;

    int stride = this.SnapShotWidth * 3;

    Marshal.Copy( pBuffer, this.savedArray, 0, BufferLen );

    GCHandle handle = GCHandle.Alloc( this.savedArray, GCHandleType.Pinned );
    int scan0 = (int) handle.AddrOfPinnedObject();
    scan0 += (this.SnapShotHeight - 1) * stride;
    Bitmap b = new Bitmap(this.SnapShotWidth, this.SnapShotHeight, -stride,
        System.Drawing.Imaging.PixelFormat.Format24bppRgb, (IntPtr) scan0 );
    handle.Free();
    SetBitmap=b;
    return 0;
}
/// <summary> capture event, triggered by buffer callback. </summary>
private void OnCaptureDone()
{
    Trace.WriteLine( "!!DLG: OnCaptureDone" );
}
/// <summary> Allocate memory space and set SetCallBack </summary>
public void GrapImg()
{
    Trace.Write ("IMG");
    if( this.savedArray == null )
    {
        int size = this.snapShotImageSize;
        if( (size < 1000) || (size > 16000000) )
            return;
        this.savedArray = new byte[ size + 64000 ];
    }
    sampGrabber.SetCallback( this, 1 );
}
/// <summary> Transfer bitmap upon firing event </summary>
public System.Drawing.Bitmap SetBitmap
{
    set
    {
        this.FrameEvent2(value);
    }
}

Most examples I saw did not release SampleGrabber-specific data. This code example should do that job properly... but errors might still occur. To get the SampleGrabber code working, the main program in CaptureTest\CaptureTest.cs must be modified too. First, the CaptureTest form needs to get two extra buttons and a PictureBox. In the code example, I added special code to add a small PictureBox. You can resize the form as well as resize and reposition the buttons and PictureBox yourself.

I made a very small PictureBox on purpose. I also added some code to hide the video file saving buttons and filename when SampleGrabber is put in the graph. I did this on purpose, as it gave me some free design space on the CaptureTest form. This also prevents a possible interaction with the video file saving functionality.

private void button1_Click(object sender, System.EventArgs e)
{
    this.capture.FrameEvent2 += new Capture.HeFrame(this.CaptureDone);
    this.capture.GrapImg();
}

private void CaptureDone(System.Drawing.Bitmap e)
{
    this.pictureBox1.Image=e;
    // Show only the selected frame ...
    // If you want to capture all frames, then remove the next line
    this.capture.FrameEvent2 -= new Capture.HeFrame(this.CaptureDone);
}

private void button2_Click(object sender, System.EventArgs e)
{
    if( (this.pictureBox1 != null)&&
        (this.pictureBox1.Image != null)&&
        (this.imageFileName.Text.Length > 0) )
    {
        this.pictureBox1.Image.Save(this.imageFileName.Text,
            System.Drawing.Imaging.ImageFormat.Bmp);
    }
}

Features are Made Optional

In the real code example, I added the new features as options. To use a new feature, the corresponding option needs to be selected first. The main reason for doing this is that a program sometimes failed at first use, due to one of the option settings. Now you can just change the option value and try again. There is one demand: a new value of an option becomes active upon (re)selecting the Audio or Video device. To get the options properly initialized, the function InitMenu() is added. This function should be called when a capture device is (re)selected.

private void initMenu()
{
    if (this.capture != null)
    {
        // Set flag only if capture device is initialized
        this.capture.AllowSampleGrabber =
            this.menuAllowSampleGrabber1.Checked;
        this.menuSampleGrabber1.Enabled =
            this.menuAllowSampleGrabber1.Checked;
        this.menuSampleGrabber1.Visible =
            this.menuAllowSampleGrabber1.Checked;
        this.capture.VideoSource = this.capture.VideoSource;
        this.capture.UseVMR9 = this.menuUseVMR9.Checked;
        this.menuUseDeInterlace1.Checked = this.FindDeinterlaceFilter(
            this.menuUseDeInterlace1.Checked);
    }
}
SampleGrabber with audio VU Meter

Show the Audio Levels via a VU Meter using the SampleGrabber

Background

When recording audio, the audio level needs to be set properly. I noticed that sometimes the audio volume was too low and sometimes the audio volume was too strong. The audible audio gives an idea about the volume of the captured audio, but that is not enough. The volume level for audible is usually set to a level the audio sounds great. For recording audio, that is just too vague. There are some interesting code examples published already, such as Analog and LED Meter, LED vu Meter User Control or LED Style Volume Meter using DirectX. One article shows fantastic looking VU meters, the next one shows a VU meter only and the last one was using DirectSound. These articles taught me how to make and use an User Control. These articles did not teach me how to get and analyze the audio, so that was what I had to figure out.

Furthermore I wanted to know the average and peak level of the captured audio. DirectShow or capture devices do not have an interface for that, so a filter is needed to analyze the audio and pass the audio levels to the program GUI. The filter could be a DMO filter, unfortunately the DMO output interface is very limited. The filter could be the SampleGrabber and this was a nice challenge for making a new code example!

During testing, I noticed that grabbing audio may affect the audio recording by just small ticks, probably caused by too much CPU time needed for grabbing and showing the audio. Especially the Hauppauge PVR150 showed me this problem. This problem sounds a little bit strange because for a video stream is usually far more data involved so there is far more chance of problems. I think that the higher sample rate for audio is the cause of the problem. For video a frame comes 25 or 30 times a second, but audio comes usually 44100 or 48000 times a second. I modified the DMO filter for this, but still that problem was showing up, so I assume it has nothing to do with a possible limitation of the SampleGrabber. The reason that this capture device gave problems might be because the audio buffer looks quite small but it might be a hidden problem also ... I tested this code with other TV cards (Pinnacle PCTV 310i, Pinnacle PCTV 330eV, SB!Live and other soundcards) those cards did not show problems. The main difference was the buffer size, it was larger ...

The Code

First, I will give a description of the code changes that should be put in DirectX.Capture\Capture.cs. The function InitSampleGrabber adds the SampleGrabber filter to the graph and this function also initializes the media type it should be used for. This function should be called upon rendering Audio for preview.

/// <summary>
/// SampleGrabber flag, if false do not insert SampleGrabber in graph
/// </summary>
private bool allowSampleGrabber = false;

/// <summary>
/// Check if usage SampleGrabber is allowed
/// </summary>
public bool AllowSampleGrabber
{
	get { return this.allowSampleGrabber; }
	set	{ this.allowSampleGrabber = value; }
}

/// <summary> Audio grabber filter interface. </summary>
private IBaseFilter audioGrabFlt = null;

/// <summary>
/// Null renderer for AudioGrabber in case off old TV card with
/// audio capturing via the soundcard. TV card is connected via
/// a wired connection with the soundcard.
/// </summary>
private IBaseFilter nullRendererFlt = null;

/// <summary>
/// Audio Grabber interface
/// </summary>
protected ISampleGrabber audioGrabber = null;

int ISampleGrabberCB.SampleCB( double SampleTime, IMediaSample pSample )
{
	Trace.Write ("Audio sample ...");
	return 0;
}

/// <summary>
/// Enable Sample Grabber event
/// </summary>
/// <param name="handler"></param>
public void EnableEvent(AudioFrame handler)
{
	if(this.audioGrabber == null)
	{
		return;
	}
	Trace.Write ("Init audio grabbing ...");

	if( this.savedArray == null )
	{
		this.savedArray = new short[ 50000 ];
	}
	//this.InitAudioGrabbing();

	if(this.audioGrabber == null)
	{
		return;
	}
	this.FrameEvent += new AudioFrame(handler);
	this.audioGrabber.SetCallback( this, 1 );
}

/// <summary>
/// Disable Sample Grabber event
/// </summary>
/// <param name="handler"></param>
public void DisableEvent(AudioFrame handler)
{
	if(this.audioGrabber == null)
	{
		return;
	}
	this.FrameEvent -= handler;
	this.audioGrabber.SetCallback(null, 0);
}

/// <summary> Interface frame event </summary>
public delegate void AudioFrame(short[] AS, int BufferLen);
/// <summary> Frame event </summary>
public event AudioFrame FrameEvent;
private	short[] savedArray;

int ISampleGrabberCB.BufferCB(double SampleTime, IntPtr pBuffer, int BufferLen)
{
	Marshal.Copy(pBuffer, this.savedArray, 0, BufferLen/2);
	this.FrameEvent(this.savedArray, BufferLen/2);
	return 0;
}

private bool InitAudioGrabber()
{
	if (!this.AudioAvailable)
	{
		// nothing to do
		return false;
	}

	if (!this.allowSampleGrabber)
	{
		return false;
	}

	this.DisposeAudioGrabber();

	int hr  = 0;

	// Get SampleGrabber if needed
	if(this.audioGrabber == null)
	{
		this.audioGrabber = new SampleGrabber() as ISampleGrabber;
	}

	if(this.audioGrabber == null)
	{
		return false;
	}

#if DSHOWNET
	this.audioGrabFlt	= (IBaseFilter)this.audioGrabber;
#else
	this.audioGrabFlt = audioGrabber as IBaseFilter;
#endif

	if(this.audioGrabFlt == null)
	{
		Marshal.ReleaseComObject(this.audioGrabber);
		this.audioGrabber = null;
	}

	AMMediaType media = new AMMediaType();

	media.majorType	= MediaType.Audio;
#if DSHOWNET
	media.subType	= PCM;
#else
	media.subType = MediaSubType.PCM;
#endif
	media.formatPtr = IntPtr.Zero;
	hr = this.audioGrabber.SetMediaType(media);
	if(hr < 0)
	{
		Marshal.ThrowExceptionForHR(hr);
	}

	hr = graphBuilder.AddFilter(audioGrabFlt, "AudioGrabber");
	if(hr < 0)
	{
		Marshal.ThrowExceptionForHR(hr);
	}

	hr = this.audioGrabber.SetBufferSamples(false);
	if( hr == 0 )
	{
		hr = this.audioGrabber.SetOneShot(false);
	}
	if( hr == 0 )
	{
		hr = this.audioGrabber.SetCallback(null, 0);
	}
	if( hr < 0 )
	{
		Marshal.ThrowExceptionForHR(hr);
	}

	return true;
}

private void SetMediaSampleGrabber()
{
	this.snapShotValid = false;
	if((this.audioGrabFlt != null)&&(this.AllowSampleGrabber))
	{
		AMMediaType media = new AMMediaType();
		media.formatType = FormatType.WaveEx;
		int hr;

		hr = this.audioGrabber.GetConnectedMediaType(media);
		if (hr < 0)
		{
			Marshal.ThrowExceptionForHR(hr);
		}
		if ((media.formatType != FormatType.WaveEx) || 
				(media.formatPtr == IntPtr.Zero))
		{
			throw new NotSupportedException
				("Unknown Grabber Media Format");
		}

		WaveFormatEx wav = new WaveFormatEx();
		wav = (WaveFormatEx)Marshal.PtrToStructure
				(media.formatPtr, typeof(WaveFormatEx));
		this.avgBytesPerSec = wav.nAvgBytesPerSec;
		this.audioBlockAlign = wav.nBlockAlign;
		this.audioChannels = wav.nChannels;
		this.audioSamplesPerSec = wav.nSamplesPerSec;
		this.audioBitsPerSample = wav.wBitsPerSample;
		Marshal.FreeCoTaskMem(media.formatPtr);
		media.formatPtr = IntPtr.Zero;
		this.snapShotValid = true;
	}

	if (!this.snapShotValid)
	{
		this.avgBytesPerSec = 0;
		this.audioBlockAlign = 0;
		this.audioChannels = 0;
		this.audioSamplesPerSec = 0;
		this.audioBitsPerSample = 0;
	}
}

private int avgBytesPerSec = 0;
private int audioBlockAlign = 0;
private int audioChannels = 0;
private int audioSamplesPerSec = 0;
private int audioBitsPerSample = 0;
private bool snapShotValid = false;

/// <summary>
/// Dispose Sample Grabber specific data
/// </summary>
public void DisposeAudioGrabber()
{
	if(this.audioGrabFlt != null)
	{
		try
		{
			this.graphBuilder.RemoveFilter(this.audioGrabFlt);
		}
		catch
		{
		}
		Marshal.ReleaseComObject(this.audioGrabFlt);
		this.audioGrabFlt = null;
	}

	if(this.audioGrabber != null)
	{
		Marshal.ReleaseComObject(this.audioGrabber);
		this.audioGrabber = null;
	}

	if(this.nullRendererFlt != null)
	{
		try
		{
			this.graphBuilder.RemoveFilter(this.nullRendererFlt);
		}
		catch
		{
		}
		Marshal.ReleaseComObject(this.nullRendererFlt);
		this.nullRendererFlt = null;
	}
	this.savedArray =  null;
}

For audio grabber, the SampleGrabber needs to be added to the graph, the following code will do that for a TV card with a audio and video capture device and the audio device gets the audio via the PCI bus (non wired audio).

// Special option to enable rendering audio via PCI bus
if((this.AudioViaPci)&&(audioDeviceFilter != null))
{
	cat = PinCategory.Preview;
	med = MediaType.Audio;
	if(this.InitAudioGrabber())
	{
		Debug.WriteLine("AudioGrabber added to graph.");
#if DSHOWNET
		hr = captureGraphBuilder.RenderStream
		     ( ref cat, ref med, audioDeviceFilter, this.audioGrabFlt, null );
#else
		hr = captureGraphBuilder.RenderStream(DsGuid.FromGuid(cat), 
		DsGuid.FromGuid(med), audioDeviceFilter, this.audioGrabFlt, null);
#endif
	}
	else
	{
#if DSHOWNET
		hr = captureGraphBuilder.RenderStream( ref cat, ref med, 
					audioDeviceFilter, null, null );
#else
		hr = captureGraphBuilder.RenderStream(DsGuid.FromGuid(cat), 
			DsGuid.FromGuid(med), audioDeviceFilter, null, null);
#endif
}
if( hr < 0 )
{
	Marshal.ThrowExceptionForHR( hr );
}

For those people who use an old TV card with wired audio must render audio for the SampleGrabber. Audio preview is not allowed in this scenario, it is already done via the normal soundcard. So the null renderer needs to be added. The following code will do that.

if((!this.AudioViaPci)&&(audioDeviceFilter != null)&&(this.audioGrabFlt != null))
{
	// Special scenario because normally no audio rendering is needed, however the
	// SampleGrabber must be inserted. So audio rendering is needed 
         // for this specific scenario. In addition the null renderer will be added.
	cat = PinCategory.Preview;
	med = MediaType.Audio;
	if(this.InitAudioGrabber())
	{
		this.nullRendererFlt = (IBaseFilter)new NullRenderer();
		hr = graphBuilder.AddFilter(this.nullRendererFlt, "Null Renderer");
		if(hr < 0)
		{
			Marshal.ThrowExceptionForHR(hr);
		}

		Debug.WriteLine("AudioGrabber added to graph.");
#if DSHOWNET
		hr = captureGraphBuilder.RenderStream( ref cat, ref med, 
			audioDeviceFilter, this.audioGrabFlt, this.nullRendererFlt);
#else
		hr = captureGraphBuilder.RenderStream(DsGuid.FromGuid(cat), 
		    DsGuid.FromGuid(med), audioDeviceFilter, this.audioGrabFlt, null);
#endif
	}
	if( hr < 0 )
	{
		Marshal.ThrowExceptionForHR( hr );
	}
}

First, I will give a description of the code changes that should be put in CaptureTest\CaptureTest.cs. This code will process the grabbed audio. For showing the audio output in led display, the audio data needs to be converted into an image. This can be done directly drawing rectangles in a specific color or via an user control. A user control draws the image also but it is a separate piece of code which can be used in other programs quite easily.. So, for showing the audio levels an user control is used. The user control used in this program is a simplified version of Gary Perkin's vuMeterLed implementation. My user control will just do what it supposed to do: show the audio levels without fancy borders. In Visual Studio 2005 I had problems in using the user control so I added the user control manually. In Visual Studio 2003, the user control can be added via the form. To get the user control working, some additional code is needed to make some controls (in)visible.

private bool sampleGrabber = false;

private bool SampleGrabber
{
	get { return this.sampleGrabber; }
	set
	{
		if ((this.capture != null) && (this.capture.AllowSampleGrabber))
		{
			this.sampleGrabber = value;
		}
		else
		{
			this.sampleGrabber = false;
		}
		this.menuSampleGrabber1.Checked = this.sampleGrabber;
		if(this.vuMeterLed1 == null)
		{
			this.vuMeterLed1 = new UserControl.VuMeterLed();
			this.vuMeterLed1.Anchor = 
				((System.Windows.Forms.AnchorStyles)
				((System.Windows.Forms.AnchorStyles.Bottom | 
				System.Windows.Forms.AnchorStyles.Left)));
			this.vuMeterLed1.Visible = false;
			this.vuMeterLed1.Location = new System.Drawing.Point
			(this.label2.Location.X + 49, this.label2.Location.Y);
			this.vuMeterLed1.Name = "vuMeterLed1";
			this.vuMeterLed1.Peak = 0;
			this.vuMeterLed1.Size = new System.Drawing.Size(100, 20);
			this.vuMeterLed1.TabIndex = 30;
			this.vuMeterLed1.Volume = 0;
			this.Controls.Add(this.vuMeterLed1);
		}
		if(this.vuMeterLed2 == null)
		{
			this.vuMeterLed2 = new UserControl.VuMeterLed();
			this.vuMeterLed2.Anchor = 
				((System.Windows.Forms.AnchorStyles)
				((System.Windows.Forms.AnchorStyles.Bottom | 
				System.Windows.Forms.AnchorStyles.Left)));
			this.vuMeterLed2.Visible = false;
			this.vuMeterLed2.Location = new System.Drawing.Point
			(this.label4.Location.X + 49, this.label4.Location.Y);
			this.vuMeterLed2.Name = "vuMeterLed2";
			this.vuMeterLed2.Peak = 0;
			this.vuMeterLed2.Size = new System.Drawing.Size(100, 20);
			this.vuMeterLed2.TabIndex = 31;
			this.vuMeterLed2.Volume = 0;
			this.Controls.Add(this.vuMeterLed2);
		}
		if (this.sampleGrabber)
		{
			this.button1.Visible = true;
			this.txtFilename.Visible = false;
			this.btnCue.Visible = false;
			this.btnStart.Visible = false;
			this.btnStop.Visible = false;
			this.label1.Visible = false;
			this.label2.Visible = true;
			this.label4.Visible = true;
			this.vuMeterLed1.Visible = true;
			this.vuMeterLed1.Enabled = true;
			this.vuMeterLed2.Visible = true;
			this.vuMeterLed2.Enabled = true;
		}
		else
		{
			this.button1.Visible = false;
			this.vuMeterLed1.Visible = false;
			this.vuMeterLed2.Visible = false;
			this.label2.Visible = false;
			this.label4.Visible = false;
			this.txtFilename.Visible = true;
			this.btnCue.Visible = true;
			this.btnStart.Visible = true;
			this.btnStop.Visible = true;
			this.label1.Visible = true;
		}
	}
}

private bool audioSampling = false;
private bool AudioSampling
{
	get { return this.audioSampling; }
	set
	{
		this.audioSampling = value;
		if(value)
		{
			this.timer1.Start();
			this.timer1running = true;
			this.capture.EnableEvent(audioHandler);
			this.button1.Text = "Stop VuMeter";
		}
		else
		{
			this.capture.DisableEvent(audioHandler);
			this.timer1running = false;
			this.timer1.Stop();
			this.button1.Text = "Start VuMeter";
		}
	}
}

private void button1_Click(object sender, System.EventArgs e)
{
	if((this.capture != null)&&(this.capture.AllowSampleGrabber))
	{
		this.AudioSampling = !this.AudioSampling;
	}
	else
	{
		this.AudioSampling = false;
		this.SampleGrabber = false;
	}
}

private void menuSampleGrabber1_Click(object sender, System.EventArgs e)
{
	if(this.SampleGrabber)
	{
		this.SampleGrabber = false;
	}
	else
	{
		this.SampleGrabber = true;
	}
}

private void menuAllowSampleGrabber1_Click(object sender, System.EventArgs e)
{
	// Set flag, if set, then after reselection of audio or video device,
	// the SampleGrabber shows up in or disappears from the menu.
	this.menuAllowSampleGrabber1.Checked = !this.menuAllowSampleGrabber1.Checked;
}

The grabbed audio is passed via an event to the main program. The program analyses the audio, which is assumed to be 16 bit stereo audio. To save CPU time, a part of the total audio data will be analyzed.
The VU meter will show the average audio level and the peak level for the left and right audio channel. A timer is used to show the VU meter levels to make the audio levels less dependent from grabbing. It is possible to use an event for showing the VU meter levels, however, drawing the VU meter levels after analysing the grabbed audio might consume too much CPU time. The audio levels of 16 bit audio can vary between 0 and 32767, so there is a dynamic range of 90 dB: 20 x 10log(32767) = 90,31 dB. This VU meter can show peak and audio levels in fifteen steps. Normally audio is displayed with a VU meter in steps of 3 dB, as a result a part of the dynamic range can be shown. Because the lower levels are not that interesting, I chose to show the higher levels only. Still the dynamic range of the VU meter is about 40 dB and that is quite a lot compared with the analog VU meter. So the actual formula to show the audio levels could be: ((20 x 10log(audio level))- 40)/ 3 to get 15 positive values to show. I simplified this formula to (6 x 10log(audio level))- 12.

private DirectX.Capture.Capture.AudioFrame audioHandler = 
		new DirectX.Capture.Capture.AudioFrame(AudioCapture);

const int MAXSAMPLES = 250; // Number of samples to be checked
static int volumePeakR = 0;
static int volumePeakL = 0;
static int volumeAvgR = 0;
static int volumeAvgL = 0;
static bool volumeInfoBusy = false;

/// <summary>
/// Analyse audio information in the buffer.
/// The code works for 16 bit, stereo audio.
/// </summary>
/// <param name="e"></param>
/// <param name="BufferLen"></param>
private static void AudioCapture(short[] e, int BufferLen)
{
	if((BufferLen <= 0)||(volumeInfoBusy))
	{
		return;
	}
	volumeInfoBusy = true;

	int leftS = 0;
	int rightS = 0;
	int avgR = 0;
	int avgL = 0;
	int peakR = 0;
	int peakL = 0;
	int size = e.GetLength(0)/ 2; // Assume this is 2 channel audio
	if(size >(BufferLen / 2))
	{
		size = BufferLen / 2;
	}

	if(size > MAXSAMPLES)
	{
		size = MAXSAMPLES;
	}

	if(size < 2)
	{
		volumeInfoBusy = false;
		return;
	}

	// Check array contents
	for(int i = 0; i < size; i += 2)
	{
		leftS = Math.Abs(e[i]);
		avgL += leftS;
		if(leftS > peakL)
		{
			peakL = leftS;
		}
		rightS = Math.Abs(e[i + 1]);
		avgR += rightS;
		if(rightS > peakR)
		{
			peakR = rightS;
		}
	} // for

	volumeAvgR = avgR / size;
	volumeAvgL = avgL / size;
	volumePeakR = peakR;
	volumePeakL = peakL;
}

bool timer1running = false;

const int DbMultiplier = 6;
const int DbOffset = 12;

private void timer1_Tick(object sender, System.EventArgs e)
{
	if(this.timer1running)
	{
		if(this.capture == null)
		{
			return;
		}

		int avgR, avgL, peakR, peakL;
		if(volumeInfoBusy)
		{
			avgR = (int)((Math.Log10(volumeAvgR)* 
						DbMultiplier)- DbOffset);
			avgL = (int)((Math.Log10(volumeAvgL)* 
						DbMultiplier)- DbOffset);
			peakR = (int)((Math.Log10(volumePeakR)* 
						DbMultiplier)- DbOffset);
			peakL = (int)((Math.Log10(volumePeakL)* 
						DbMultiplier)- DbOffset);
#if DEBUG
			Debug.WriteLine("L="+ avgL.ToString() + " "+ 
				peakL.ToString() + " R=" + avgR.ToString() + 
				" " + peakR.ToString());
#endif
			this.vuMeterLed1.Peak = peakL;
			this.vuMeterLed1.Volume = avgL;
			this.vuMeterLed2.Peak = peakR;
			this.vuMeterLed2.Volume = avgR;
			volumeInfoBusy = false;
		}
	}
}

Points of Interest

Compared with the previous articles, Audio File Saving for the DirectX.Capture Class, Video File Saving in Windows Media Video Format for the DirectX.Capture Class Library and DirectShow: TV Fine-tuning using IKsPropertySet in C#, most of the features are kept in and the SampleGrabber functionality has been added.

The code example has been tested with Visual Studio 2003 as well as Visual Studio 2005. Conflicts between these two compiler versions might occur. I added the conditional VS2003 to show the code differences. A difference was that in Visual Studio 2003, a signal was named Closed, while in Visual Studio 2005, this signal had the name FormClosed. This code example has two versions: the Visual Studio 2003 version using DShowNET as DirectShow interface library, and the Visual Studio 2005 version using DirectShowLib-2005 as DirectShow interface library. It should still be possible to use DShowNET with Visual Studio 2005, but I did not test that. If both Visual Studio versions are needed, then use different directories for this code example to prevent build problems. It is not my intention to solve coding conflicts and build problems that might occur between the several Visual Studio versions.

The DirectX.Capture examples that go with this article contain new solutions to increase stability. Still, exceptions may occur, but most of them can be solved by either redesigning this code example or by catching and handling the exceptions in a more appropriate way. Keep in mind that this code example is for learning purposes only. It teaches you how to use DirectShow in C# and teaches you to use GUI. Exceptions that occur should not be seen as a problem, but as a challenge! The major advantage of an exception is that it tells you when something goes wrong. As a side effect, the program fails and, by debugging, the cause of the problem can be found much easier because you know where to start.

Feedback and Improvements

I hope this code helps you in understanding the structure of the DirectX.Capture class. I also hope I provided you with an enhancement that might be useful to you. Feel free to post any comments and questions.

History

  • August 10, 2007: Initially written the SampleGrabber code example for grabbing a frame
  • April 1, 2009: Created SampleGrabber examples for grabbing a frame in a new article
  • May 8, 2009: Updated - added code example showing audio levels via VU meter

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

almere109
Web Developer
Netherlands Netherlands
No Biography provided

Comments and Discussions

 
AnswerRe: How to capture and send(over network) Audio data at realtime ? Pinmemberalmere10911-Sep-09 6:21 
GeneralExcellent thanks!!! very functional PinmemberManuel Enrique1-Aug-09 17:48 
Questionunable to use De interlaced filter ! Pinmemberpasupuletiramesh28-Jul-09 7:04 
AnswerRe: unable to use De interlaced filter ! Pinmemberalmere10928-Jul-09 9:25 
QuestionSaving User Settings! Pinmemberpasupuletiramesh31-May-09 4:09 
AnswerRe: Saving User Settings! Pinmemberalmere10931-May-09 4:49 
GeneralProblem With Live Video when viewing Fast moving object !!! Pinmemberpasupuletiramesh9-May-09 19:45 
GeneralRe: Problem With Live Video when viewing Fast moving object !!! Pinmemberalmere1099-May-09 22:42 
GeneralSaving Preview Pin And Capture Pin Settings! Pinmemberpasupuletiramesh9-May-09 6:05 
GeneralRe: Saving Preview Pin And Capture Pin Settings! Pinmemberalmere1099-May-09 11:39 
AnswerRe: Saving Preview Pin And Capture Pin Settings! [modified] Pinmemberpasupuletiramesh9-May-09 15:50 
GeneralExcellent. Pinmemberxliqz9-May-09 0:34 
GeneralRe: Excellent. Pinmemberalmere1099-May-09 0:45 
GeneralDMO filter Pinmembergino de cock26-Apr-09 11:07 
GeneralRe: DMO filter Pinmemberalmere10926-Apr-09 21:42 
GeneralProblem with Resolution!!! Pinmemberpasupuletiramesh22-Apr-09 1:43 
GeneralRe: Problem with Resolution!!! Pinmemberalmere10922-Apr-09 8:58 
GeneralRe: Problem with Resolution!!! PinmemberSunMee29-Apr-09 5:29 
GeneralRe: Problem with Resolution!!! Pinmemberalmere10929-Apr-09 22:16 
AnswerRe: Problem with Resolution!!! Pinmemberpasupuletiramesh9-May-09 19:54 
GeneralRe: Problem with Resolution!!! Pinmembermissautumn23-May-09 0:24 
GeneralRe: Problem with Resolution!!! Pinmemberalmere10923-May-09 1:59 
GeneralDeInterlace Filter support needed Pinmembersundaram0611-Apr-09 6:04 
GeneralRe: DeInterlace Filter support needed Pinmemberalmere10911-Apr-09 8:00 
GeneralRe: DeInterlace Filter support needed Pinmembersundaram0612-Apr-09 0:34 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web03 | 2.8.141223.1 | Last Updated 8 May 2009
Article Copyright 2009 by almere109
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid