Click here to Skip to main content
Email Password   helpLost your password?
SoundCatcher

Introduction

This project demonstrates an implementation of the waterfall spectrogram and use of statistical data to trigger events in near real-time. This code is an elaboration of my previous submission (SoundViewer). This demonstration utilizes the Wave classes developed by Ianier Munoz.

Using the Code

Audio is supplied by the default input device which is typically the microphone. Events are triggered when audio amplitude exceeds the desired threshold value, which can be set under Options on the menu bar. To make this more useful, I've added functionality to save the stream to disk which results in a nice sound activated recorder.

Points of Interest

In order to draw the spectrogram fast enough to allow for near real-time operation, I needed to write directly to memory using unsafe code.

// lock image
PixelFormat format = canvas.PixelFormat;
BitmapData data = 
    canvas.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadOnly, format);
int stride = data.Stride;
int offset = stride - width * 4;

// draw image
try
{
  unsafe
  {
    byte* pixel = (byte*)data.Scan0.ToPointer();
    // for each column
    for (int y = 0; y <= height; y++)
    {
      if (y < _fftLeftSpect.Count)
      {
        // for each row
        for (int x = 0; x < width; x++, pixel += 4)
        {
          double amplitude = ((double[])_fftLeftSpect[_fftLeftSpect.Count - y - 1])
                [(int)(((double)(_fftLeft.Length) / (double)(width)) * x)];
          double color = GetColor(min, max, range, amplitude);
          pixel[0] = (byte)0;
          pixel[1] = (byte)color;
          pixel[2] = (byte)0;
          pixel[3] = (byte)255;
        }
        pixel += offset;
      }
    }
  }
}
catch (Exception ex)
{
  Console.WriteLine(ex.ToString());
}

// unlock image
canvas.UnlockBits(data);

I noticed that the results vary wildly depending on the hardware and associated drivers being used.

Some things I'd like to experiment with further when I get the time:

  1. Use of frequency domain to produce "motion" detector equivalent
  2. Use of spectrogram in sound identification
  3. Improving performance/robustness

History

You must Sign In to use this message board.
 
 
Per page   
 FirstPrevNext
GeneralHow to generate wave-form if we have the byte-array with us.
Mark02
0:06 12 Mar '10  
Hi Jeff,


I found that the method "Process(ref byte[] wave)" plays the main role here.

This method gets the appropriate byte array as the input and going further,
the methods-
"public void RenderTimeDomainRight(ref PictureBox pictureBox)"
and
"public void RenderTimeDomainLeft(ref PictureBox pictureBox)"

are displaying the waveforms.


What if I, already have with me, this "byte[] wave" byte-array for which the wave-form has to be displayed?
An audio-stream gives me this byte-array at regular intervals of time one-after-the-other.

I am currently giving this "wave" directly as input to the method i.e. "Process(ref byte[] wave)"
instead of going through all the "DataArrived" stuff.


Or is there some way I could use "ISpeechBaseStream" object to give audio stream as input ?

Please can you help.



Thanks in advance
Mark
GeneralIntegration Time
GeoNav
5:17 17 Feb '10  
Hi,
Is it possible to average the audio over a period, say a few seconds? I guess that will reduce a lot of noise. Any ideas in this direction?
GeneralSound Equalizer
BazKhan
20:34 25 Jan '10  
Dear Jeff Morton,

Let us know whether or not it is possible to create a sound equalizer using this code. Would you like to guide us?

Don't believe in love...

GeneralAmplitude -sound loudness
ginwithlime
5:10 9 Jan '10  
Your project is great. I want to use it on my project too. The project aims to analyze sounds coming from microphone in real-time by their sound loudness or amplitude. Is this application perfect for our project?
I wanna know if it can measure the loudness of the sound. Coz I've seen the app and it displays the numeric figures in dB. I hope this is what I'm really looking for.

Thanks! Great Job!
GeneralRe: Amplitude -sound loudness
Jeff Morton
6:36 9 Jan '10  
Yes, that's essentially what it does.
QuestionHow to stop the thread if a invoke methode gets called in the DataArrived Event
Michael Muehllehner
4:57 7 Dec '09  
Hello,
i want to use this great application to show me the loudest frequency in the fft. It works very well, the RenderFrequencyDomain allready calculates the loudest frequency.
I have to use a invoke to display this number on a textbox. The application works, i got the frequency displayed on a textbox, but i can not stop the thread.

Code in the DataArrived Methode: (i returned the maxHz in the RenderFrequencyDomainLeft)

double leftHz = _audioFrame.RenderFrequencyDomainLeft(...)
Invoke(new MethodInvoker(delegate{
textBox1.Text = leftHz.ToString() + " Hz";
}));

When i want to close the application the call stack hangs in the following step:
- FormOnClosing
-> Stop();
-> _recorder.Dispose();
-> WaitForAllBuffers();
-> Buf.WaitFor();
-> m_Recording = m_RecordEvent.WaitOne(); // The WaitOne Methode hangs

If i give a timeout to the WaitOne Methode, the closing call stack hangs in the following step:
- FormOnClosing
-> Stop();
-> _recorder.Dispose();
-> m_Thread.Join(); // The Join Methode hangs

If i give a timeout to the Join Methode, i got a objectdisposedexeption in the Invoke Methode i wrote for updating the textbox.

I want to know if there is any good solution to access user controls with an invoke in the dataarrived event. And with that i can stop the threads without timeout programming.

thx a lot
Mike
GeneralHow to make the IFFT?
afmf
6:33 3 Nov '09  
I made an high pass filter after the FFT and I wanted to record the result.
For that, I need to do a IFFT.
Can you tell me how do I do it with your code? tks

aF -.-

GeneralHow to reduce the noise?
GeoNav
2:55 31 Oct '09  
I am able to get the audio from wavOut class and process them to get the various displays. But there is a lot of noise. How do I reduce it? Also, the Time Domain Display does not show any significant variation of amplitude. Any way out?
Generallineout audio
GeoNav
0:10 29 Oct '09  
How can I change the source to line-out instead of line-in and display the results in the same way?
QuestionChange Source to wav file
GeoNav
22:36 21 Oct '09  
Is it possible to change the source to an existing wav file? or alternately, capture the sound as it is being played by the system?
AnswerRe: Change Source to wav file
Jeff Morton
12:23 22 Oct '09  
I'm sure that it is.
GeneralRe: Change Source to wav file
GeoNav
19:35 26 Oct '09  
Could you give me some directions to input the wav file data to the program instead of the inline mic? I can get the format of the file. But thereafter, how do I get the audio frames of the file input for FFT?
Also, is it possible to do band pass filtering in this code?
GeneralRe: Change Source to wav file
afmf
6:38 3 Nov '09  
To make a band pass the filter, simply put the FFT with the value zero on the frequencies that you don't wish. For example, if you want the human voice (it's more and less 130-1100Hz). You can convert to Hz using the scaleHz on the values of the FFT array.

To get the audio frames you will have to do the inverse of the FFT calculated (IFFT).
I want this as well :P If he answers me or you find out, please post here.

aF was here -.-

GeneralError in Program
Piligrim2007
12:04 20 Oct '09  
Hi!
There is a huge Laugh mistake in the source!
Exactly we have bug in Form1.cs file in the function "private void Start()":

-String, that was written by author:_waveFormat = new WaveFormat(_audioFrameSize, _audioBitsPerSample, _audioChannels);

Generally, we have "bad" sampling rate exactly due to error in WaveFormat - call, such that the first argument should be sampling rateSmile (public WaveFormat(int rate, int bits, int channels))

!!!We should use string like: -String, that was written by author:_waveFormat = new WaveFormat(_SamplingRate, _audioBitsPerSample, _audioChannels);

So, guys... all the problems about "Error in Frequency domain" are being solved (I think so).
GeneralRe: Error in Program
Jeff Morton
12:23 22 Oct '09  
Glad to hear you are doing well.

Did this solve your issue?
GeneralRe: Error in Program
Piligrim2007
1:52 24 Oct '09  
Thank You (and for a good source too)!

SureSmile
I am completely solve my issue)
QuestionCan't run the program in Vista 64bit
Yasir Salih Osman
23:36 7 Aug '09  
Hi Jeff

I'm using vista 64bit eddition and I can't get the program working
it always says device stopped polling seems its unable to start it in vista
do u hav any idea what is the cause of this problem??
AnswerRe: Can't run the program in Vista 64bit
Fullmetal99012
8:20 11 Sep '09  
I got it to work by changing the debug platform from to x86.
Generalhow can i do it reverse? [modified]
mojimoj
22:05 2 Aug '09  
I have spectrum signal and i want to play it's sound,
how can i do it????

modified on Monday, August 3, 2009 3:33 AM

GeneralExcellent piece of work Jeff
maxima120
13:17 4 Jul '09  
This is really greate code you did.

It seems however the program cannot find playback devices. Is it on purpose or is it my hardware/drivers?

I need to detect certain characteristics of sound made by another program. It would be great if I could capture straight from playback.

Thank you
GeneralRe: Excellent piece of work Jeff
Jeff Morton
6:46 5 Jul '09  
I never added this functionality since I was primarily interested in microphone input.

Adding this to the existing code should be straightforward.
GeneralThrows exception when using Freq < 11025 Hz
k^s
8:07 10 Jun '09  
Hi, i've been testing your project and i've to say that is very nice, i've been looking for something like this sometimes.

I've been testing different "SamplesPerSecond" values, but when using values under 11025 ( minimum is 8000) it throws an exception.

The first thing i've seen is that the buffer size for AudioFrame.Process() methodmust be dividable by 4 (len % 4 = 0).

The second is that the FourierTransform.FFT() method throws an "Index out of range exception". I've been debubbing it and i found that in line 69
tr = xre[k + n2] * c + xim[k + n2] * s;
"k + n2" is absolutely out of range, about 100 positions over the limit.

Is there any bug or is not posible to plot frequencies under 11025Hz?

Note: i am plotting using:
public void init()
{
oAudioFrame = new AudioFrame(false);
oAudioFrame.IsDetectingEvents = true;
oAudioFrame.AmplitudeThreshold = 16384;
}

public void Plot()
{
oAudioFrame.Process(ref sound_data);
oAudioFrame.RenderTimeDomainRight(ref pb1);
}

QuestionHow do I get it to work?
Maiden Computer Company
11:48 22 Apr '09  
I receive this in the status window when running the source code in Vista 64-bit:

4/22/2009 4:44:10 PM : Audio input device detected
4/22/2009 4:44:10 PM : Audio input device polling stopped

When I run the executable, I get an unhandled exception and it does the same as the source if I continue:

************** Exception Text **************
System.NullReferenceException: Object reference not set to an instance of an object.
at SoundCatcher.FormMain.Start()
at SoundCatcher.FormMain.FormMain_Load(Object sender, EventArgs e)
at System.Windows.Forms.Form.OnLoad(EventArgs e)
at System.Windows.Forms.Control.CreateControl(Boolean fIgnoreVisible)
at System.Windows.Forms.Control.CreateControl()
at System.Windows.Forms.Control.WmShowWindow(Message& m)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.3074 (QFE.050727-3000)
CodeBase: file:///C:/Windows/Microsoft.NET/Framework64/v2.0.50727/mscorlib.dll
----------------------------------------
SoundCatcher
Assembly Version: 1.0.0.0
Win32 Version: 1.0.0.0
CodeBase: file:///C:/Users/David/Documents/SoundCatcher/SoundCatcher.exe
----------------------------------------
System.Windows.Forms
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.3053 (netfxsp.050727-3000)
CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.3053 (netfxsp.050727-3000)
CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.3053 (netfxsp.050727-3000)
CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System.Configuration
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.3053 (netfxsp.050727-3000)
CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Configuration/2.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Xml
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.3074 (QFE.050727-3000)
CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Xml/2.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------

Any ideas?
AnswerRe: How do I get it to work?
Jeff Morton
11:51 22 Apr '09  
This is a known issue with the wave library. Compile it using the x86 instruction set and it should work fine.
GeneralError in Frequency domain?
Elysium
12:17 8 Mar '09  
I'm not sure about this but I suspect there's a problem with your frequency spectrum. I've set your 'test' flag to true so that it works on a simple sine wave and the spectrum shows a dominant frequency with side content indicating the presence of other frequencies which simply aren't there. You should see a _single_ spike in the spectrum corresponding to the frequency of the sine wave being transformed. Otherwise, great job Smile


Last Updated 28 Jan 2008 | Advertise | Privacy | Terms of Use | Copyright © CodeProject, 1999-2010