Email Password Remember me?  Lost your password?

• Download demo project - 85.58 KB
• Download source - 132.18 KB

Introduction

This project demonstrates an implementation of the waterfall spectrogram and use of statistical data to trigger events in near real-time. This code is an elaboration of my previous submission (SoundViewer). This demonstration utilizes the Wave classes developed by Ianier Munoz.

Using the Code

Audio is supplied by the default input device which is typically the microphone. Events are triggered when audio amplitude exceeds the desired threshold value, which can be set under Options on the menu bar. To make this more useful, I've added functionality to save the stream to disk which results in a nice sound activated recorder.

Points of Interest

In order to draw the spectrogram fast enough to allow for near real-time operation, I needed to write directly to memory using unsafe code.

// lock image
PixelFormat format = canvas.PixelFormat;
BitmapData data = 
    canvas.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadOnly, format);
int stride = data.Stride;
int offset = stride - width * 4;

// draw image
try
{
  unsafe
  {
    byte* pixel = (byte*)data.Scan0.ToPointer();
    // for each column
    for (int y = 0; y <= height; y++)
    {
      if (y < _fftLeftSpect.Count)
      {
        // for each row
        for (int x = 0; x < width; x++, pixel += 4)
        {
          double amplitude = ((double[])_fftLeftSpect[_fftLeftSpect.Count - y - 1])
                [(int)(((double)(_fftLeft.Length) / (double)(width)) * x)];
          double color = GetColor(min, max, range, amplitude);
          pixel[0] = (byte)0;
          pixel[1] = (byte)color;
          pixel[2] = (byte)0;
          pixel[3] = (byte)255;
        }
        pixel += offset;
      }
    }
  }
}
catch (Exception ex)
{
  Console.WriteLine(ex.ToString());
}

// unlock image
canvas.UnlockBits(data);

I noticed that the results vary wildly depending on the hardware and associated drivers being used.

Some things I'd like to experiment with further when I get the time:

1. Use of frequency domain to produce "motion" detector equivalent
2. Use of spectrogram in sound identification
3. Improving performance/robustness

History

• 01/16/2008: Created

You must Sign In to use this message board.

Per page 102550

FirstPrevNext

How to make the IFFT? afmf 6:33 3 Nov '09   I made an high pass filter after the FFT and I wanted to record the result. For that, I need to do a IFFT. Can you tell me how do I do it with your code? tks aF -.- Sign In·View Thread·PermaLink How to reduce the noise? GeoNav 2:55 31 Oct '09   I am able to get the audio from wavOut class and process them to get the various displays. But there is a lot of noise. How do I reduce it? Also, the Time Domain Display does not show any significant variation of amplitude. Any way out? Sign In·View Thread·PermaLink lineout audio GeoNav 0:10 29 Oct '09   How can I change the source to line-out instead of line-in and display the results in the same way? Sign In·View Thread·PermaLink Change Source to wav file GeoNav 22:36 21 Oct '09   Is it possible to change the source to an existing wav file? or alternately, capture the sound as it is being played by the system? Sign In·View Thread·PermaLink Re: Change Source to wav file Jeff Morton 12:23 22 Oct '09   I'm sure that it is. Sign In·View Thread·PermaLink Re: Change Source to wav file GeoNav 19:35 26 Oct '09   Could you give me some directions to input the wav file data to the program instead of the inline mic? I can get the format of the file. But thereafter, how do I get the audio frames of the file input for FFT? Also, is it possible to do band pass filtering in this code? Sign In·View Thread·PermaLink 2.00/5 Re: Change Source to wav file afmf 6:38 3 Nov '09   To make a band pass the filter, simply put the FFT with the value zero on the frequencies that you don't wish. For example, if you want the human voice (it's more and less 130-1100Hz). You can convert to Hz using the scaleHz on the values of the FFT array. To get the audio frames you will have to do the inverse of the FFT calculated (IFFT). I want this as well :P If he answers me or you find out, please post here. aF was here -.- Sign In·View Thread·PermaLink Error in Program Piligrim2007 12:04 20 Oct '09   Hi! There is a huge mistake in the source! Exactly we have bug in Form1.cs file in the function "private void Start()": -String, that was written by author:_waveFormat = new WaveFormat(_audioFrameSize, _audioBitsPerSample, _audioChannels); Generally, we have "bad" sampling rate exactly due to error in WaveFormat - call, such that the first argument should be sampling rate (public WaveFormat(int rate, int bits, int channels)) !!!We should use string like: -String, that was written by author:_waveFormat = new WaveFormat(_SamplingRate, _audioBitsPerSample, _audioChannels); So, guys... all the problems about "Error in Frequency domain" are being solved (I think so). Sign In·View Thread·PermaLink 1.00/5 Re: Error in Program Jeff Morton 12:23 22 Oct '09   Glad to hear you are doing well. Did this solve your issue? Sign In·View Thread·PermaLink 2.00/5 Re: Error in Program Piligrim2007 1:52 24 Oct '09   Thank You (and for a good source too)! Sure I am completely solve my issue) Sign In·View Thread·PermaLink Can't run the program in Vista 64bit Yasir Salih Osman 23:36 7 Aug '09   Hi Jeff I'm using vista 64bit eddition and I can't get the program working it always says device stopped polling seems its unable to start it in vista do u hav any idea what is the cause of this problem?? Sign In·View Thread·PermaLink Re: Can't run the program in Vista 64bit Fullmetal99012 8:20 11 Sep '09   I got it to work by changing the debug platform from to x86. Sign In·View Thread·PermaLink how can i do it reverse? [modified] mojimoj 22:05 2 Aug '09   I have spectrum signal and i want to play it's sound, how can i do it???? modified on Monday, August 3, 2009 3:33 AM Sign In·View Thread·PermaLink Excellent piece of work Jeff maxima120 13:17 4 Jul '09   This is really greate code you did. It seems however the program cannot find playback devices. Is it on purpose or is it my hardware/drivers? I need to detect certain characteristics of sound made by another program. It would be great if I could capture straight from playback. Thank you Sign In·View Thread·PermaLink Re: Excellent piece of work Jeff Jeff Morton 6:46 5 Jul '09   I never added this functionality since I was primarily interested in microphone input. Adding this to the existing code should be straightforward. Sign In·View Thread·PermaLink Throws exception when using Freq < 11025 Hz k^s 8:07 10 Jun '09   Hi, i've been testing your project and i've to say that is very nice, i've been looking for something like this sometimes. I've been testing different "SamplesPerSecond" values, but when using values under 11025 ( minimum is 8000) it throws an exception. The first thing i've seen is that the buffer size for AudioFrame.Process() methodmust be dividable by 4 (len % 4 = 0). The second is that the FourierTransform.FFT() method throws an "Index out of range exception". I've been debubbing it and i found that in line 69 tr = xre[k + n2] * c + xim[k + n2] * s; "k + n2" is absolutely out of range, about 100 positions over the limit. Is there any bug or is not posible to plot frequencies under 11025Hz? Note: i am plotting using: public void init() { oAudioFrame = new AudioFrame(false); oAudioFrame.IsDetectingEvents = true; oAudioFrame.AmplitudeThreshold = 16384; } public void Plot() { oAudioFrame.Process(ref sound_data); oAudioFrame.RenderTimeDomainRight(ref pb1); } Sign In·View Thread·PermaLink How do I get it to work? Maiden Computer Company 11:48 22 Apr '09   I receive this in the status window when running the source code in Vista 64-bit: 4/22/2009 4:44:10 PM : Audio input device detected 4/22/2009 4:44:10 PM : Audio input device polling stopped When I run the executable, I get an unhandled exception and it does the same as the source if I continue: ************** Exception Text ************** System.NullReferenceException: Object reference not set to an instance of an object. at SoundCatcher.FormMain.Start() at SoundCatcher.FormMain.FormMain_Load(Object sender, EventArgs e) at System.Windows.Forms.Form.OnLoad(EventArgs e) at System.Windows.Forms.Control.CreateControl(Boolean fIgnoreVisible) at System.Windows.Forms.Control.CreateControl() at System.Windows.Forms.Control.WmShowWindow(Message& m) at System.Windows.Forms.Control.WndProc(Message& m) at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m) at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam) ************** Loaded Assemblies ************** mscorlib Assembly Version: 2.0.0.0 Win32 Version: 2.0.50727.3074 (QFE.050727-3000) CodeBase: file:///C:/Windows/Microsoft.NET/Framework64/v2.0.50727/mscorlib.dll ---------------------------------------- SoundCatcher Assembly Version: 1.0.0.0 Win32 Version: 1.0.0.0 CodeBase: file:///C:/Users/David/Documents/SoundCatcher/SoundCatcher.exe ---------------------------------------- System.Windows.Forms Assembly Version: 2.0.0.0 Win32 Version: 2.0.50727.3053 (netfxsp.050727-3000) CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll ---------------------------------------- System Assembly Version: 2.0.0.0 Win32 Version: 2.0.50727.3053 (netfxsp.050727-3000) CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll ---------------------------------------- System.Drawing Assembly Version: 2.0.0.0 Win32 Version: 2.0.50727.3053 (netfxsp.050727-3000) CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll ---------------------------------------- System.Configuration Assembly Version: 2.0.0.0 Win32 Version: 2.0.50727.3053 (netfxsp.050727-3000) CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Configuration/2.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll ---------------------------------------- System.Xml Assembly Version: 2.0.0.0 Win32 Version: 2.0.50727.3074 (QFE.050727-3000) CodeBase: file:///C:/Windows/assembly/GAC_MSIL/System.Xml/2.0.0.0__b77a5c561934e089/System.Xml.dll ---------------------------------------- Any ideas? Sign In·View Thread·PermaLink Re: How do I get it to work? Jeff Morton 11:51 22 Apr '09   This is a known issue with the wave library. Compile it using the x86 instruction set and it should work fine. Sign In·View Thread·PermaLink 1.00/5 Error in Frequency domain? Elysium 12:17 8 Mar '09   I'm not sure about this but I suspect there's a problem with your frequency spectrum. I've set your 'test' flag to true so that it works on a simple sine wave and the spectrum shows a dominant frequency with side content indicating the presence of other frequencies which simply aren't there. You should see a _single_ spike in the spectrum corresponding to the frequency of the sine wave being transformed. Otherwise, great job Sign In·View Thread·PermaLink Re: Error in Frequency domain? Jeff Morton 12:41 8 Mar '09   What frequency? I think I may have corrected this in a later revision that I didn't post. It was a rounding error in drawing the wave if I remember correctly. I'll try to find the correction and post it here. Thanks for the constructive critcism. Sign In·View Thread·PermaLink 2.00/5 Re: Error in Frequency domain? Elysium 23:02 8 Mar '09   I did have another thought I'll need to check...to get the single spike in the freq domain you'll need an integral number of sine wave cycles in your sample window otherwise you will see some other spurious frequency content (not sure if that is what we are seeing). Sign In·View Thread·PermaLink Question on how to get the frequencies kyanmark_johnwill 14:52 2 Mar '09   Hi, Mr. Morton! My groupmates and I have a project in which we aim to get the the person's voice frequency so that we could tell his/her vocal range (i.e. Soprano, Alto, Bass, Tenor). We are very interested in your project here, and we would like to utilize some of your techniques. We were just wondering how we could get the frequencies. Instead of displaying it on the screen, how could we get the frequencies of the voice each second and could we possibly put the values in an array or arraylist? If so, could you help us locate the specific part of your code that could help us accomplish this? Thank you for your time, and I hope for your reply :D Sign In·View Thread·PermaLink Re: Question on how to get the frequencies hazeljane 4:49 13 Mar '09   hi, We have the same project, getting the persons voice frequency and analyzing their vocal range. I would like to ask on what algorithm you are using for the project?. How would you get the voice ranges? Thanks. Hazel Sign In·View Thread·PermaLink Re: Question on how to get the frequencies Jeff Morton 11:34 13 Mar '09   Sounds great. I was considering messing around with voice detection as well, although there is no serious analysis occuring in this project since I'm just triggering an event based on an amplitude threshold and not actually detecting voice. All of the basics are covered in this project though. You would just need to elborate on what's here already. If you succeed please upload your project so others can learn from it. I know I would be interested in seeing the results myself. Sign In·View Thread·PermaLink 2.00/5 Re: Question on how to get the frequencies afmf 2:54 23 Oct '09   Jeff, can you help us! How can we get the frequency each time fft is calculated? aF was here -.- Sign In·View Thread·PermaLink

Last Visit: 10:07 24 Nov '09     Last Update: 10:07 24 Nov '09 123 Next »