Click here to Skip to main content
15,886,864 members
Articles / Multimedia / GDI
Article

Sound Activated Recorder with Spectrogram in C#

Rate me:
Please Sign up or sign in to vote.
4.92/5 (54 votes)
27 Jan 2008GPL3 402.8K   32.1K   207   105
Audio event processing with visual display
SoundCatcher

Introduction

This project demonstrates an implementation of the waterfall spectrogram and use of statistical data to trigger events in near real-time. This code is an elaboration of my previous submission (SoundViewer). This demonstration utilizes the Wave classes developed by Ianier Munoz.

Using the Code

Audio is supplied by the default input device which is typically the microphone. Events are triggered when audio amplitude exceeds the desired threshold value, which can be set under Options on the menu bar. To make this more useful, I've added functionality to save the stream to disk which results in a nice sound activated recorder.

Points of Interest

In order to draw the spectrogram fast enough to allow for near real-time operation, I needed to write directly to memory using unsafe code.

C#
// lock image
PixelFormat format = canvas.PixelFormat;
BitmapData data = 
    canvas.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadOnly, format);
int stride = data.Stride;
int offset = stride - width * 4;

// draw image
try
{
  unsafe
  {
    byte* pixel = (byte*)data.Scan0.ToPointer();
    // for each column
    for (int y = 0; y <= height; y++)
    {
      if (y < _fftLeftSpect.Count)
      {
        // for each row
        for (int x = 0; x < width; x++, pixel += 4)
        {
          double amplitude = ((double[])_fftLeftSpect[_fftLeftSpect.Count - y - 1])
                [(int)(((double)(_fftLeft.Length) / (double)(width)) * x)];
          double color = GetColor(min, max, range, amplitude);
          pixel[0] = (byte)0;
          pixel[1] = (byte)color;
          pixel[2] = (byte)0;
          pixel[3] = (byte)255;
        }
        pixel += offset;
      }
    }
  }
}
catch (Exception ex)
{
  Console.WriteLine(ex.ToString());
}

// unlock image
canvas.UnlockBits(data);

I noticed that the results vary wildly depending on the hardware and associated drivers being used.

Some things I'd like to experiment with further when I get the time:

  1. Use of frequency domain to produce "motion" detector equivalent
  2. Use of spectrogram in sound identification
  3. Improving performance/robustness

History

  • 01/16/2008: Created

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)


Written By
Systems / Hardware Administrator
United States United States
This member has not yet provided a Biography. Assume it's interesting and varied, and probably something to do with programming.

Comments and Discussions

 
GeneralMy vote of 5 Pin
Nasenbaaer12-May-21 9:33
Nasenbaaer12-May-21 9:33 
QuestionSoundcatcher have an issue Pin
Member 109051932-Mar-17 18:36
Member 109051932-Mar-17 18:36 
QuestionSampling/buffering issue Pin
Member 1254606925-May-16 7:12
Member 1254606925-May-16 7:12 
Hello everybody,


Can anyone please help me with the following:

I used the open source code from mr. Morton to create a sound activated recorder.

The recorder works fine one some laptops/PC's, however, on other ones, the quality of the recorded file is very bad, as if the sound is 'chopped'.

I placed an example on https://www.youtube.com/watch?v=G4HqlUGcqMU&feature=youtu.be

Is this about some bad entered buffering/sampling parameter, or is it a bigger problem?

I would be relieved if someone can help me figure this out.


Jef

This is the code:


Quote:
private void RecordForm_Load(object sender, EventArgs e)
{
_isPlayer = true; // audio output for testing
_isTest = false; // signal generation for testing
_isSaving = false;
_isShown = true;
_recordings = new List<string>();
_current = 0;
exit = false;

gedaan = false;
replaying = false;


if (WaveNative.waveInGetNumDevs() != 0)
{
if (_isPlayer == true)
_streamOut = new FifoStream();
_audioFrame = new AudioFrame(_isTest);
_audioFrame.IsDetectingEvents = true;

_audioFrame.AmplitudeThreshold = Properties.Settings.Default.AmpTreshold;
numericUpDown1.Value = _audioFrame.AmplitudeThreshold / 1000;

_streamMemory = new MemoryStream();
Start();
buttonLive.BackColor = Color.Red;
}
else
{
noinput = true;
string message = "";
string title = "";
switch (taal)
{
case "NL":
message = "Geen audio-apparaat gedetecteerd ...";
title = "Fout";
break;
case "FR":
message = "Aucun appareil audio détecté ...";
title = "Erreur";
break;
case "EN":
message = "No audio device detected ...";
title = "Error";
break;
case "DE":
break;
}

MessageBox.Show(message, title, MessageBoxButtons.OK, MessageBoxIcon.Error);
}

}

private void Start()
{
Stop();
try
{
_waveFormat = new WaveFormat(44100, 16, 2);
_recorder = new WaveInRecorder(-1, _waveFormat, 8192 * 2, 3, new BufferDoneEventHandler(DataArrived));
if (_isPlayer == true)
_player = new WaveOutPlayer(-1, _waveFormat, 8192 * 2, 3, new BufferFillEventHandler(Filler));
}
catch (Exception ex)
{
//_recorder = null;
//_player = null;
//GC.Collect();
//Start();
}
}
private void Stop()
{

if (_recorder != null)
try
{
//_recorder = null;
_recorder.Dispose();
}
finally
{
_recorder = null;
}
if (_isPlayer == true)
{
if (_player != null)
try
{
_player.Dispose(true);
}
finally
{
_player = null;
}
_streamOut.Flush(); // clear all pending data
}
}

private void Filler(IntPtr data, int size)
{
if (_isPlayer == true)
{
if (_playerBuffer == null || _playerBuffer.Length < size)
_playerBuffer = new byte[size];
if (_streamOut.Length >= size)
_streamOut.Read(_playerBuffer, 0, size);
else
for (int i = 0; i < _playerBuffer.Length; i++)
_playerBuffer[i] = 0;
System.Runtime.InteropServices.Marshal.Copy(_playerBuffer, 0, data, size);
}
}

private void DataArrived(IntPtr data, int size)
{
if (!gedaan)
{
if (_isSaving == true)
{
byte[] recBuffer = new byte[size];
System.Runtime.InteropServices.Marshal.Copy(data, recBuffer, 0, size);
_streamMemory.Write(recBuffer, 0, recBuffer.Length);
}
if (_recorderBuffer == null || _recorderBuffer.Length != size)
_recorderBuffer = new byte[size];
if (_recorderBuffer != null)
{
System.Runtime.InteropServices.Marshal.Copy(data, _recorderBuffer, 0, size);
if (_isPlayer == true)
_streamOut.Write(_recorderBuffer, 0, _recorderBuffer.Length);
try
{
_audioFrame.Process(ref _recorderBuffer);
}
catch (Exception eks)
{

}

if (_audioFrame.IsEventActive == true)
{
//beeps afspringen
foreach (Form form in Application.OpenForms)
{
if (form.GetType() == typeof(MainForm))
{
//nog te testen
((MainForm)form).wordtOpgeroepen = true;
}
}
//player afspringen
try
{
controls.stop();
player.close();
_current = _recordings.Count();
replaying = false;
}
catch
{

}

buttonBack.BackColor = Control.DefaultBackColor;
buttonForward.BackColor = Control.DefaultBackColor;
buttonLive.BackColor = Color.Red;
if (_recorder != null)
{
//sh*t happens
try
{
label1.Invoke(new UpdateLabelCallback(this.UpdateLabel));
}
catch (Exception ez)
{

}

}

if (_isSaving == false)
{
_sampleFilename = DateTime.Now.ToString("yyyy-MM-dd (HHmmss)") + ".wav";
_timeLastDetection = DateTime.Now;
_isSaving = true;

}
else
{
_timeLastDetection = DateTime.Now;
foreach (Form form in Application.OpenForms)
{
if (form.GetType() == typeof(MainForm))
{
((MainForm)form).updateRecBool();
}
}
}
//Invoke(new MethodInvoker(AmplitudeEvent));
}
else
{
if (!replaying)
{
if (DateTime.Now.Subtract(_timeLastDetection).Seconds > 1)
{
foreach (Form form in Application.OpenForms)
{
if (form.GetType() == typeof(MainForm))
{
//nog te testen
((MainForm)form).wordtOpgeroepen = false;
}
}
}
}
//
}
//1 is seconds to save
if (_isSaving == true && DateTime.Now.Subtract(_timeLastDetection).Seconds > 1)
{
bool isExists = System.IO.Directory.Exists(Properties.Settings.Default.RecPath);

if (!isExists)
{
System.IO.Directory.CreateDirectory(Properties.Settings.Default.RecPath);
}

byte[] waveBuffer = new byte[16];
_streamWave = WaveStream.CreateStream(_streamMemory, _waveFormat);
waveBuffer = new byte[_streamWave.Length - _streamWave.Position];
_streamWave.Read(waveBuffer, 0, waveBuffer.Length);
//if (Properties.Settings.Default.SettingOutputPath != "")
_streamFile = new FileStream(Properties.Settings.Default.RecPath + "\\" + _sampleFilename, FileMode.Create);
/*else
_streamFile = new FileStream(_sampleFilename, FileMode.Create);*/
_streamFile.Write(waveBuffer, 0, waveBuffer.Length);
if (_streamWave != null) { _streamWave.Close(); }
if (_streamFile != null) { _streamFile.Close(); }
_streamMemory = new MemoryStream();
_isSaving = false;
_recordings.Add(Properties.Settings.Default.RecPath + "\\" + _sampleFilename);
_current = _recordings.Count();
try
{
label1.Invoke(new UpdateLabelCallback(this.UpdateLabel));
}
catch (Exception dddd)
{

}


if (_current == 999)
{
_current = 0;
_recordings.Clear();
}
//Invoke(new MethodInvoker(FileSavedEvent));
}
_audioFrame.RenderTimeDomainLeft(ref pictureBoxTimeDomainLeft);
}
}
else
{
try
{
foreach (Form form in Application.OpenForms)
{
if (form.GetType() == typeof(MainForm))
{
//nog te testen
((MainForm)form).wordtOpgeroepen = false;
}
}
}
catch (Exception ekzep)
{

}

}
}

QuestionNo input device detected Pin
Member 115043816-Nov-15 6:47
Member 115043816-Nov-15 6:47 
QuestionAbout can't exit the application, I had only disabled this row. Pin
neil23335-Oct-15 15:15
neil23335-Oct-15 15:15 
AnswerRe: About can't exit the application, I had only disabled this row. Pin
Member 120005677-Oct-15 23:20
Member 120005677-Oct-15 23:20 
QuestionFrequency-range Pin
Member 120005675-Oct-15 0:02
Member 120005675-Oct-15 0:02 
QuestionThanks Pin
se5a25-Jun-15 14:52
se5a25-Jun-15 14:52 
Questionfrequency Pin
Member 1067234319-Jun-15 10:51
Member 1067234319-Jun-15 10:51 
AnswerRe: frequency Pin
Member 120005675-Oct-15 0:03
Member 120005675-Oct-15 0:03 
QuestionShutdown and stereo/mono fixes Pin
chronogeo19-Oct-14 22:27
chronogeo19-Oct-14 22:27 
AnswerRe: Shutdown and stereo/mono fixes Pin
chronogeo19-Oct-14 22:28
chronogeo19-Oct-14 22:28 
GeneralRe: Shutdown and stereo/mono fixes Pin
chronogeo19-Oct-14 22:29
chronogeo19-Oct-14 22:29 
GeneralRe: Shutdown and stereo/mono fixes Pin
chronogeo19-Oct-14 22:31
chronogeo19-Oct-14 22:31 
QuestionUI Thread locks up on close Pin
Member 1107721311-Sep-14 22:07
Member 1107721311-Sep-14 22:07 
QuestionHelp please: Nasalance ratio Pin
alijahan25-Jun-14 6:16
alijahan25-Jun-14 6:16 
GeneralThanks a lot! Pin
lopopoo21-Nov-13 4:25
lopopoo21-Nov-13 4:25 
QuestionExcellent article; quick question: would it be possible to "listen" to speaker output? Pin
Member 1029319528-Sep-13 7:17
Member 1029319528-Sep-13 7:17 
QuestionDelay/Latency Pin
hieronymusde13-Aug-13 11:21
hieronymusde13-Aug-13 11:21 
GeneralMy vote of 5 Pin
hieronymusde13-Aug-13 10:04
hieronymusde13-Aug-13 10:04 
QuestionHelp for operation on a modulated signal Pin
galbandrea14-Apr-13 22:13
galbandrea14-Apr-13 22:13 
Bug"В экземпляре объекта не задана ссылка на объект." Pin
Member 99607234-Apr-13 1:54
Member 99607234-Apr-13 1:54 
Questionis there a way to select the iinput device (directx name ?) Pin
patrick zuili17-Mar-13 18:34
patrick zuili17-Mar-13 18:34 
Generalamazing one Pin
Member 810718412-Jan-13 5:07
Member 810718412-Jan-13 5:07 
GeneralMy vote of 5 Pin
javcastaalm23-Nov-12 9:08
javcastaalm23-Nov-12 9:08 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.