Click here to Skip to main content
Click here to Skip to main content

Circular Buffer

, 11 Sep 2002
Rate this:
Please Sign up or sign in to vote.
C# implementation of a Circular Buffer

Circular Buffers are use for data transfer between two processes. The Producer process places items into the Circular Buffer and the Consumer process removes them. The variable capacity of the Circular Buffer accommodates timing differences between the Producer and Consumer processes.

The Circular Buffer can execute faster than than other queues that hold a variable amount of data since a fixed size block of memory is allocated just once from memory management and then reused (the Circular Buffer can be visualized as such but is actually a linear buffer with indices that wrap, modulo the buffer size, when the end of the buffer is reached).

Often, the Circular Buffer is used to decouple two processes that operate at different speeds. For example, a faster Producer process can "burst" data into the buffer and continue with its processing. A slower Consumer of that data can then read it at its own rate without synchronizing and slowing the Producer. In this type of application, the average rate, over time, of both processes must be the same to avoid an over or under flow condition of the Circular Buffer (this is the "Synchronous Mode" of operation). Also, sequencing is critical. Unless one of the synchronizing methods described below is used, the Producer process must always execute first.

In other types of applications, overflow of the Circular Buffer and attendant data loss is acceptable. For example--Error Logging. Here process state data is continuously written into the Circular Buffer at a high rate but only the last few logged items are useful in troubleshooting a problem that might take hours or days to repeat. A buffer of size N is saturated with the last N-1 items written by the Producer and read when the error condition is detected. This is the "Asynchronous Mode" of the Circular Buffer.

In the Synchronous Mode, two methods are available to ensure that no data is lost. The Blocking method and the WaterMark method.

Calling Blocking methods will cause a calling thread to block until the Circular Buffer has the requested number of items. Note that blocking pairs must be used. That is, if the DequeueBlocking or CopyToBlocking methods are used, the EnqueueBlocking method must also be used. The WaterMark method will fire an event when the number of items in the Circular buffer reaches a preset level - eliminating wasted processing time due to process blocks. Example usage:

theCB.SetWaterMark(8); // notify everytime cb has 8 items 
theCB.WaterMarkNotify += new QueueCB.CBEventHandler(WaterMarkEvent);

The Winform application I provide isn't terribly useful for architecting a system using the Circular Buffer. I used it to test the class and then added the GUI for fun. But it might be a useful learning device if you are unfamiliar with the concepts. It simulates a N=36 size Circular Buffer with Producer and Consumer threads. It operates a little like a clock doesn't it.

Sample Image

The IEnumerator interface is supported and the items can be read from the Circular Buffer with foreach().

The DLL is signed for installation in the GAC.

Using the Circular Buffer in multi-threaded applications is very complex and subject to possible timing hazards. This is version 1.0.0.0 so beware!

Watch out for the blocking methods, at the last minute I decided to use an AutoResetEvent for synchronization because I wanted to learn how to use one. But to the degree tested, everything seems to work. Please report bugs (and fixes?) to me.

Thanks to the .NET team for giving us such amazing technology and for making this stuff so much fun.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

Robert Hinrichs

United States United States
No Biography provided

Comments and Discussions

 
GeneralMy vote of 5 PinmemberC_Johnson3-May-11 23:43 
QuestionBut the code? Pinmemberjanesconference21-Oct-09 3:53 
GeneralQuite Buggy Pinmemberirares16-Apr-09 7:15 
GeneralGUI PinmemberAnavrin26-Sep-06 4:37 
GeneralThread Doesnt Stop Pinsusskonnichiwa27-Sep-04 23:50 
GeneralFound bug - I guess Pinmemberorenderi30-Jan-03 6:08 
Hi,
I created a tester for your class:
I used sync mode and blocking methods.
The producers does not stop and wait when it full, the consumer too, does 2 rounds of consuming and then stop(it sould stop immediately).
I think this project is very useful and I want to help u improve it.
 
10x
 
Oren
 
my code - pls cut and run it:
 
using System;
using System.Collections;
using System.ComponentModel;
using System.Drawing;
using System.Threading;
 
using System.Windows.Forms;
using NiceVision.Common.Utils;
 

namespace NiceVision.Test.TestGUI
{
public class ProdConsTestForm : NiceVision.Common.Forms.NvcForm
{
private NiceVision.Common.GUI.NvcButton addProducerButton;
private NiceVision.Common.GUI.NvcButton addConsumerButton;
private System.ComponentModel.IContainer components = null;
private QueueCB myQueue = null;
private System.Random rand = null;
private int numOfProducers = 0;
private System.Windows.Forms.GroupBox groupBox1;
private System.Windows.Forms.GroupBox groupBox2;
private NiceVision.Common.GUI.NvcTextBox producerCountTextBox;
private NiceVision.Common.GUI.NvcLabel nvcLabel1;
private NiceVision.Common.GUI.NvcLabel nvcLabel2;
private NiceVision.Common.GUI.NvcTextBox consumerCountTextBox;
private NiceVision.Common.GUI.NvcStatusBar nvcStatusBar1;
private NiceVision.Common.GUI.NvcLabel nvcLabel3;
private System.Windows.Forms.NumericUpDown producerSpeedNumericUpDown;
private System.Windows.Forms.NumericUpDown consumerSpeedNumericUpDown;
private NiceVision.Common.GUI.NvcLabel nvcLabel4;
private NiceVision.Common.GUI.NvcComboBox producersComboBox;
private NiceVision.Common.GUI.NvcLabel nvcLabel5;
private NiceVision.Common.GUI.NvcButton removeProducerButton;
private NiceVision.Common.GUI.NvcLabel nvcLabel6;
private NiceVision.Common.GUI.NvcComboBox consumersComboBox;
private NiceVision.Common.GUI.NvcButton removeConsumersButton;
private int numOfConsumers = 0;
 

public ProdConsTestForm()
{
// This call is required by the Windows Form Designer.
InitializeComponent();
 
// TODO: Add any initialization after the InitializeComponent call
myQueue = new QueueCB(10);
rand = new System.Random();
}
 
///
/// Clean up any resources being used.
///

protected override void Dispose( bool disposing )
{
if( disposing )
{
if (components != null)
{
components.Dispose();
}
}
base.Dispose( disposing );
}
 
#region Designer generated code
///
/// Required method for Designer support - do not modify
/// the contents of this method with the code editor.
///

private void InitializeComponent()
{
this.components = new System.ComponentModel.Container();
this.addProducerButton = new NiceVision.Common.GUI.NvcButton(this.components);
this.addConsumerButton = new NiceVision.Common.GUI.NvcButton(this.components);
this.groupBox1 = new System.Windows.Forms.GroupBox();
this.producersComboBox = new NiceVision.Common.GUI.NvcComboBox(this.components);
this.producerSpeedNumericUpDown = new System.Windows.Forms.NumericUpDown();
this.nvcLabel3 = new NiceVision.Common.GUI.NvcLabel(this.components);
this.nvcLabel1 = new NiceVision.Common.GUI.NvcLabel(this.components);
this.producerCountTextBox = new NiceVision.Common.GUI.NvcTextBox(this.components);
this.groupBox2 = new System.Windows.Forms.GroupBox();
this.consumerSpeedNumericUpDown = new System.Windows.Forms.NumericUpDown();
this.nvcLabel4 = new NiceVision.Common.GUI.NvcLabel(this.components);
this.nvcLabel2 = new NiceVision.Common.GUI.NvcLabel(this.components);
this.consumerCountTextBox = new NiceVision.Common.GUI.NvcTextBox(this.components);
this.nvcStatusBar1 = new NiceVision.Common.GUI.NvcStatusBar(this.components);
this.nvcLabel5 = new NiceVision.Common.GUI.NvcLabel(this.components);
this.removeProducerButton = new NiceVision.Common.GUI.NvcButton(this.components);
this.nvcLabel6 = new NiceVision.Common.GUI.NvcLabel(this.components);
this.consumersComboBox = new NiceVision.Common.GUI.NvcComboBox(this.components);
this.removeConsumersButton = new NiceVision.Common.GUI.NvcButton(this.components);
this.groupBox1.SuspendLayout();
((System.ComponentModel.ISupportInitialize)(this.producerSpeedNumericUpDown)).BeginInit();
this.groupBox2.SuspendLayout();
((System.ComponentModel.ISupportInitialize)(this.consumerSpeedNumericUpDown)).BeginInit();
this.SuspendLayout();
//
// addProducerButton
//
this.addProducerButton.BackColor = System.Drawing.Color.WhiteSmoke;
this.addProducerButton.Location = new System.Drawing.Point(16, 24);
this.addProducerButton.Name = "addProducerButton";
this.addProducerButton.Size = new System.Drawing.Size(48, 23);
this.addProducerButton.TabIndex = 0;
this.addProducerButton.Text = "Add";
this.addProducerButton.Click += new System.EventHandler(this.addProducerButton_Click);
//
// addConsumerButton
//
this.addConsumerButton.BackColor = System.Drawing.Color.WhiteSmoke;
this.addConsumerButton.Location = new System.Drawing.Point(16, 24);
this.addConsumerButton.Name = "addConsumerButton";
this.addConsumerButton.Size = new System.Drawing.Size(48, 23);
this.addConsumerButton.TabIndex = 1;
this.addConsumerButton.Text = "Add";
this.addConsumerButton.Click += new System.EventHandler(this.addConsumerButton_Click);
//
// groupBox1
//
this.groupBox1.Controls.AddRange(new System.Windows.Forms.Control[] {
this.removeProducerButton,
this.nvcLabel5,
this.producersComboBox,
this.producerSpeedNumericUpDown,
this.nvcLabel3,
this.nvcLabel1,
this.producerCountTextBox,
this.addProducerButton});
this.groupBox1.Location = new System.Drawing.Point(8, 8);
this.groupBox1.Name = "groupBox1";
this.groupBox1.Size = new System.Drawing.Size(368, 144);
this.groupBox1.TabIndex = 2;
this.groupBox1.TabStop = false;
this.groupBox1.Text = "Producer";
//
// producersComboBox
//
this.producersComboBox.DropDownStyle = System.Windows.Forms.ComboBoxStyle.DropDownList;
this.producersComboBox.Location = new System.Drawing.Point(88, 112);
this.producersComboBox.Name = "producersComboBox";
this.producersComboBox.Size = new System.Drawing.Size(264, 21);
this.producersComboBox.TabIndex = 5;
//
// producerSpeedNumericUpDown
//
this.producerSpeedNumericUpDown.Location = new System.Drawing.Point(88, 88);
this.producerSpeedNumericUpDown.Name = "producerSpeedNumericUpDown";
this.producerSpeedNumericUpDown.Size = new System.Drawing.Size(40, 20);
this.producerSpeedNumericUpDown.TabIndex = 4;
this.producerSpeedNumericUpDown.Value = new System.Decimal(new int[] {
1,
0,
0,
0});
//
// nvcLabel3
//
this.nvcLabel3.BackColor = System.Drawing.Color.Transparent;
this.nvcLabel3.Location = new System.Drawing.Point(16, 88);
this.nvcLabel3.Name = "nvcLabel3";
this.nvcLabel3.Size = new System.Drawing.Size(72, 16);
this.nvcLabel3.TabIndex = 3;
this.nvcLabel3.Text = "Speed (sec.)";
//
// nvcLabel1
//
this.nvcLabel1.BackColor = System.Drawing.Color.Transparent;
this.nvcLabel1.Location = new System.Drawing.Point(16, 56);
this.nvcLabel1.Name = "nvcLabel1";
this.nvcLabel1.Size = new System.Drawing.Size(40, 16);
this.nvcLabel1.TabIndex = 2;
this.nvcLabel1.Text = "Count";
//
// producerCountTextBox
//
this.producerCountTextBox.Location = new System.Drawing.Point(56, 56);
this.producerCountTextBox.Name = "producerCountTextBox";
this.producerCountTextBox.ReadOnly = true;
this.producerCountTextBox.Size = new System.Drawing.Size(56, 20);
this.producerCountTextBox.TabIndex = 1;
this.producerCountTextBox.Text = "0";
//
// groupBox2
//
this.groupBox2.Controls.AddRange(new System.Windows.Forms.Control[] {
this.removeConsumersButton,
this.nvcLabel6,
this.consumersComboBox,
this.consumerSpeedNumericUpDown,
this.nvcLabel4,
this.nvcLabel2,
this.consumerCountTextBox,
this.addConsumerButton});
this.groupBox2.Location = new System.Drawing.Point(8, 160);
this.groupBox2.Name = "groupBox2";
this.groupBox2.Size = new System.Drawing.Size(368, 144);
this.groupBox2.TabIndex = 3;
this.groupBox2.TabStop = false;
this.groupBox2.Text = "Consumer";
//
// consumerSpeedNumericUpDown
//
this.consumerSpeedNumericUpDown.Location = new System.Drawing.Point(88, 88);
this.consumerSpeedNumericUpDown.Name = "consumerSpeedNumericUpDown";
this.consumerSpeedNumericUpDown.Size = new System.Drawing.Size(40, 20);
this.consumerSpeedNumericUpDown.TabIndex = 6;
this.consumerSpeedNumericUpDown.Value = new System.Decimal(new int[] {
1,
0,
0,
0});
//
// nvcLabel4
//
this.nvcLabel4.BackColor = System.Drawing.Color.Transparent;
this.nvcLabel4.Location = new System.Drawing.Point(16, 88);
this.nvcLabel4.Name = "nvcLabel4";
this.nvcLabel4.Size = new System.Drawing.Size(72, 16);
this.nvcLabel4.TabIndex = 5;
this.nvcLabel4.Text = "Speed (sec.)";
//
// nvcLabel2
//
this.nvcLabel2.BackColor = System.Drawing.Color.Transparent;
this.nvcLabel2.Location = new System.Drawing.Point(16, 56);
this.nvcLabel2.Name = "nvcLabel2";
this.nvcLabel2.Size = new System.Drawing.Size(40, 16);
this.nvcLabel2.TabIndex = 4;
this.nvcLabel2.Text = "Count";
//
// consumerCountTextBox
//
this.consumerCountTextBox.Location = new System.Drawing.Point(56, 56);
this.consumerCountTextBox.Name = "consumerCountTextBox";
this.consumerCountTextBox.ReadOnly = true;
this.consumerCountTextBox.Size = new System.Drawing.Size(56, 20);
this.consumerCountTextBox.TabIndex = 3;
this.consumerCountTextBox.Text = "0";
//
// nvcStatusBar1
//
this.nvcStatusBar1.Location = new System.Drawing.Point(0, 319);
this.nvcStatusBar1.Name = "nvcStatusBar1";
this.nvcStatusBar1.Size = new System.Drawing.Size(384, 22);
this.nvcStatusBar1.TabIndex = 4;
//
// nvcLabel5
//
this.nvcLabel5.BackColor = System.Drawing.Color.Transparent;
this.nvcLabel5.Location = new System.Drawing.Point(16, 112);
this.nvcLabel5.Name = "nvcLabel5";
this.nvcLabel5.Size = new System.Drawing.Size(60, 16);
this.nvcLabel5.TabIndex = 6;
this.nvcLabel5.Text = "Producers";
//
// removeProducerButton
//
this.removeProducerButton.BackColor = System.Drawing.Color.WhiteSmoke;
this.removeProducerButton.Location = new System.Drawing.Point(72, 24);
this.removeProducerButton.Name = "removeProducerButton";
this.removeProducerButton.Size = new System.Drawing.Size(56, 23);
this.removeProducerButton.TabIndex = 7;
this.removeProducerButton.Text = "Remove";
this.removeProducerButton.Click += new System.EventHandler(this.removeProducerButton_Click);
//
// nvcLabel6
//
this.nvcLabel6.BackColor = System.Drawing.Color.Transparent;
this.nvcLabel6.Location = new System.Drawing.Point(16, 112);
this.nvcLabel6.Name = "nvcLabel6";
this.nvcLabel6.Size = new System.Drawing.Size(60, 16);
this.nvcLabel6.TabIndex = 8;
this.nvcLabel6.Text = "Consumers";
//
// consumersComboBox
//
this.consumersComboBox.DropDownStyle = System.Windows.Forms.ComboBoxStyle.DropDownList;
this.consumersComboBox.Location = new System.Drawing.Point(88, 112);
this.consumersComboBox.Name = "consumersComboBox";
this.consumersComboBox.Size = new System.Drawing.Size(264, 21);
this.consumersComboBox.TabIndex = 7;
//
// removeConsumersButton
//
this.removeConsumersButton.BackColor = System.Drawing.Color.WhiteSmoke;
this.removeConsumersButton.Location = new System.Drawing.Point(72, 24);
this.removeConsumersButton.Name = "removeConsumersButton";
this.removeConsumersButton.Size = new System.Drawing.Size(56, 23);
this.removeConsumersButton.TabIndex = 9;
this.removeConsumersButton.Text = "Remove";
this.removeConsumersButton.Click += new System.EventHandler(this.removeConsumersButton_Click);
//
// ProdConsTestForm
//
this.AutoScaleBaseSize = new System.Drawing.Size(5, 13);
this.ClientSize = new System.Drawing.Size(384, 341);
this.Controls.AddRange(new System.Windows.Forms.Control[] {
this.nvcStatusBar1,
this.groupBox2,
this.groupBox1});
this.Name = "ProdConsTestForm";
this.Text = "Producer/Consumer Tester";
this.groupBox1.ResumeLayout(false);
((System.ComponentModel.ISupportInitialize)(this.producerSpeedNumericUpDown)).EndInit();
this.groupBox2.ResumeLayout(false);
((System.ComponentModel.ISupportInitialize)(this.consumerSpeedNumericUpDown)).EndInit();
this.ResumeLayout(false);
 
}
#endregion
 
private void addProducerButton_Click(object sender, System.EventArgs e)
{
// Create the thread object, passing in the invoke event method
// via a ThreadStart delegate. This does not start the thread.
ProdConsThread oThread = new ProdConsThread(new ThreadStart(AddProducer));
 
// Start the thread
oThread.Start();
 
producersComboBox.Items.Add(oThread);
producersComboBox.SelectedIndex = 0;

numOfProducers++;
producerCountTextBox.Text = numOfProducers + "";
}
 
public void AddProducer()
{
int speed = (int)producerSpeedNumericUpDown.Value * 1000;
while(true)
{
try
{
Thread.Sleep(speed);
int data = rand.Next(0, 100);
myQueue.EnqueueBlocking(data);
Console.WriteLine("[Producer][Thread{0}][Data:{1}][Speed:{2}][QueueCount:{3}]", Thread.CurrentThread.GetHashCode(), data, speed, myQueue.Count);
}
catch(Exception e)
{
Console.WriteLine("[ProducerException][Thread{0}][Info:{1}]", Thread.CurrentThread.GetHashCode(), e.Message);
}
}
}

private void addConsumerButton_Click(object sender, System.EventArgs e)
{
// Create the thread object, passing in the invoke event method
// via a ThreadStart delegate. This does not start the thread.
ProdConsThread oThread = new ProdConsThread(new ThreadStart(AddConsumer));
 
// Start the thread
oThread.Start();

consumersComboBox.Items.Add(oThread);
consumersComboBox.SelectedIndex = 0;
 
numOfConsumers++;
consumerCountTextBox.Text = numOfConsumers + "";
}
 
public void AddConsumer()
{
int speed = (int)consumerSpeedNumericUpDown.Value * 1000;
while(true)
{
try
{
Thread.Sleep(speed);
int data = (int) myQueue.DequeueBlocking();
Console.WriteLine("[Consumer][Thread{0}][Data:{1}][Speed:{2}][QueueCount:{3}]", Thread.CurrentThread.GetHashCode(), data, speed, myQueue.Count);
}
catch(Exception e)
{
Console.WriteLine("[ConsumerException][Thread{0}][Info:{1}]", Thread.CurrentThread.GetHashCode(), e.Message);
}
}
}
 
private void removeProducerButton_Click(object sender, System.EventArgs e)
{
ProdConsThread oThread = null;
try
{
oThread = (ProdConsThread)producersComboBox.SelectedItem;
oThread.Stop();
 
numOfProducers--;
producerCountTextBox.Text = numOfProducers + "";
 
producersComboBox.Items.Remove(oThread);
producersComboBox.SelectedIndex = 0;
}
catch(Exception exp){}
}
 
private void removeConsumersButton_Click(object sender, System.EventArgs e)
{
ProdConsThread oThread = null;
try
{
oThread = (ProdConsThread)consumersComboBox.SelectedItem;
oThread.Stop();
 
numOfConsumers--;
consumerCountTextBox.Text = numOfConsumers + "";
 
consumersComboBox.Items.Remove(oThread);
consumersComboBox.SelectedIndex = 0;
}
catch(Exception exp){}
}
 
}
}
 

GeneralRe: Found bug - I guess PinmemberRobert Hinrichs30-Jan-03 7:48 
GeneralRe: Info? PinmemberRobert Hinrichs30-Jan-03 7:51 
GeneralRe: Info? Pinmemberorenderi1-Feb-03 22:25 
GeneralRe: Info? PinmemberJohnny_B15-Apr-04 17:59 
GeneralRe: Info? Pinmembercodehacker3810-May-04 13:59 
GeneralRe: Info? Pinmembermsquare12-Feb-09 9:27 
GeneralGreat project - can I use it for my app Pinmemberorenderi29-Jan-03 21:55 
GeneralRe: Great project - can I use it for my app PinmemberRobert Hinrichs30-Jan-03 3:21 
GeneralRe: Great project - can I use it for my app Pinmemberorenderi30-Jan-03 6:10 
QuestionCan this be used for interprocess communication? Pinmembersytelus28-Jan-03 22:08 
AnswerRe: Can this be used for interprocess communication? PinmemberRobert Hinrichs29-Jan-03 4:22 
Generalconstructor PinsussLilly20-Nov-02 11:01 
GeneralRe: constructor PinsussBobH20-Nov-02 13:47 
GeneralRe: constructor PinsussLilly21-Nov-02 5:23 
GeneralRe: constructor PinsussBobH22-Nov-02 3:12 
GeneralMissing files Pinmemberleppie12-Sep-02 13:07 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.141223.1 | Last Updated 12 Sep 2002
Article Copyright 2002 by Robert Hinrichs
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid