A Persistent Circular Queue Buffer

• Download source - 10.2 KB

Introduction

The inspiration for this piece of code was the following article which talks about a persistent message broker:
https://www.oreilly.com/library/view/zeromq/9781449334437/ch04s12.html

The article had the following improvement suggestions:

• Use a single disk file for all data, rather than multiple files. Operating systems are usually better at handling a few large files than many smaller ones.

• Organize that disk file as a circular buffer so that new requests can be written contiguously (with very occasional wraparound). One thread, writing full speed to a disk file, can work rapidly.

Hence came up with the first step of implementing the suggestions. Currently we are able to write data in circular fashion in the disk and also read from it consistently anytime even when the program restarts.

The original code was taken from this link but it didn't implement a circular queue properly: 

https://stackoverflow.com/questions/33793535/how-to-be-sure-data-is-on-disk-after-fwrite

After modifying the codes a little I was able to get it working.

Using the code

Using the code is pretty simple.

First push some items in the queue after specifying a filename which stores the data.

//  Templated declaration  
    FixedSizeQueue<int> fq;
//  Specify the filename
    fq.open("sample_file.dat", 6);

//  Push some data

    int a1[] = {0, 1, 2};
    int a2[] = {3, 4};
    fq.push_values(a1, 3);
    fq.push_values(a2, 2);

Then fetch the saved data with a get_values calls

//  fetch data

int values[100];
size_t numberofvalues = fq.get_values(values);

Do remember to include the header file : circularqueue.h

Points of Interest 

Using a different file to store the head, tail and size info so that next time the program restarts, the head, tail and size can be restored properly and the queue acts as a persistent circular queue.

History

Code submitted: 11th August, 2022

Code updated: 20th August, 2022