Click here to Skip to main content
Click here to Skip to main content

SAFMQ Store and Forward Message Queue

, 16 Jan 2006
Rate this:
Please Sign up or sign in to vote.
An OpenSource cross-compilable/cross-platform message queue server like MSMQ or MQSeries.

Introduction

The SAFMQ server provides asynchronous messaging. Message Publishers send or enqueue a message with the SAFMQ server. The SAFMQ server stores that message until the point at which the message can be forwarded on to the client. Message Publishers are assured that the messages are delivered. That's how SAFMQ got its name.

Timely Messaging

SAFMQ provides the ability to perform timely message delivery. So, if a Message Publisher wants a message to be read by a Message Subscriber in a certain amount of time, or not read at all, then the Message Publisher can prescribe a Time-To-Live for the message it publishes. SAFMQ will notify the Message Publisher about messages which have outlived their Time-To-Live, or a Message Publisher can choose to ignore the event.

Round-Trip, PsudoSynchronous Messaging

Round-Trip, or PsudoSynchronous Messaging is when a Message Publisher acts as a Message Subscriber after sending a "query" message. A Message Publisher may want to receive information back from the Message Subscriber. Thus, after the first message is sent by a Message Publisher and is received by a Message Subscriber, the original Message Publisher and Message Subscriber switch rolls.

SAFMQ provides a special messaging context element for Round-Trip/PsudoSynchronous Messaging. It's called a Receipt ID. Whenever a message is enqueued in a SAFMQ server, it is given a Universally Unique Identifier or UUID for short. When a round-trip message event is taking place, the original Message Subscriber publishes a message with a Receipt ID identical to the Message ID assigned to the message sent by the original Message Publisher. Then the original Message Publisher (now a subscriber) waits for a message with a Receipt ID equal to the Message ID the original Message Publisher sent.

Batch Processing

Not every task is best handled real-time. Sometimes, there are real money benefits to sending transactions to a trading partner in a large group or batch. SAFMQ can be an intermediary between real-time systems and a back-end batch processor. The real-time system knows that the messages will be delivered, and the batch processor can let data queue up until it is ready to send all the data. The batch system can even respond via SAFMQ, and a real-time system can instantly see the results.

Background

SAFMQ started as an in-house project at the Prescription Benefit Manager (PBM) I currently work for. In our line of business, we receive prescription drug claims from pharmacies, and are given approximately 10 seconds or less to process the claim, determine the price of the drug being purchased, determine the discount the cardholder's group has at that pharmacy, determine the patient's copy, how much funds he or she has left in his or her account, and send all that information back to the pharmacy, all while the customer is waiting at the counter. It's all a little like processing credit cards for purchases. That being said, we need a way to look at claims as they happen, real-time, but the database the production claim adjudicator is using can't be accessed without the chance of slowing down the claim processing. To fix the problem, we built SAFMQ and publish claims (in XML format) to the message queue real-time, and another system reads the claim details and writes them to a data warehouse type database, enabling our customer service agents the ability to lookup claim results real-time.

Using the code

Here's an example showing the number of messages that can be enqueued with the server per second:

/*
This demo program creates a queue named 
"test", writes 10000 messages,
then determines how ling it took to write 
the messages to the server and outputs the results.
*/

#include <windows.h>
#include <iostream>
#include "lib/MessageQueue.h"
#include "lib/MQFactory.h"
#include "lib/MQConnection.h"

using namespace safmq;
using namespace std;

#define MCOUNT 10000
int main(int argc, char* argv[]) {
    try {
        ErrorCode     ec;
        // Build a connection to the server
        MQConnection* con = 
          MQFactory::BuildConnection("safmq://localhost","admin","");
        // remove the queue just in case it was already there
        con->DeleteQueue("test");
        // create the queue named "test"
        con->CreateQueue("test");

        // Open an instance of the queue
        MessageQueue *queue = new MessageQueue(con,"test",false);

        // Get the start time for our process
        DWORD start = GetTickCount();

        for (int x=0; x<MCOUNT; x++) { {
            QueueMessage msg;
            // Set up the message
            msg.setLabel("Test Message");
            msg.setBodyType(BT_TEXT);
            *msg.getBufferStream() << "Test test test" << endl;

            // Sed the message to the server
            ec = queue->Enqueue(msg);
            if (ec != EC_NOERROR) {
               cout << "Error Code: " << ec << endl;
               break;
            }
            // output some status dots
            if (x%100 == 0)
                putchar('.');
        } }
        putchar('\n');

        // Get the end time
        DWORD end = GetTickCount();

        con->DeleteQueue("test");
        delete queue;
        delete con;

        // output the stats on our run.
        double duration = (end-start)/1000.0;
        cout << "Durration: " << duration << endl;
        cout << "Rate:style='mso-spacerun:yes'>      " 
             << (double)MCOUNT / duration 
             << " Messages per second." << endl;

    } catch (ErrorCode e) {
        cout << "Error Code: " << e << endl;
    } catch (exception& e) {
        cout << e.what() << endl;
    }
    return 0;
}

Points of Interest

Some of the things I tried to use when designing the SAFMQ client API was using the I/O stream pattern. If you dig around in the code, you'll find a directory containing classes to implement a std::basic_iostream derived class that wraps socket communication. You might find, there is some repetition between the code written for SAFMQ and MFC. That's because the code is actually cross-compilable, and can be compiled on Linux and other UNIX based systems.

Additionally, there're classes for performing SSL communication either directly or via the I/O stream pattern. This way, you can chain together the type of underlying transport mechanism, and use class construction to determine whether you use SSL or not. You'll see this in action in the MQFactory class, which is part of the client API.

History

SAFMQ's home page is located at SourceForge, you'll find the documentation for the API setting up the server, and the Java interface, there.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

mattbattey
Architect
United States United States
A programmer for 20 years and professionaly employed for 12, I am currently Cheif Engineer for Pharmacy Chare Professionals, Inc., located in Omaha, NE.
 
My experience is in the area of OO Design, Application, and programming, technical team leadership, RDBMS applications, ISAM applications, Image Processing, Mathematical image generation, Client-Server business applications, eBusiness applications, XML & EDI B2B communications, Java application development, C/C++ application development, CFML/ASP/VB development, on systems like Win2K/NT/98/95, Linux, Irix, Solaris, and MacOS.

Comments and Discussions

 
GeneralStatus of this code PinmemberRobert Mecklenburg17-Oct-06 7:00 
GeneralRe: Status of this code Pinmembermattbattey20-Nov-06 14:13 
GeneralRe: Status of this code Pinmemberjmansion3-Jul-07 19:14 
Matt,
 
Are you still here as it were?
 
James

General/var/safmq/queues Pinmemberjm.alkema26-Aug-06 2:41 
GeneralE-Mail Bounced PinmemberNeville Franks13-Jul-06 12:21 
GeneralRe: E-Mail Bounced PinmemberBarryHolleran30-Apr-12 14:32 
GeneralRe: E-Mail Bounced PinmemberNeville Franks2-May-12 19:27 
GeneralAdd JMS support to the Java package PinmemberRonny AERTS6-Mar-06 5:40 
GeneralRe: Add JMS support to the Java package Pinmembermattbattey20-Nov-06 14:15 
Generalconfiguration file in xml format PinmemberRonny AERTS6-Mar-06 5:37 
GeneralInstalling SAFMQ on a windows machine as a service PinmemberRonny AERTS6-Mar-06 4:37 
GeneralRe: Installing SAFMQ on a windows machine as a service Pinmembermattbattey6-Mar-06 4:49 
GeneralInteresting++ PinmemberNeville Franks17-Jan-06 0:01 
GeneralRe: Interesting++ Pinmembermattbattey17-Jan-06 6:31 
GeneralRe: Interesting++ PinmemberNeville Franks17-Jan-06 19:24 
GeneralRe: Interesting++ Pinmembermattbattey18-Jan-06 4:43 
GeneralRe: Interesting++ Pinmemberrpompeo6817-Jan-06 23:29 
GeneralRe: Interesting++ Pinmembermattbattey18-Jan-06 4:53 
GeneralInteresting - Nice work .. PinmemberGarth J Lancaster16-Jan-06 22:52 
GeneralRe: Interesting - Nice work .. Pinmembermattbattey17-Jan-06 4:59 
GeneralRe: Interesting - Nice work .. PinmemberGarth J Lancaster17-Jan-06 10:15 
GeneralRe: Interesting - Nice work .. Pinmembermattbattey17-Jan-06 11:26 
GeneralRe: Interesting - Nice work .. PinmemberGarth J Lancaster17-Jan-06 11:37 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.140821.2 | Last Updated 16 Jan 2006
Article Copyright 2006 by mattbattey
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid