How to build a chat server based on an IOCP framework

• Download source files - 30 Kb

Introduction

This is my first article ever and all feedback is appreciated.

A while ago I needed a server framework that used I/O completion ports and I did not find any solutions that fit my need. Those that I found where either too complex or not designed for reusability. Therefore I started to make my own, and this is the result. All classes are fully commented with doxygen comments.

The framework

The server framework is basically just two classes, and those are described below.

DtServerSocket

DtServerSocket handles the listen socket, keeps track of all clients, contains the IO Completion function and a maintenance function.

class DtServerSocket
{

protected:
	SOCKET		m_sdListen;  	/// Socket used to listen.
	DWORD		m_dwPort;	/// Port that we listen on.
	size_t		m_nClients;  	/// Number of loaded clients
	size_t		m_nMaxClients;	/// Maximum number of clients that we may have.
	DWORD		m_dwServerFull;	/// When the server got full.
	typedef vector< DtServerSocketClient* > CLIENTS;
	CLIENTS		m_paClients;

	DtCriticalSection m_cs;

public:
	DtServerSocket(void);
	~DtServerSocket(void);


	/// Load a client, the only thing you have to do is to derive this method.
	/// @param pClient Should return a pointer to a DtServerSocketClient derived class.
	/// @param nId A id that identifieds the client.
	virtual void LoadClient(DtServerSocketClient** pClient, int& nId) = 0;


	/// @param dwListenPort The port that we should wait for connections on.
	/// @param nClients Number of initial clients
	/// @param nMaxClients Total number of clients that the server could have.
	/// @returns 0 if success or system error code.
	DWORD StartServer(DWORD dwListenPort, int nClients, int nMaxClients);

	/// Stops the server.
	void StopServer(void);

	/// Check's how long the clients have been connected.
	/// You should run this function in your main loop.
	virtual void Maintenance();

	/// IoCompletion of the client threads calls this method.
	static void CALLBACK DoneIO(DWORD dwErrorCode,
		DWORD dwNumberOfBytesTransferred,
		LPOVERLAPPED lpOverlapped);


	/// Override this one to give the server a name. Gr8 for debugging purposes.
	virtual const char* GetServerName() { return "A server"; };

	/// Override this one to get log printings
	virtual void OnWriteLog(int nPrio, int nClientId, const char* pszCategory,
                 const char* pszString) const {};
	void WriteLog(int nPrio, int nClientId, const char* pszCategory,
	             const char* pszString, ...) const;

};

We derive a class called CChatServerSocket from DtServerSocket and override two functions:

	/// this function is used to load a new client that is derived from DtServerSocketClient
	virtual void LoadClient(Datatal::DtServerSocketClient** pClient, int& nId);

	/// Useful for diagnostics if we run diffrent servers in the same .exe
	const char* GetServerName() { return "ChatServer"; };

DtServerSocketClient

DtServerSocketClient contains all i/o functions for each client that have been accepted by DtServerSocket.

Sending data

To handle output buffering we have implented a first-in/first-out linked list to enqueue all outgoing data:

	/// Container for our outbuffers.
	struct Outbuffer
	{
		char* pBuffer;
		DWORD dwSize;
		Outbuffer* pNext;
	};
	/// Linked list with all our outbuffers.
	struct OutbufferList
	{
		Outbuffer* pFirst;
		Outbuffer* pLast;
		OutbufferList()
		{
			pFirst = NULL;
			pLast = NULL;
		}
		void Append(char* pBuffer, DWORD dwSize)
		{
			if (!pBuffer || !dwSize) throw std::invalid_argument("pBuffer and nSize cannot be NULL");

			Outbuffer* pNewNode = new Outbuffer;
			pNewNode->pBuffer = pBuffer;
			pNewNode->pNext = NULL;
			pNewNode->dwSize = dwSize;

			if (pLast)
				pLast->pNext = pNewNode;
			else
				pFirst = pNewNode;

			pLast = pNewNode;
		}
		void RemoveFirst()
		{
			if (!pFirst) throw std::out_of_range("pFirst is NULL");
			Outbuffer* pOld = pFirst;
			pFirst = pFirst->pNext;
			if (pOld == pLast) pLast = NULL;
			delete[] pOld->pBuffer;
			delete pOld;
		}
	};
	

When we send data, it is simply added to the list and a WriteOperation is invoked:

/// Enqueue stuff to our outgoing buffer.
bool Datatal::DtServerSocketClient::Send(char* data, int nSize)
{
	//Lock outbuffer
	m_CritWrite.Lock();
	m_lOutBuffers.Append(data, (DWORD)nSize);
	m_CritWrite.Unlock();
	WriteLog(Datatal::LP_NORMAL, GetClientId(), "Send", "Appending new outbuffer, size: %d", nSize);

	//Trigger that we got a write operation
	WriteOperation();
	return true;
}

ChatServer

The chatserver class implementation:

DtServerSocket -> CChatServerSocket

DtServerSocket	Base class for all IOCP servers. (included in the DtLibrary)
CChatServerSocket	Contain functions to send chat messages to all/specific clients.

DtServerSocketClient -> ChatProtocol -> CChatServerClient

DtServerSocketClient	Base class for all client sockets in the IOCP servers. (included in the DtLibrary)
ChatProtocol	Chat Protocol Layer
CChatServerClient	Client layer, keeps track of the user (logged in, username etc)

Designing the protocol

The first thing that we have to do is to create a protocol that will be used to send data back and forth between client/server. The protocol is implemented as a struct (data container), an enum (function codes) and finally another enum for the status codes..

<STX><USHORT><DWORD><CHAR><databuffer><ETX>

STX	ASCII 0x02, start transaction, tells us that this is the beginning of the transaction
USHORT	which function that we want to run
DWORD	size of the databuffer.
CHAR	status code. Will be changed to an error code if something fails.
databuffer	All data that will be sent between the client/server is packed in this char buffer.
ETX	ASCII 0x03, end transaction, is used to confirm that we got a complete transaction..

Here is everything translated into code:

  /// In this enum we define the different transaction codes..
  enum TRANS_CODES
  {
    TC_LOGIN,            /// want to login
    TC_LIST_CHANNELS,    /// List all channels
    TC_LIST_USERS,       /// list all users / users on a specific channel
    TC_SEND_MESSAGE,     /// Set a message to a channel/user
    TC_SEND_MESSAGE_OUT  /// Outgoing message, (sent to client)
  };

         /// Status codes.
  enum TRANS_STATUS
  {
    TS_OK,        /// Everything went ok
    TS_NO_DATA,   /// Everything went ok, but we got no data
    TS_INVALID,   /// Invalid transaction
    TS_ERROR,     /// Something went wrong, presumably invalid data.
    TS_NO_ACCESS, /// The user do not have the required credentials 
    TS_EXCEPTION  /// Something unexpected happened.
  };

  /// Packet Structure - The heart of the protocol.
  struct Packet
  {
    USHORT	nFunctionCode;		/// functioncode
    DWORD 	dwDataSize;    		/// size of the data stored in pData
    char 	Status;				/// statuscode. 0 = ok;
    char* 	pData;				/// Buffer
    DWORD 	dwBufferSize;		/// maxsize of pData (allocated size)
    ~Packet()   { if (pData) delete[] pData;  };
    Packet()
    {
      pData = NULL;
      nFunctionCode = 0;
      dwDataSize = 0;
      dwBufferSize = 0;
      Status = 0;
    };
    Packet(int FunctionCode)
    {
      pData = NULL;
      nFunctionCode = FunctionCode;
      dwDataSize = 0;
      dwBufferSize = 0;
      Status = 0;
    };
  };

Implementing CChatServer

We create a class called CChatServer, derive it from DtServerSocket, and implement the LoadClient function:

void CChatServerSocket::LoadClient(Datatal::DtServerSocketClient** pClient, 
                                   int nId)
{
  CChatServerClient* pNewClient = new CChatServerClient;
  if (pNewClient)
  {

    // You may call whatever you want to init the server client.

  } 
  else
  {
    char szLog[128];
    sprintf(szLog, "Failed to load hvd client %d", nId);
    throw Datatal::DtServerException(-1, "LoadClient", szLog);

  }// if (pNewClient)

  *pClient = pNewClient;
}

Since we do not want to do any extra initial operations on the clients, we just create them and pass them back to the base class.

Next thing is to create a ChatProtocol, and a class that implements the protocol.

The ChatProtocol will be used both in the server and the client application. The Protocol struct and the enums are placed in this class along with some static functions that help us to pack the data buffer into the packet. ChatProtocol is derived from DtServerSocketClient and implement functions that translates our packets into plain char buffers and then passes/retrieve them to/from DtServerSocketClient.

Create ChatServerSocketClient class and implement the logic in it.

Normally I handle all data using CMarkup, a nice xml class from http://www.firstobject.com/, but in this example the data is separated with 0x04 and I use strtok to unpack everything. The class is derived from ChatProtocol and we will implement all logic in it.

Functions implemented in the chatserver

• Users cannot do anything unless they have logged in.
• All users will continue to be logged in until the client connection is closed.
• While logged in they can send messages to everyone or someone particular.
• Users can retrieve a list of all logged in users.

Summary

That's it. I will not describe the server any further, just take a look in the code. A fully functional client have been uploaded in a separate project.

References

The server framework is based on the IOCP example made by Ben Eleizer, http://www.codeproject.com/internet/iocp.asp, although it's quite heavily modified. If you want more details about IOCP read that article.

Another good article about IOCP by Microsoft can be found at: http://msdn.microsoft.com/msdnmag/issues/1000/winsock/

The client can be found at http://www.codeproject.com/internet/chatclient.asp

History

• 2003-08-08 First version.
• 2003-08-19 Article updated.