I have been writing server applications on Windows server platforms for quite sometime. We all know that I/O completion port is one of the nicest things available for writing server applications (or even in clients where you can apply Worker-Boss pattern). There are tons of articles and samples to show how I/O completion port can be used with TCP/IP. Surprisingly enough, I did not find any source code examples to demonstrate the use of I/O completion ports with UDP.
To spice up things a little, I took the multicast topic to demonstrate the code. Most of the multicast code is borrowed from Mr. Bob Quinn's article.
This sample consists of two projects (Client and Server). In order to test this application as it is, routers in your network should be enabled for multicast routing. If not, modify samples appropriately to act just UDP server and client.
Since there is so much written about I/O completion port and multicast, I won't go into all the details in this article.
Server application design and implementation is an art. You can put so many bells and whistles. Typical server application should have the following:
If any of the object is to be created and destroyed often, consider making a pool of such objects. Typical example would be your sockets, user context objects, database connections.
Ease of configuration
Do not hardcode configuration information in the code. Try to use XML configuration files. Let the application read these vales during the startup. Typical examples would be, IP address of other servers, Oracle TNS names, user ids and passwords (!!!), number of worker threads, email addresses etc.
Besides, there should be some mechanism to change these parameters while the server is running.
Monitoring and logging
Since most of these servers will be running somewhere in the data center, it is nice to have a small component sitting in your server application which will send log messages to central server from where you multicast those messages to other client applications. Make sure you have very good filtering mechanism so that you can control the number of messages being logged during the run time.
Your servers have to be running nonstop (four nines most of the time). That means you should have proper error handling mechanism. Make use of SEH in a responsible manner. Besides keep track of number of TCP connections (using SNMP) and CPU usage, memory usage (Microsoft provides a library by which you can query all these parameters from within your program).
Make sure you thread protects the shared data.
You can not serve infinite number of clients from one server box. Always keep the 'scale-out' factor in mind. Do proper capacity planning to quantify your server hardware requirement. 80-20 rule is quite effective.
#define BUFSIZE 1024 #define MAXADDRSTR 16
#define DEFAULT_GROUP_ADDRESS "22.214.171.124"
#define DEFAULT_PORT 7125
LONG nCount = 0;
DWORD WINAPI WorkerThread( LPVOID WorkContext );
BOOL HandleIncomingData( UCHAR* pBuf);
BOOL CreateNetConnections( VOID );
BOOL CreateWorkers( UINT );
UCHAR achInBuf [BUFSIZE];
char achMCAddr[MAXADDRSTR] = DEFAULT_GROUP_ADDRESS;
u_short nPort = DEFAULT_PORT;
int ret = 0;
version = (MAKEWORD(2, 2));
ret = WSAStartup(version, &data);
if (ret != 0)
ret = WSAGetLastError();
if (ret == WSANOTINITIALISED)
BOOL CreateNetConnections (void)
BOOL fFlag = TRUE;
struct ip_mreq stMreq;
g_hSocket = socket(AF_INET, SOCK_DGRAM,0);
if (g_hSocket == INVALID_SOCKET)
printf ("socket() failed, Err: %d\n", WSAGetLastError());
nRet = setsockopt(g_hSocket,SOL_SOCKET,
SO_REUSEADDR, (char *)&fFlag, sizeof(fFlag));
if (nRet == SOCKET_ERROR)
printf ("setsockopt() SO_REUSEADDR failed,
stLclAddr.sin_family = AF_INET;
stLclAddr.sin_addr.s_addr = htonl(INADDR_ANY);
stLclAddr.sin_port = htons(nPort);
nRet = bind(g_hSocket,(struct sockaddr*) &stLclAddr,sizeof(stLclAddr));
if (nRet == SOCKET_ERROR)
printf ("bind() port: %d failed, Err: %d\n",
stMreq.imr_multiaddr.s_addr = inet_addr(achMCAddr);
stMreq.imr_interface.s_addr = INADDR_ANY;
nRet = setsockopt(g_hSocket,IPPROTO_IP,
if (nRet == SOCKET_ERROR)
printf("setsockopt() IP_ADD_MEMBERSHIP address %s failed,
g_hCompletionPort = CreateIoCompletionPort (INVALID_HANDLE_VALUE,
fprintf (stdout, "g_hCompletionPort Create Failed\n");
Overlapped.hEvent = g_hReadEvent;
Overlapped.Internal = 0;
Overlapped.InternalHigh = 0;
Overlapped.Offset = 0;
Overlapped.OffsetHigh = 0;
b = ReadFile ((HANDLE)g_hSocket,&achInBuf,
if (!b && GetLastError () != ERROR_IO_PENDING)
fprintf (stdout, "ReadFile Failed\n");
BOOL CreateWorkers (UINT dwNumberOfWorkers)
for (i = 0; i < dwNumberOfWorkers; i++)
ThreadHandle = CreateThread (NULL,0,
fprintf (stdout, "Create Worker Thread Failed\n");
DWORD WINAPI WorkerThread (LPVOID WorkContext)
b = GetQueuedCompletionStatus(g_hCompletionPort,
if (b || lpo)
ol.hEvent = g_hReadEvent;
ol.Offset = 0;
ol.OffsetHigh = 0;
b = ReadFile ((HANDLE)nSocket,&ReadBuffer,
if (!b )
DWORD dwErrCode = GetLastError();
if( dwErrCode != ERROR_IO_PENDING )
printf("Something has gone
wrong:Error code - %d\n",dwErrCode );
SUBLANG_DEFAULT), (LPTSTR) &lpMsgBuf, 0, NULL);
else if( dwErrCode == ERROR_IO_PENDING )
fprintf (stdout, "WorkThread Wait Failed\n");
BOOL HandleIncomingData( UCHAR* pBuf)
lpstSysTime = (SYSTEMTIME *)(pBuf);
printf("[%d]UTC Time %02d:%02d:%02d:%03d on %02d-%02d-%d \n",nCount,
lpstSysTime->wMonth, lpstSysTime->wDay, lpstSysTime->wYear);
printf("Group IP address: %s\n",achMCAddr);
printf("Port number : %d\n",nPort);
HANDLE hWait2Exit = CreateEvent(NULL,FALSE,TRUE,"MCLIENT");
g_hReadEvent = CreateEvent(NULL,TRUE,TRUE,NULL);
SetThreadPriority (GetCurrentThread (), THREAD_PRIORITY_TIME_CRITICAL);
if (!CreateNetConnections ())
printf("Error condition @ CreateNetConnections , exiting\n");
if (!CreateWorkers (5))
printf("Error condition @CreateWorkers, exiting\n");