Introduction
First and foremost, the intention of this article is as a Tutorial. My intention is for this to become a loose standard or guideline for how networking interfaces should behave. (I also personally believe that USDTP is a useful protocol in its own right, however.)
What I am trying to say is that you please not just copy the code and replace every instance of "USDTP" with "MSCCP" (My Super Cool Copied Protocol) or something to that effect. This is designed to teach you how to build an application like this, not to just Copy/Paste large blocks of code. However, I would be greatly honored if you were to use this code "as is"; changing the protocol would really defeat the point of me creating the protocol in the first place.
I am, of course, entirely open to suggestions and the like.
Now, on to business...
Background
In this tutorial, we will be building a rather-complicated networking protocol and implementation based on TCP/IP in C#. This assumes at least a functioning knowledge of C# and .NET, but really nothing more than syntax and common sense. If you find yourself pouring over long blocks of stuff you already know, this is intentional: I'm trying to cover everything, so feel free to skip ahead.
The goal of this article is to create a protocol which will be named USDTP (Universal Segmented Data Transfer Protocol). In essence, this protocol will allow the transmission of delineated blocks of data with a header.
Here is a good way to think about it:
[Header][[Segment][...][...]]
The actual message will appear as:
USDTP[Header][TotalSegments][[SegmentLength][SegmentData]][[...][...]][...]
And the connection headers will be: (ignore the bracketed items)
.USDTP <?> [Request]
.USDTP <-> [Accepted]
.USDTP <X> [Refused]
- "USDTP" (the Message Prefix) and ".USDTP ???" (the Connection Headers) will be encoded in UTF8.
- [Header] will be one byte long.
- [TotalSegments] will be a signed long (8 bytes).
- [SegmentLength] will be a signed long (8 bytes).
- [SegmentData] will be exactly [SegmentLength] bytes in length.
Getting Started
Okay, let's get started by creating the USDTPMessage
class. This class will form the basis of the entire protocol.
using System;
namespace USDTP
{
public class USDTPMessage
{
public byte[][] Payload;
public byte Header;
public USDTPMessage(byte header, byte[][] payload)
{
Payload = payload;
Header = header;
}
}
}
As you can see, this class is very simple minded, but its power lies in the fact that it is so simple.
Header
is the header byte.Payload
is the segments. (See what terrible things arrays can do to grammar.)
Each inner array (i.e. Payload[0], Payload[1], etc.) is an individual segment containing many bytes (i.e. Payload[0][0], Payload[0][1], Payload[1][0], etc...). These segments can, of course, contain different number of bytes.
The Packer
Now a simple question remains: How to get this class, USDTPMessage
, into an array of bytes plus a header (i.e. Header
+ Payload
) suitable for network transfer.
If you recall, earlier we defined the protocol, now we must make the ever important leap from telling intuitive and intelligent humans how to reach this point to telling mindless, heartless, and soulless computers how to do the same.
Philosophy aside, here is how we are to do this:
using System;
using System.Text;
using System.Collections.Generic;
using USDTP;
namespace USDTP.Packing
{
public static class USDTPPacker
{
public static byte[] FullPack(USDTPMessage message)
{
List<byte> PackedData = new List<byte>();
PackedData.AddRange(Encoding.UTF8.GetBytes("USDTP"));
PackedData.Add(message.Header);
PackedData.AddRange(BitConverter.GetBytes(message.Payload.LongLength));
foreach (byte[] ba in message.Payload)
{
PackedData.AddRange(BitConverter.GetBytes(ba.LongLength));
PackedData.AddRange(ba);
}
return PackedData.ToArray();
}
}
}
Just in case that wasn't clear, I'll walk you through it.
We begin with the "required-for-everything" namespace, System
, plus the following namespaces:
System.Text
[Encoding
]System.Collections.Generic
[List<T>
]USDTP
[USDTPMessage
]
Next we define the current namespace USDTP.Packing
, which is exactly the same as putting:
namespace USDTP
{
namespace Packing
{
}
}
Following that, we define the static
class USDTPPacker
, which has only one method: FullPack
which is also static
, returns an array of byte
s, and takes one USDTPMessage
as its argument. The intention of this method is to take the USDTPMessage message
and output an array of byte
s which may be later sent across the network.
We begin by declaring a generic List
of byte
s which will contain the data as it is being packed. Following this, the customary "USDTP" prefix is added to the list in UTF8
format. Next, we add the Header
of the message
and the total number of segments (as a long
), using System.BitConverter
to convert the long
to the equivalent array of byte
s. Then, for each segment, we add its length using the same method as above; and, finally, add the actual data. After this, it is a simple matter of converting PackedData
(a list
) to the required return type (an array of byte
s) using ToArray()
.
The UnPacker
Now that we have a way to pack (encode) the data, we need a way to unpack (decode) the data after it has been received. The problem is, not all data will be received at one time, or may be sent in different TCP packets and arrive at different times. TCP handles the order of the packets for us, but the issue remains that there are noticeable gaps in delivery time. To solve this, we are going to build a non-static
class that will be capable of unpacking the message in chunks, regardless of how many bytes are available at the given time. It should also be able to process multiple messages in the same set of data (i.e. if more than one message is received at a time), or even partial messages.
Because bytes are received from the network in blocks (more on that later), it is necessary to be able to process these blocks on the fly, without having to wait for more data. To do this, we will need a stage counter to remember where we are in the process and be able to cache bytes until we are ready to use them. To do this, the variable lBytesTillNextAction
will contain the number of bytes left till we have enough data stored in a buffer (named TmpBuffer
) to act on it.
The process for receiving a message is the natural result of the order in which the bytes are received:
[First] Prefix Header Segement1 Segemnet2 ... [Last]
Meaning "USDTP" should be the first thing we receive, then the header, and finally the segments.
Here is the full code to do what we have envisioned, plus a little more.
using System;
using System.Text;
using System.Collections.Generic;
using System.Threading;
using USDTP;
namespace USDTP.Packing
{
public class USDTPUnPacker
{
public Queue<USDTPMessage> Messages = new Queue<USDTPMessage>();
public readonly AutoResetEvent reWait = new AutoResetEvent(false);
private List<byte> TmpBuffer = new List<byte>();
private List<byte[]> TmpPayload = new List<byte[]>();
private byte TmpHeader = 0;
private bool bHasMessage = false;
private long lSegsExp = 0;
private long lBytesTillNextAction = 0;
private int iStage = 0;
private Semaphore semLock = new Semaphore(1, 1);
public USDTPUnPacker() { }
private void Reset()
{
bHasMessage = false;
lBytesTillNextAction = 0;
iStage = 0;
TmpBuffer.Clear();
TmpPayload.Clear();
TmpHeader = 0;
lSegsExp = 0;
GC.Collect();
}
private bool ProcessTemp()
{
if (iStage == 1)
{
String proto = Encoding.UTF8.GetString(TmpBuffer.ToArray());
TmpBuffer.Clear();
if (proto != "USDTP")
throw new Exception("Invalid USDTP message header received!");
lBytesTillNextAction = 1;
iStage = 2;
}
else if (iStage == 2)
{
TmpHeader = TmpBuffer[0];
TmpBuffer.Clear();
iStage = 3;
lBytesTillNextAction = 8;
}
else if (iStage == 3)
{
lSegsExp = BitConverter.ToInt64(TmpBuffer.ToArray(), 0);
TmpBuffer.Clear();
if (lSegsExp == 0)
return true;
iStage = 4;
lBytesTillNextAction = 8;
}
else if (iStage == 4)
{
lSegsExp--;
lBytesTillNextAction = BitConverter.ToInt64(TmpBuffer.ToArray(), 0);
TmpBuffer.Clear();
iStage = 5;
}
else if (iStage == 5)
{
TmpPayload.Add(TmpBuffer.ToArray());
TmpBuffer.Clear();
if (lSegsExp == 0)
return true;
lBytesTillNextAction = 8;
iStage = 4;
}
return false;
}
private int SortBytes(byte[] Data)
{
bool hasmore = true;
int msgs = 0;
int offset = 0;
while (hasmore)
{
if (!bHasMessage)
{
bHasMessage = true;
lBytesTillNextAction = 5;
iStage = 1;
}
long maxget = Math.Min(lBytesTillNextAction, Data.Length);
hasmore = (offset + maxget < Data.Length);
byte[] dat = new byte[maxget];
Buffer.BlockCopy(Data, offset, dat, 0, (int)maxget);
TmpBuffer.AddRange(dat);
offset += (int)maxget;
lBytesTillNextAction -= maxget;
try
{
if (lBytesTillNextAction == 0 && ProcessTemp())
{
byte[][] payload = new byte[TmpPayload.Count][];
for (int i = 0; i < TmpPayload.Count; i++)
payload[i] = TmpPayload[i];
Messages.Enqueue(new USDTPMessage(TmpHeader, payload));
Reset();
msgs++;
}
}
catch (Exception ex)
{
Reset();
throw ex;
}
}
return msgs;
}
private void RecieveBytes(byte[] Data, out int iFinished, ref Exception Ex)
{
iFinished = 0;
semLock.WaitOne();
try
{
iFinished = SortBytes(Data);
if (iFinished > 0)
reWait.Set();
}
catch (Exception ex)
{
Ex = ex;
}
finally
{
semLock.Release();
}
}
public void RecieveBytes(byte[] Data, bool Quiet, out int Finished)
{
Exception ex = null;
RecieveBytes(Data, out Finished, ref ex);
if (ex != null && !Quiet) throw ex;
}
public int RecieveBytes(byte[] Data, bool Quiet)
{
int Fin = 0;
RecieveBytes(Data, Quiet, out Fin);
return Fin;
}
public int RecieveBytes(byte[] Data)
{
return RecieveBytes(Data, false);
}
}
}
We begin with the "required-for-everything" namespace, System
, plus the following namespaces:
System.Text
[Encoding
]System.Collections.Generic
[List<T>
, Queue<T>
]System.Threading
[AutoResetEvent
, Semaphore
]USDTP
[USDTPMessage
]
From the previous examples, you should be already clear on class, variable, and method declarations, so we will get right into the meat.
Properties (Worth Explaining)
- [
public
] Messages
(Queue<T>
of byte
s): Holds all messages that have been UnPacked. This is stored on a "first-in, first-out" basis, so it preserves the order in which messages are received. After a message has been UnPacked it is Enqueue()
d into this queue. When the user is ready to consume (retrieve/use) the message, Dequeue()
is called.
For more information on Queue
s, visit MSDN's article on Queue<T>
. - [
public readonly
] reWait
(AutoResetEvent
): This EventWaitHandle
is set when a message has been received (by RecieveBytes()
) and is used later in USDTP.USDTPSocket.WaitMessage()
. It is readonly because we want it to be public
, but never to be changed by the public.
For more information on AutoResetEvent
s and EventWaitHandle
s, visit MSDN's article on AutoResetEvent
. - [
private
] SemLock
(Semaphore
): This Semaphore
controls access to the processing methods. Because data is received asynchronously, it is possible (without the semaphore
) for two instances of the processing functions to be running at one time, which would be disastrous. (I learned this the hard way.)
For more information on Semaphore
s, visit MSDN's article on Semaphore
.
Methods
- [
private
] void
Reset
(
): Resets properties so that another message can be processed. - [
private
] bool
ProcessTemp
(
): This method takes all the bytes stored into TmpBuffer
by SortBytes()
and stores them in their proper location to be later built into the message. It works in a staged manner so that it can process individual blocks on the fly.
- The 5 bytes of the "USDTP" expected prefix have been received. Check to make sure that the bytes were indeed received properly and, if they were not, throw an exception to reset. Otherwise, get ready to receive the 1 byte message header and set the stage to 2.
- The 1 byte message header has been received, so store it in
TmpHeader
and prepare to receive the 8 bytes that constitute the number of expected segments. Set the stage to 3. - The 8 byte signed
long
containing the number of segments to expect has been received. Convert these bytes into the long
and store the new number in lSegsExp
. If more segments are expected, prepare to receive the 8 byte segment length long
and set the stage to 4. Otherwise, return true
because we have completed the message. - The 8 byte signed
long
containing the size of the current segment has been received. Subtract one from the number of segments we still need, even though this segment is still not complete. Set the number of expected bytes until the next action (lBytesTillNextAction
) to the value of the received long
and set the stage to 5. - The remainder of the segment has been received, so store it in
TmpPayload
. If no more segments remain, return true
because we have finished the message. Otherwise, prepare to receive the 8 byte segment length long
and set the stage to 4 because we will still have more segments to process.
- [
private
] int
SortBytes
(byte[]
): This method takes an array of byte
s and processes it one chunk at a time, adding the necessary bytes into TmpBuffer
and controlling when ProcessTmp()
is called. It is capable of handling multiple messages at one time and returns the number of messages it finished unpacking with the provided chunk of data.
This is one of, if not the, most complicated method in the entire project so I will break it down line-by-line below. - [
private
] void
RecieveBytes
(byte[]
, out int
, ref Exception
) {Overload 1}: This method wraps SortBytes()
in shell that performs a number of auxiliary functions. This includes acquiring and, afterwards, releasing the Semaphore
(SemLock
), trapping exceptions, setting the AutoResetEvent
(reWait
) and "out
-ing" the number of completed messages.
Be sure to always acquire a Semaphore
before a try
block and release it in that try
block's finally
clause so that it is always freed, no matter what happens before it is released, be it a success or failure. I cannot stress enough how important this is. - [
public
] {varies} ReceveBytes
({varies}) {Overloads 2-4}: These are public
overloads of RecieveBytes
that are called by USDTP.USDTPSocket
s to inform the UnPacker
bytes have been received by the network. These simply pass control to the above private
overload. Many overloads are used to save coding time.
I hope that wasn't too confusing!
Breakdown of SortBytes
Lines are counted starting from the opening brace of the method, ignoring all other braces, comments, and blank lines.
Line 1: bool hasmore
declared and set to true
. This variable is true
if there is more data remaining in Data
to be processed.
Line 2: int msgs
declared and set to 0
. This variable holds the number of messages this call has completed and is returned at the end.
Line 3: int offset
declared and set to 0
. This variable is the offset into Data
at which coping should begin.
Line 4: Loop while we still have data in Data
to process...
Line 5: If we don't currently have a message being processed...
Line 6: Set bHasMessage
to true
because we now have a message.
Line 7: We now need the "USDTP" prefix for the message, so that means we need five bytes...
Line 8: and the stage should be 1
.
Line 9: long maxget
is set to either the amount of data we need to begin action or all the data we can retrieve at this time, whichever is least.
Line 10: hasmore
is set to true
if data still remains to be processed (offset
+ maxget
is less than the total number of bytes in Data
).
Line 11: byte
array dat
is declared and its size is set to maxget
. This will hold the data we want to copy into TmpBuffer
.
Line 12: Buffer.BlockCopy
is used to copy the desired contents of Data
(offset
through offset
+ maxget
) into dat
.
Line 13: The desired data is added to TmpBuffer
from dat
.
Line 14: offset
is increased by maxget
because the data that maxget
represents has already been read.
Line 15: lBytesTillNextAction
id decreased by maxget
because the maxget
amount of data has been added to TmpBuffer
.
Line 16: try
block is entered to catch errors occurring while processing TmpBuffer
.
Line 17: If we are ready to process TmpBuffer
(lBytesTillNextAction
is 0), call ProcessTmp()
and if that returns true
...
(Note: Due to the way &&
is resolved in C#, ProcessTmp()
will only run if lBytesTillNextAction
is 0
).
Line 18: payload
is declared as a number-of-arrays-of-byte
s-in-TmpPayload
-sized array of arrays of bytes
. (That was a mouthful!)
Lines 19-20: The contents of TmpPayload
are copied into payload
using a for
loop.
Line 21: The USDTPMessage
is created from its parts (TmpHeader
and payload
) and Enqueue()
d into Message
.
Line 22: We call Reset()
because we have completed a message and this instance now needs to be reset to its initial state.
Line 23: We increment the number of messages we have completed (msgs
).
Line 24: An error has been caught...
Line 25: So call Reset
because this instance needs to be reset.
Line 26: And re-throw
the exception, in the hope that it may later be caught.
Line 27: Return the number of messages we unpacked (msgs
).
And that's all there is to it, 27 functioning lines.
The Event Handler
Now that we have the Packer and UnPacker, it would seem it is time to build the actual networking implementation, however because we desire an event-driven architecture we must first design the handler object for these events. This object should have two public delegate
s, one to handle all Socket
events and the other for all Listener
events. In case you aren't sure what exactly those are, it shall be my great pleasure to explain.
Sockets
A Socket
is basically an object on a computer that can connect to Listener
s on the same or different machines. (Technically, this is only true for TCP Socket
s, but that is what we will be using.) For this example, we assume they will be able to both send and receive byte
s.
Message_Recieved
: Occurs when a message has been received from the host of the socket. Connection_Lost
: Occurs when the connection to the host has been lost. Creation_Complete
: Occurs then an asynchronous creation attempt is completed. (More on this later.)
Listeners
A Listener
is a type of Socket
, however it "listens" for incoming connections from other Sockets
and, if it decides to accept them, it creates new Socket
objects to handle the new connections. (Notice the plural forms; Listener
s are able to do this repeatedly.)
Pending_Request_Accepted
: Occurs when a connection has been accepted, processed, and is ready to be used by the user.
With that covered, we can now build the handler:
using System;
namespace USDTP
{
public class USDTPEventHandler
{
public enum USDTPSocketEventType
{
Message_Recieved,
Connection_Lost,
Creation_Complete
}
public enum USDTPListenerEventType
{
Pending_Request_Accepted
}
public delegate void dUSDTPSocketEvent
(USDTPSocket socket, USDTPSocketEventType eventtype, Object EventInfo);
public delegate void dUSDTPListenerEvent
(USDTPListener listener, USDTPListenerEventType eventtype, Object EventInfo);
private event dUSDTPSocketEvent USDTPSocketEvent;
private event dUSDTPListenerEvent USDTPListenerEvent;
private bool bPreformSocketCall = false;
private bool bPreformListenerCall = false;
public USDTPEventHandler() { }
public void RegisterSocketEventHandler(dUSDTPSocketEvent handler)
{
USDTPSocketEvent += handler;
bPreformSocketCall = true;
}
public void RegisterListenerEventHandler(dUSDTPListenerEvent handler)
{
USDTPListenerEvent += handler;
bPreformListenerCall = true;
}
public void UnRegisterSocketEventHandler(dUSDTPSocketEvent handler)
{
USDTPSocketEvent -= handler;
}
public void UnRegisterListenerEventHandler(dUSDTPListenerEvent handler)
{
USDTPListenerEvent -= handler;
}
internal void CallSocketEvent
(USDTPSocket socket, USDTPSocketEventType eventtype, Object EventInfo)
{
if (bPreformSocketCall && USDTPSocketEvent.GetInvocationList().Length > 0)
USDTPSocketEvent(socket, eventtype, EventInfo);
}
internal void CallListenerEvent
(USDTPListener listener, USDTPListenerEventType eventtype, Object EventInfo)
{
if (bPreformListenerCall &&
USDTPListenerEvent.GetInvocationList().Length > 0)
USDTPListenerEvent(listener, eventtype, EventInfo);
}
}
}
This handler essentially allows the registration of delegate
s without worrying if a delegate
is actually registered when a call to the event
is made. It also prevents the use of individually specialized delegate
s being used more than once, which would create a true abomination of code. By consolidating all events into one global handler, it allows maximum code-reuse and supreme conditional runtime flexibility without the complication of raw event registration. Due to the simplistic nature of this class, I'm going to point you toward this CSharpHelp article on event
s and delegate
s.
However, there are several things worth noting. When an event
is called when no delegate
s have been assigned to it, an exception is thrown. This is resolved by setting [bPreform
*Call
] to true
if a delegate
has been registered for that event
. The event
will only ever be called if this bool
is true
. However, we come to the point that, if the last delegate
is un-registered, that an exception will once again be thrown. However the event
is definitely now defined, so we can check if it has any delegate
s in its InvocationList
(the delegate
s it will call when fired) and, if it does, we fire the event
.
The Socket
We have reached the point in our development process where networking is now necessary: all that remains is the Socket and the Listener. The socket implementation is just under 400 lines long, so I'm going to break it down into separate parts.
Namespaces and Class Definition
using System;
using System.Text;
using System.Net;
using System.Net.Sockets;
using System.Threading;
namespace USDTP
{
using Packing;
public class USDTPSocket
{
You should be used to this by now.
System.Text
[Encoding
] System.Net
[IPAddress
, EndPoint
, IPEndPoint
] System.Net.Sockets
[TcpClient
, NetworkSream
] System.Threading
[Thread
, ThreadPriority
, ThreadStart
] USDTP
[USDTPMessage
] - (
USDTP.
)Packing
[USDTPPacker
, USDTPUnPacker
]
USDTPSocket
will be our socket class.
Properties (Standard)
private TcpClient tcConnection;
private NetworkStream nsNetStream;
private USDTPUnPacker upUnPacker;
private Thread tWorker;
public readonly bool CanSend;
public readonly bool CanRecieve;
private USDTPEventHandler ehHandler;
private bool bUseHandler = false;
Nothing too special here, except for tcConnection
and nsNetStream
.
tcConnection
is a TcpClient
that, when this socket is created, will be connected to the remote host. This provides the underlying TCP functions so that we don't have to worry about them. Yet another reason to prove that Microsoft loves us. TcpClients
provide us with a class that literally will save us hours of work, NetworkStream
. These classes definitely deserve a through explanation, which MSDN has already prepared just for us. If you do not already know how TcpClient
s and NetworkStream
s function, I assume you have read this article.
tWorker
is a Thread
that will usually run a method that gets received data from the network. If you are unclear about either the static
or instance members of Thread
, it is critical you visit MSDN's article on Thread
.
Properties (Get-Only)
public bool Connected
{
get
{
return tcConnection.Connected;
}
}
public IPEndPoint RemoteEndPoint
{
get
{
return (IPEndPoint)tcConnection.Client.RemoteEndPoint;
}
}
public int Messages
{
get
{
if (!CanRecieve) return -1;
return upUnPacker.Messages.Count;
}
}
This is, once again, fairly straightforward. The first two properties simply expose important, but otherwise private
, properties of the connection. Messages
allows the retrieval of the number of messages Enqueue()
d in the USDTPUnPacker upUnPacker
. It returns -1
if receiving is not supported.
Methods
- [
public
] USDTPMessage
GetMessage()
:
public USDTPMessage GetMessage()
{
if (!CanRecieve) throw new InvalidOperationException
("receiving is not supported by this instance!");
try
{
return upUnPacker.Messages.Dequeue();
}
catch (InvalidOperationException)
{
return null;
}
}
Immediately retrieves a message from the Queue
in upUnPacker
. This class throws an InvalidOperationException
if receiving is not supported. Note also that Queue
throws an InvalidOperationExeption
if no item exists to be Dequeue()
d. This exception is caught and the method returns null
if this occurs.
- [
public
] USDTPMessage
WaitMessage()
:
public USDTPMessage WaitMessage()
{
if (!CanRecieve) throw new InvalidOperationException
("receiving is not supported by this instance!");
upUnPacker.reWait.WaitOne();
return GetMessage();
}
All this method does is wait for the AutoResetEvent reWait
in the USDTPUnPacker upUnPack
to become set when a message is received. The thread blocks until this occurs. After the event is signalled, it calls GetMessage()
to get the message that has been received.
- [
public
] void
SendMessage(USDTPMessage)
:
public void SendMessage(USDTPMessage message)
{
if (!CanSend) throw new InvalidOperationException
("Sending is not supported by this instance!");
byte[] buffer = USDTPPacker.FullPack(message);
nsNetStream.Write(buffer, 0, buffer.Length);
}
This is fairly straightforward.
Line 1: Throw an InvalidOperationException
if sending is not supported.
Line 2: Fill an array of bytes with the packed message generated with USDTP.Packing.USDTPPacker.FullPack()
.
Line 3: Send the packed data across the network using the NetworkStream nsNetStream
.
- [
public
] void
Disconnect()
:
public void Disconnect()
{
if (!Connected) return;
try
{
nsNetStream.Close(5);
tcConnection.Close();
tcConnection.Client.Close(5);
}
catch {}
}
All this does here is close all the open network interfaces. Be sure they are closed in this order.
- [
private
] void
Manage()
:
private void Manage()
{
while (Connected)
{
try
{
while (tcConnection.Available > 0)
{
byte[] dataRead = new byte[tcConnection.Available];
int iTotalBytes = nsNetStream.Read(dataRead, 0,
tcConnection.Available);
if (iTotalBytes == 0)
{
Thread.Sleep(1);
continue;
}
int finished = upUnPacker.RecieveBytes(dataRead, true);
if (finished > 0 && bUseHandler)
for(int i = 0; i < finished; i++)
ehHandler.CallSocketEvent(this,
USDTPEventHandler.USDTPSocketEventType.Message_Recieved, null);
Thread.Sleep(0);
}
}
catch (ObjectDisposedException)
{
break;
}
catch { }
finally
{
Thread.Sleep(10);
}
}
if (bUseHandler)
ehHandler.CallSocketEvent(this,
USDTPEventHandler.USDTPSocketEventType.Connection_Lost, null);
}
This method is run constantly on Thread tWorker
and reads data in from the network and passes it on to the USDTPUnPacker upUnPacker
. I believe the comments in the above code are sufficient to explain this method.
- [
internal
] {constructor} USDTPSocket(TcpClient,bool,bool)
:
internal USDTPSocket(TcpClient conn, bool send, bool recieve)
{
tcConnection = conn;
nsNetStream = tcConnection.GetStream();
CanSend = send;
CanRecieve = recieve;
if (CanRecieve)
{
upUnPacker = new USDTPUnPacker();
tWorker = new Thread(Manage);
tWorker.Priority = ThreadPriority.Normal;
tWorker.Start();
}
}
This is the only constructor for this method. It is simple, but allows this class to be used both on the server and the client. The server internally creates one of these for each connection and provides it for later use. The client uses the static Create
*()
methods to create an instance and connect to the server. It also starts the Manage()
method running on tWorker
if it actually needs to be run (i.e. receiving is supported).
- [
public
] void
RegisterHandler(USDTPEventHandler)
:
public void RegisterHandler(USDTPEventHandler handler)
{
if (bUseHandler) throw new InvalidOperationException
("An USDTPEventHandler has already been registered!");
ehHandler = handler;
bUseHandler = true;
}
- [
public
] void
UnRegisterHandler()
:
public void UnRegisterHandler()
{
bUseHandler = false;
}
Enums
- [
public
] CreationResult
(byte
):
public enum CreationResult : byte
{
SendRecieve,
Refused,
Invalid
}
- [
public
] CreationType
(byte
):
public enum CreationType : byte
{
SendRecieve
}
Internal Classes
Static Methods - "Constructors"
- [
public
static
] USDTPSocket
Create(IPEndPoint, CreationType,
out CreationResult)
:
public static USDTPSocket Create(IPEndPoint Host, CreationType Use,
out CreationResult Result)
{
TcpClient tcCli = new TcpClient();
tcCli.Connect(Host);
NetworkStream nsNetStream = tcCli.GetStream();
nsNetStream.Write(Encoding.UTF8.GetBytes(".USDTP "), 0, 7);
if (Use == CreationType.SendRecieve)
nsNetStream.Write(Encoding.UTF8.GetBytes("<?>"), 0, 3);
int iCntr = 0;
byte[] dataHeader = new byte[10];
while (iCntr < 10)
{
int iVal = nsNetStream.ReadByte();
if (iVal == -1)
{
Thread.Sleep(5);
continue;
}
dataHeader[iCntr] = (byte)iVal;
iCntr++;
}
String sResponse = Encoding.UTF8.GetString(dataHeader);
if (sResponse == ".USDTP <->")
{
Result = CreationResult.SendRecieve;
return new USDTPSocket(tcCli, true, true);
}
else if (sResponse == ".USDTP <X>")
{
Result = CreationResult.Refused;
try
{
nsNetStream.Close();
tcCli.Close();
tcCli.Client.Close();
}
catch { }
return null;
}
else
{
Result = CreationResult.Invalid;
try
{
nsNetStream.Close();
tcCli.Close();
tcCli.Client.Close();
}
catch { }
return null;
}
}
The only portion we need to cover is the initial reading of the 10-byte response.
int iCntr = 0;
byte[] dataHeader = new byte[10];
while (iCntr < 10)
{
int iVal = nsNetStream.ReadByte();
if (iVal == -1)
{
Thread.Sleep(5);
continue;
}
dataHeader[iCntr] = (byte)iVal;
iCntr++;
}
All this does is to read 10 bytes one-at-a-time from the network using NetworkStream nsNetStream.ReadByte()
. This method attempts to read one byte from the network and, if a byte is available, returns it as an int
, otherwise, it returns -1
. It is simple enough to check if the byte read is -1
or not, so we simply do that for each attempt. If it is -1
, we Sleep()
the thread so we don't consume a large amount of CPU time waiting for more data.
- [
private
static
] void
WorkAsyncCreate(Object)
:
private static void WorkCreateAsync(Object Arg)
{
Object[] Args =(Object[])Arg;
IPEndPoint epHost = (IPEndPoint)Args[0];
CreationType ctUse = (CreationType)Args[1];
USDTPEventHandler.dUSDTPSocketEvent seCallback =
(USDTPEventHandler.dUSDTPSocketEvent)Args[2];
CreationResult Result = CreationResult.Invalid;
USDTPSocket usSock = Create(epHost, ctUse, out Result);
AsyncCreationInfo acInf = new AsyncCreationInfo();
acInf.Result = Result;
acInf.usSocket = usSock;
seCallback(null,
USDTPEventHandler.USDTPSocketEventType.Creation_Complete, acInf);
}
This method is run on a separate thread and simply enables the asynchronous creation of an USDTPSocket
. When creation is complete, regardless of success or failure, the provided delegate
(a USDTPEventHandler.dUSDTPSocketEvent
) is called.
- [
public
static
] USDTPSocket
CreateAsync(IPEndPoint, CreationType, USDTPEventHandler.dUSDTPSocketEvent)
:
public static void CreateAsync(IPEndPoint Host, CreationType Use,
USDTPEventHandler.dUSDTPSocketEvent Callback)
{
Thread tCreator = new Thread(WorkCreateAsync);
tCreator.Priority = ThreadPriority.AboveNormal;
tCreator.Start(new Object[] { Host, Use, Callback });
}
A call to this method begins the asynchronous creation of a USDTPSocket
(see above).
And, for good measure, we close the trailing parenthesis.
}
}
The Listener
As we reach the end of this tutorial, only one task remains, the creation of the USDTPListener
class. This class will server as the Listener implementation. It will be able to accept connections on multiple IP addresses and ports. It should also verify that a client is USDTP-valid before accepting it. Additionally, stop, pause, and start functionality should be available. Once again, because this file is large, it will be separated into parts.
Namespaces and Class Definition
using System;
using System.Text;
using System.Collections.Generic;
using System.Net;
using System.Net.Sockets;
using System.Threading;
namespace USDTP
{
public class USDTPListener
{
System.Text
[Encoding
] System.Collections.Generic
[Queue<T>
, List<T>
] System.Net
[IPAddress
, IPEndPoint
] System.Net.Sockets
[TcpClient
, TcpListener
, NetworkStream
] System.Threading
[Thread
, ThreadPriority
, ThreadStart
] - (
USDTP
) [USDTPSocket
]
Properties (Standard)
private List<TcpListener> ltlListeners = new List<TcpListener>();
private bool bRun = false;
private bool bAlive = true;
private bool bHalt = true;
private Thread tWorker;
private USDTPEventHandler ehHandler;
private bool bUseHandler = false;
public Queue<USDTPSocket> PendingRequests = new Queue<USDTPSocket>();
The only properties worth noting here are ltlListeners
and PendingRequests
.
ltlListners
is a List<T>
of TcpListener
s. This property contains all the active underlying listeners in use by the object.
PendingRequests
is a Queue<T>
of USDTPSocket
s. This property contains all the accepted and validated USDTPSockets
that are ready to be used by the user (by means of Dequeue()
).
Properties (Get-Only)
public bool isRunning
{
get
{
return bRun;
}
}
public bool isAlive
{
get
{
return bAlive;
}
}
Management Methods
These methods help to accept and process incoming connections.
- [
private
] void
HandleClient(TcpClient)
:
private void HandleClient(TcpClient tcCli)
{
NetworkStream nsNetStream = tcCli.GetStream();
int iCntr = 0;
byte[] dataHeader = new byte[10];
while (iCntr < 10)
{
int iVal = nsNetStream.ReadByte();
if (iVal == -1)
{
Thread.Sleep(5);
continue;
}
dataHeader[iCntr] = (byte)iVal;
iCntr++;
}
String sResponse = Encoding.UTF8.GetString(dataHeader);
if (sResponse == ".USDTP <?>")
{
nsNetStream.Write(Encoding.UTF8.GetBytes(".USDTP <->"), 0, 10);
PendingRequests.Enqueue(new USDTPSocket(tcCli, true, true));
if (bUseHandler)
ehHandler.CallListenerEvent(this, USDTPEventHandler.
USDTPListenerEventType.Pending_Request_Accepted, null);
}
else
{
nsNetStream.Write(Encoding.UTF8.GetBytes(".USDTP <X>"), 0, 10);
nsNetStream.Close();
tcCli.Close();
tcCli.Client.Close();
}
}
This method is called when a new TcpClient
has been accepted by Manage()
and is ready to be processed. It takes the client and reads the expected 10-byte header from the network. If this header is valid, it sends the acceptance message (defined at the beginning of this tutorial), Enqueue()
s the new USDTPClient
into PendingRequesets
, and fires the proper event on ehHandler
. Otherwise it sends the refusal message and disconnects from the client. The workings of this method should, by now, be quite clear.
- [
private
] void
Manage()
:
private void Manage()
{
while (bAlive)
{
while (bRun)
{
foreach (TcpListener tlList in ltlListeners)
{
try
{
while (tlList.Pending())
{
TcpClient tcCli = tlList.AcceptTcpClient();
HandleClient(tcCli);
}
}
catch { }
Thread.Sleep(0);
}
Thread.Sleep(1);
}
Thread.Sleep(25);
}
}
This method is run on tWorker
. It constantly checks each and every registered listener (in ltlListners
) to see if it has new clients. If it does, if passes the new TcpClient
to HandleClient()
. It has two inner loops to compensate for the stop/pause/start functions.
Constructors
- [
public
] {constructor} USDTPListner()
:
public USDTPListener()
{
tWorker = new Thread(Manage);
tWorker.Priority = ThreadPriority.AboveNormal;
tWorker.Start();
}
All this does is to start Manage()
on tWorker
.
- {deconstructor}
~USDTPListner()
:
~USDTPListener()
{
Stop();
}
State Methods
- [
public
] void
Stop()
:
public void Stop()
{
if (!bAlive) return;
try
{
bRun = false;
bAlive = false;
Thread.Sleep(0);
if (!tWorker.Join(100))
{
tWorker.Abort();
tWorker.Join();
}
foreach (TcpListener tlList in ltlListeners)
{
try
{
tlList.Stop();
}
catch { }
}
ltlListeners.Clear();
}
catch
{
}
}
This method is called when the object is to stop all activity.
- [
public
] void
Pause(bool)
:
public void Pause(bool HaltAttempts)
{
bRun = false;
bHalt = HaltAttempts;
if (bHalt)
{
foreach (TcpListener tlList in ltlListeners) tlList.Stop();
}
}
This method is called to temporarily pause all processing activity. If HaltAttempts
is true
, all listeners are "paused" as well, so incoming requests will be denied.
- [
public
] void
Start()
:
public void Start()
{
bRun = true;
if (bHalt)
{
foreach (TcpListener l in ltlListeners) l.Start();
bHalt = false;
}
}
This method is called to start or resume activity. Note how it handles if bHalt
is true
.
Listener Methods
- [
public
] void
RegisterListener(IPAddress, int)
:
public void RegisterListener(IPAddress localaddr, int port)
{
TcpListener tlList = new TcpListener(localaddr, port);
if (!bHalt) tlList.Start();
ltlListeners.Add(tlList);
}
This adds a listener to listen for connection attempts and starts it if necessary.
Event Handler Methods
You should be familiar with these from USDTPSocket
.
- [
public
] void
RegisterHandler(USDTPEventHandler)
:
public void RegisterHandler(USDTPEventHandler handler)
{
if (bUseHandler) throw new InvalidOperationException
("An USDTPEventHandler has already been registered!");
ehHandler = handler;
bUseHandler = true;
}
- [
public
] void
UnRegisterHandler()
:
public void UnRegisterHandler()
{
bUseHandler = false;
}
Using USDTP
We have reached the end of the tutorial section of this article. If you have read entirely through this article, use of the code we have created should be pretty clear. However, if you have not or it is not, here is a short and dirty demo.
using System;
using System.Net;
using USDTP;
namespace USDTP_Tester
{
class Program
{
private static void HandleListener
(USDTPListener listener, USDTPEventHandler.USDTPListenerEventType eventtype,
Object EventInfo)
{
Console.WriteLine("Listener Event: " + eventtype.ToString());
}
private static void HandleSocket
(USDTPSocket socket, USDTPEventHandler.USDTPSocketEventType eventtype,
Object EventInfo)
{
Console.WriteLine("Socket Event: " + eventtype.ToString());
}
static void Main(string[] args)
{
USDTPEventHandler Handler = new USDTPEventHandler();
Handler.RegisterListenerEventHandler(HandleListener);
Handler.RegisterSocketEventHandler(HandleSocket);
USDTPListener Listener = new USDTPListener();
Listener.RegisterHandler(Handler);
Listener.RegisterListener(IPAddress.Any, 367);
Listener.Start();
USDTPSocket ssSock = null;
USDTPSocket.CreationResult res;
USDTPSocket csSock = USDTPSocket.Create(new IPEndPoint
(IPAddress.Loopback, 367),
USDTPSocket.CreationType.SendRecieve, out res);
while (ssSock == null)
{
Console.WriteLine("Press Any Key To Retrieve Server-Side Client!");
Console.ReadKey(true);
try
{
ssSock = Listener.PendingRequests.Dequeue();
}
catch
{
Console.WriteLine(" >> Error Occured!");
}
}
ssSock.RegisterHandler(Handler);
USDTPMessage msgout = new USDTPMessage(0x00, new byte[][]
{ new byte[] { 0x01, 0x02, 0x03, 0x04 }, new byte[]
{ 0x05, 0x06, 0x07 }, new byte[] { 0x08, 0x09 } });
csSock.SendMessage(msgout);
USDTPMessage msgin = ssSock.WaitMessage();
Console.WriteLine("Message received!");
Console.WriteLine(" >> Header: " + msgin.Header);
foreach (byte[] bp in msgin.Payload)
{
Console.WriteLine(" >> Payload Segment:");
foreach (byte b in bp)
Console.WriteLine(" >> " + b);
}
Console.WriteLine("Press Any Key To Shutdown!");
Console.ReadKey();
csSock.Disconnect();
ssSock.Disconnect();
Listener.Stop();
Console.WriteLine("Press Any Key To Exit!");
Console.ReadKey(true);
}
}
}
This should produce output similar to the following...
Listener Event: Pending_Request_Accepted
Press Any Key To Retrieve Server-Side Client!
<Key Press>
Socket Event: Message_Recieved
Message received!
>> Header: 0
>> Payload Segment:
>> 1
>> 2
>> 3
>> 4
>> Payload Segment:
>> 5
>> 6
>> 7
>> Payload Segment:
>> 8
>> 9
Press Any Key To Shutdown!
<Key Press>
Socket Event: Connection_Lost
Press Any Key To Exit!
<Key Press>
<Console Closes>
I hope that answers all questions.
Conclusion
In this tutorial, we created a protocol called USDTP and an implementation of it in C#. This included a Socket
and Listener
class, as well as a Packer, Unpacker, and Event Handler.
Feel free to drop me a line with any questions.
History
- 13th June, 2008: First version
Nathan Sharp is a student who has been programming for eight years now. He began programming as a hobby with simple HTML and quickly learned JavaScript. Nearly half a year later he began dabbling in C++ and eventually discovered Visual Studio and Managed C++. Shortly afterwards he begin C# and was instantly hooked - his time since then (nearly six years) is now spent with this language. Nathan regularly builds both desktop and web applications (in C#) and considers himself to be quite adept. Along with C#, Nathan knows around 14 other programming lanaguages, including PHP, Lua, SQL, and Assembly.