Click here to Skip to main content
Click here to Skip to main content

An Asynchronous Socket Server and Client

By , 29 Apr 2009
 
Prize winner in Competition "C# Apr 2006"
Article screenshot

Introduction

I've been working with sockets since 2000, using Delphi 5.0 and some third-party libraries (Synapse). My very first socket application just copied files between many clients and one server. The client app checks a folder to see if files exist, asks the server where to copy the files in the network and, after copying the files, flags the database record indicating that a file has been moved. The server listens to the client connections, and both exchange XML messages indicating the state of each file copy. Synapse is a blocking socket implementation, and I needed a thread pooling mechanism that works like an HTTP server, because I couldn't keep the connection open (one thread per connection). My solution was to use some IOCP functions to pool the client requests (code) and close the connection after the message exchange was terminated.

Now, using C#, I decided to write a socket server and client library that helps me to only have to think about the message exchange (the process) and let .NET do the hard job. So, I needed the following features:

  • Asynchronous processing
  • Some encryption and compression capabilities
  • Encapsulate the socket, and encrypt the services in the interfaces and separate them from the host implementation

Socket Connection

The ISocketConnection is the base interface for the socket connections, and describes all the connection properties and the methods. The ConnectionID property defines a unique connection ID using a GUID string. The CustomData property defines a custom object that can be associated with the connection. The Header property is the socket service header used in each message that is encapsulated in a packet message. Only messages with a defined header will be accepted. The LocalEndPoint and RemoteEndPoint are the socket IP end points used in the connection. SocketHandle is the socket handle given by the OS.

The IClientSocketConnection and IServerSocketConnection inherit the ISocketConnection, and each one has special functions. The IClientSocketConnection can reconnect to the server using the BeginReconnect method, and the IServerSocketConnection can communicate with other connections in the server host using the BeginSendTo and BeginSendToAll methods, and can get the ConnectionId using the GetConnectionById method. Every connection knows the host, the encryption, the compression type, and can send, receive, and disconnect itself from the other part. This interface is used in the ISocketService interface to allow the user to interact with socket connections.

Internally, in the library implementation, all the connection interfaces are created using the base connection implementations: BaseSocketConnection, ClientSocketConnection, and ServerSocketConnection.

Socket Service

The ISocketService describes the connection events. These events are fired by the host, and have a ConnectionEventArgs argument which has an ISocketConnection that identifies the connection. In the OnReceived and OnSent events, a MessageEventArgs is passed, which has the sent or received array of bytes. In the OnDisconnected event, a DisconnectedEventArgs is passed; the Exception property indicates if the disconnection has been caused by an exception.

Here is an example of a ISocketService implementation:

public class SimpleEchoService : ISocketService
{
    public void OnConnected(ConnectionEventArgs e)
    {
        //----- Check the host!
        if (e.Connection.HostType == HostType.htServer)
        {
            //----- Enqueue receive!
            e.Connection.BeginReceive();
        }
        else
        {
            //----- Enqueue send a custom message!
            byte[] b =
              GetMessage(e.Connection.SocketHandle.ToInt32());
            e.Connection.BeginSend(b);
        }
    }

    public void OnSent(MessageEventArgs e)
    {
        //----- Check the host. In this case both start a receive!
        if (e.Connection.HostType == HostType.htServer)
        {
            //----- Enqueue receive!
            e.Connection.BeginReceive();
        }
        else
        {
            //----- Enqueue receive!
            e.Connection.BeginReceive();
        }
    }

    public override void OnReceived(MessageEventArgs e)
    {
        //----- Check the host!
        if (e.Connection.HostType == HostType.htServer)
        {
            //----- If server, send the data buffer received!
            byte[] b = e.Buffer;
            e.Connection.BeginSend(b);
        }
        else
        {
            //----- If client, generate another
            //----- custom message and send it!
            byte[] b = GetMessage(e.Connection.SocketHandle.ToInt32());
            e.Connection.BeginSend(b);
        }
    }

    public override void OnDisconnected(DisconnectedEventArgs e)
    {
        //----- Check the host!
        if (e.Connection.HostType == HostType.htServer)
        {
            //----- Nothing!
        }
        else
        {
            //----- Reconnect with server!
            e.Connection.AsClientConnection().BeginReconnect();
        }
    }
}

The ISocketService implementation can be done in the same host assembly, or another assembly referenced by the host. This allows the user to separate the host implementation from the socket service, helping the administration in a server or a domain.

Connection Host

With the ISocketService created, you need to host the service and the service connections. Both the server and the client host have the same parent class, BaseSocketConnectionHost, which keeps a list of connections, encrypts and compresses the data buffers, enqueues the service requests and ensures that all data buffer has been sent or received, checks messages headers, and checks for idle connections. The CheckTimeoutTimer, periodically, at IdleCheckInterval, checks if the connections become idle, using the IdleTimeOutValue as the idle timeout. Header is the socket service header used by the host. HostType indicates if a host is a server or a client host. SocketBufferSize defines the size of the socket send and receive buffer. SocketService is the instance of ISocketService that drives the message exchange between the connections.

Encrypt and Compress

Every time you send and receive messages, the host checks if the data must be encrypted and/or compressed, and this work is made by the CryptUtils static class. The CreateSymmetricAlgoritm creates an ISymmetricAlgoritm based on the encryptType parameter. The DecryptData and DecryptDataForAuthenticate are used, respectively, to decrypt the received message and check the hash sign on the authenticate procedure. The EncryptData and EncryptDataForAuthenticate, respectively, encrypt the data to be sent and sign the authenticated message.

The encrypted data buffer is labelled with the service header and the data buffer length, becoming a packet buffer. This packet buffer is controlled by the MessageBuffer class that keeps information about the packet buffer offset, length, the remaining bytes, and the raw buffer.

Enqueuing requests

Every time you call BeginReceive or BeginSend in ISocketService, the host checks if some request has been initiated. If a request is in process, the host enqueues the request. If not, it fires the request.

Send request

In the BeginSend method, the following enqueuing is used:

internal void BeginSend(BaseSocketConnection connection, byte[] buffer)
{
...
    //----- Check Queue!
    lock (connection.WriteQueue)
    {

        if (connection.WriteQueueHasItems)
        {
            //----- If the connection is sending, enqueue the message!
            connection.WriteQueue.Enqueue(writeMessage);
        }
        else
        {

            //----- If the connection is not sending, send the message!
            connection.WriteQueueHasItems = true;

...

When the message is sent, in the send callback, the host checks the queue again and initiates another send process, if needed:

private void BeginSendCallback(IAsyncResult ar)
{
    ...
    //----- Check Queue!
    lock (connection.WriteQueue)
    {

        if (connection.WriteQueue.Count > 0)
        {

            //----- If has items, send it!
            MessageBuffer dequeueWriteMessage =
                          connection.WriteQueue.Dequeue();
            ...

        }
        else
        {
            connection.WriteQueueHasItems = false;
        }

    }
...

Receive request

The same technique applies to the receive method: all the calls to BeginReceive are enqueued if the receive method is in action. If no receive process was initiated, the host starts to receive:

internal void BeginReceive(BaseSocketConnection connection)
{
    ...
    //----- Check Queue!
    lock (connection.SyncReadCount)
    {

        if (connection.ReadCanEnqueue)
        {

            if (connection.ReadCount == 0)
            {

                //----- if the connection is not receiving, start the receive!
                MessageBuffer readMessage = new MessageBuffer
                            (FSocketBufferSize);

                ...

            }

            //----- Increase the read count!
            connection.ReadCount++;

        }

    }
    ...

After that, when the message is received and parsed in the receive callback, the host checks the read queue again, and initiates another receive process, if needed:

private void BeginReadCallback(IAsyncResult ar)
{
    ...
    //----- Check Queue!
    lock (connection.SyncReadCount)
    {

        connection.ReadCount--;

        if (connection.ReadCount > 0)
        {

            //----- if the read queue has items, start to receive!
            ...

        }

    }
    ...

Ensure send and receive

To ensure that all data buffer is sent, the BaseSocketConnectionHost checks the bytes sent, and compares it to the MessageBuffer class. It continues to send the remaining bytes till all the data buffer is sent:

private void BeginSendCallback(IAsyncResult ar)
{
...
    byte[] sent = null;
    int writeBytes = .EndSend(ar);

    if (writeBytes < writeMessage.PacketBuffer.Length)
    {
        //----- Continue to send until all bytes are sent!
        writeMessage.PacketOffSet += writeBytes;
        .BeginSend(writeMessage.PacketBuffer, writeMessage.PacketOffSet,
                   writeMessage.PacketRemaining, SocketFlags.None ...);
    }
    else
    {
        sent = new byte[writeMessage.RawBuffer.Length];
        Array.Copy(writeMessage.RawBuffer, 0, sent, 0, 
                    writeMessage.RawBuffer.Length);
        FireOnSent(connection, sent);
    }
}

The same approach is used in the receive data buffers because, to read data, a MessageBuffer is used as the read buffer. When the receive callback is called, it continues to read till all the bytes in the message are read:

private void BeginReadCallback(IAsyncResult ar)
{
    ...
    CallbackData callbackData = (CallbackData)ar.AsyncState;

    connection = callbackData.Connection;
    readMessage = callbackData.Buffer;

    int readBytes = 0;
    ...
    readBytes = .EndReceive(ar);
    ...

    if (readBytes > 0)
    {
        ...
        //----- Has bytes!
        ...
        //----- Process received data!
        readMessage.PacketOffSet += readBytes;
        ...

        if (readSocket)
        {


            //----- Read More!
            .BeginReceive(readMessage.PacketBuffer,
                          readMessage.PacketOffSet,
                          readMessage.PacketRemaining,
                          SocketFlags.None, ...);
        }
    }
    ...

Check message header

If the socket service uses some header, all the send and receive processes need to create a packet message indicating the header and the message length. This packet label is created using the following structure:

The first label's part is the socket service header. The header is an array of bytes of any length, and you need some advice here: if you choose a very small header, maybe you can have a message with the same array of bytes somewhere, and the host will lose the sequence. If you choose a very long array of bytes, the host can spend the processor's time to verify if the message header is equal to the socket service. The second part is the packet message length. This length is calculated adding the raw message data buffer length, encrypted and/or compressed, plus the header length.

Sending packets

As said before, every time you send messages, the host checks if the data must be encrypted and/or compressed, and, if you choose to use some header, the raw buffer is controlled by the MessageBuffer class. This class is created using the GetPacketMessage static method:

public static MessageBuffer GetPacketMessage(
       BaseSocketConnection connection, ref byte[] buffer)
{

    byte[] workBuffer = null;

    workBuffer = CryptUtils.EncryptData(connection, buffer);

    if (connection.Header != null && connection.Header.Length >= 0)
    {
        //----- Need header!
        int headerSize = connection.Header.Length + 2;
        byte[] result = new byte[workBuffer.Length + headerSize];

        int messageLength = result.Length;

        //----- Header!
        for (int i = 0; i < connection.Header.Length; i++)
        {
            result[i] = connection.Header[i];
        }

        //----- Length!
        result[connection.Header.Length] =
           Convert.ToByte((messageLength & 0xFF00) >> 8);
        result[connection.Header.Length + 1] =
           Convert.ToByte(messageLength & 0xFF);

        Array.Copy(workBuffer, 0, result,
                   headerSize, workBuffer.Length);

        return new MessageBuffer(ref buffer, ref result);

    }
    else
    {
        //----- No header!
        return new MessageBuffer(ref buffer, ref workBuffer);
    }
}

Receiving packets

The receive process, if you're using some socket service header, needs to check the header, and continues to read bytes till all the packet message is received. This process is executed in the read callback:

private void BeginReadCallback(IAsyncResult ar)
{
...

    byte[] received = null
    byte[] rawBuffer = null;
    byte[] connectionHeader = connection.Header;

    readMessage.PacketOffSet += readBytes;

    if ((connectionHeader != null) && (connectionHeader.Length > 0))
    {

        //----- Message with header!
        int headerSize = connectionHeader.Length + 2;

        bool readPacket = false;
        bool readSocket = false;

        do
        {
            connection.LastAction = DateTime.Now;

            if (readMessage.PacketOffSet > headerSize)
            {
                //----- Has Header!
                for (int i = 0; i < connectionHeader.Length; i++)
                {
                    if (connectionHeader[i] != readMessage.PacketBuffer[i])
                    {
                        //----- Bad Header!
                        throw new BadHeaderException(
                          "Message header is different from Host header.");
                    }
                }

                //----- Get Length!
                int messageLength =
                  (readMessage.PacketBuffer[connectionHeader.Length] << 8) +
                  readMessage.PacketBuffer[connectionHeader.Length + 1];

                if (messageLength > FMessageBufferSize)
                {
                    throw new MessageLengthException("Message " +
                      "length is greater than Host maximum message length.");
                }

                //----- Check Length!
                if (messageLength == readMessage.PacketOffSet)
                {
                    //----- Equal -> Get rawBuffer!
                    rawBuffer =
                      readMessage.GetRawBuffer(messageLength, headerSize);

                    readPacket = false;
                    readSocket = false;
                }
                else
                {
                    if (messageLength < readMessage.PacketOffSet)
                    {
                        //----- Less -> Get rawBuffer and fire event!
                        rawBuffer =
                          readMessage.GetRawBuffer(messageLength, headerSize);

                        //----- Decrypt!
                        rawBuffer = CryptUtils.DecryptData(connection,
                                    ref rawBuffer, FMessageBufferSize);

                        readPacket = true;
                        readSocket = false;

                        received = new byte[rawBuffer.Length];
                        Array.Copy(rawBuffer, 0, received, 0, 
                            rawBuffer.Length);
                        FireOnReceived(connection, received, false);
                    }
                    else
                    {
                        if (messageLength > readMessage.PacketOffSet)
                        {
                            //----- Greater -> Read Socket!
                            if (messageLength > readMessage.PacketLength)
                            {
                                readMessage.Resize(messageLength);
                            }

                            readPacket = false;
                            readSocket = true;
                        }
                    }
                }
            }
            else
            {
                if (readMessage.PacketRemaining < headerSize)
                {
                    //----- Adjust room for more!
                    readMessage.Resize(readMessage.PacketLength + headerSize);
                }

                readPacket = false;
                readSocket = true;
            }

        } while (readPacket);

        if (readSocket)
        {
            //----- Read More!
            ...
            .BeginReceive(readMessage.PacketBuffer, readMessage.PacketOffSet,
                          readMessage.PacketRemaining, SocketFlags.None, ...);
            ...
        }
    }
    else
    {
        //----- Message with no header!
        rawBuffer = readMessage.GetRawBuffer(readBytes, 0);
    }

    if (rawBuffer != null)
    {
        //----- Decrypt!
        rawBuffer = CryptUtils.DecryptData(connection,
                    ref rawBuffer, FMessageBufferSize);

        received = new byte[rawBuffer.Length];
        Array.Copy(rawBuffer, 0, received, 0, rawBuffer.Length);
        FireOnReceived(connection, received, true);

        readMessage.Resize(FSocketBufferSize);
...

The read callback method first checks if the connection has some header and, if not, just gets the raw buffer and continues. If the connection has some header, the method needs to check the message header against the socket service header. Before doing that, it checks if the packet message length is greater than the connection header length, to ensure that it can parse the total message length. If not, it reads some bytes. After checking the header, the method parses the message length, and checks with the packet length. If the length is equal, it gets the raw buffer and terminates the loop. If the message length is less than that of the packet message, we have the message plus some data. So, the method gets the raw buffer and continues to read using the same MessageBuffer class. If the length of the message is greater than that of the packet message, before reading some data, it just resizes the packet buffer to the message size, ensuring enough room for more read bytes.

Checking idle connections

Using the BeginSend and BeginReceive methods of ISocketConnection doesn't return some IAsyncResult to know if the method was completed or not allowing disconnection after some timeout value. To prevent this, the BaseSocketConnectionHost has a System.Threading.Timer that periodically checks the LastAction property of BaseSocketConnection. If LastAction is greater than the idle timeout, the connection is closed.

Crypto Service

The ICryptoService describes the authentication methods fired when the connection is made to the other part. The OnSymmetricAuthenticate method is fired when EncryptType.etRijndael or EncryptType.etTripleDES is used, and OnSSLXXXXAuthentication is fired when EncryptType.etSSL is used. Like ISocketService, the ICryptService can be done in the same host assembly, or another assembly referenced by the host, so you can have one ICryptoService implementation used in many ISocketService implementations.

SSL authentication

There's a new stream class called SslStream in .NET 2.0 which can authenticate SSL streams. The SslStream's constructor accepts a NetworkStream class, and this stream is created using the Socket class. So, using SslStream, you can send and receive data buffers using socket connections.

Server authentication

The SslStream authentication is done in both the client and the server, but each one has different parameters. In the server side, you need to pass a certificate using the X509Certificate2 class, either finding in the certificate store using X509Store, or by creating it from a certification file (.cer). Also, you can request a client authentication and check the certificate's revocation. The following code is an example of an SSL server authentication using ICryptService:

public void OnSSLServerAuthenticate(out X509Certificate2 certificate,
               out bool clientAuthenticate, ref bool checkRevocation)
{
    //----- Set server certificate, client
    //----- authentication and certificate revocation!
    X509Store store = new X509Store(StoreName.My,
                      StoreLocation.LocalMachine);
    store.Open(OpenFlags.ReadOnly);

    X509Certificate2Collection certs =
      store.Certificates.Find(X509FindType.FindBySubjectName,
      "ALAZ Library", false);
    certificate = certs[0];

    clientAuthenticate = false;
    checkRevocation = false;

    store.Close();
}

Client authentication

On the client side of the SSL authentication, you need to pass the host name of the server certificate, and if this name doesn't match, the authentication fails. You can pass a client certificate collection using X509Certificate2Collection. If the server doesn't request a client authentication, you don't need to pass the collection but, if the server requests it, you can find the certificates using X509Store. You can also request a client certificate's revocation. This is an example of SSL client authentication in ICryptoService:

public void OnSSLClientAuthenticate(out string serverName,
            ref X509Certificate2Collection certs, ref bool checkRevocation)
{
    serverName = "ALAZ Library";
    /*
    //----- Using client certificate!
    X509Store store = new X509Store(StoreName.My,
                      StoreLocation.LocalMachine);
    store.Open(OpenFlags.ReadOnly);

    certs = store.Certificates.Find(
            X509FindType.FindBySubjectName,
            serverName, true);
    checkRevocation = false;

    store.Close();
    */
}

Certificates

To create certificates, you can use the MakeCert.exe tool found in .NET, and there's a lot of information available about it. You can take a look at John Howard's page, this MS post, and this website.

Symmetric authentication

To implement some symmetric encryption and authentication in this library, I decided to put a post in Microsoft newsgroups. Unfortunately, for the post, but luckily for the knowledge sharing (many thanks to Joe Kaplan, Dominick Baier, and Valery Pryamikov), I decided to use William Stacey's implementation example "A generic method to send secure messages using an exchanged session key". In this code, the symmetric key used in the session is encrypted and signed using RSA key pairs, and the client part needs to know the encrypted server's public key, meaning that this key isn't received from the server in the authentication process. Both the client and the server need to know this key through a manual process. To ensure this, the OnSymmetricAuthenticate needs a RSACryptoServiceProvider class providing the key pair for encryption. You can fill the RSACryptoServiceProvider from an XML string, a file, a CspParameters class, or a certificate. Here is an example of symmetric authentication:

public void OnSymmetricAuthenticate(HostType hostType,
            out RSACryptoServiceProvider serverKey)
{
    /*
       * A RSACryptoServiceProvider is needed to encrypt and send session key.
       * In server side you need public and private key to decrypt session key.
       * In client side you need only public key to encrypt session key.
       *
       * You can create a RSACryptoServiceProvider from a string
       * (file, registry), a CspParameters or a certificate.
    */

    //----- Using string!
    /*
    serverKey = new RSACryptoServiceProvider();
    serverKey.FromXMLString("XML key string");
    */

    //----- Using CspParameters!
    CspParameters param = new CspParameters();
    param.KeyContainerName = "ALAZ_ECHO_SERVICE";
    serverKey = new RSACryptoServiceProvider(param);

    /*
    //----- Using Certificate Store!
    X509Store store = new X509Store(StoreName.My,
                      StoreLocation.LocalMachine);
    store.Open(OpenFlags.ReadOnly);

    X509Certificate2 certificate = store.Certificates.Find(
                     X509FindType.FindBySubjectName,
                     "ALAZ Library", true)[0];
    serverKey = new RSACryptoServiceProvider();

    if (hostType == HostType.htClient)
    {
        //----- In client only public key is needed!
        serverKey = (RSACryptoServiceProvider)certificate.PublicKey.Key;
    }
    else
    {
        //----- In server, both public and private key is needed!
        serverKey.FromXmlString(certificate.PrivateKey.ToXmlString(true));
    }

    store.Close();
    */
}

The authentication message

The symmetric authentication uses the AuthMessage structure to exchange session keys between the client and the server. The SessionKey and SessionIV properties are, respectively, the symmetric key and the initialization vector of the algorithm. The Sign property is the hash code generated by the client using the sign RSACryptoServiceProvider class created internally, and its public key is exchanged using the SourceKey property. This internal sign key pair is necessary to sign the AuthMessage, and the server can ensure that the AuthMessage is accurate. This process is done using the following code:

Client side

...
//----- Sign Message!
private byte[] signMessage = new byte[]
                         { <sign message array of bytes for authentication> };
...
protected virtual void InitializeConnection(BaseSocketConnection connection)
{
...

//----- Symmetric!
if (connection.EncryptType == EncryptType.etRijndael ||
    connection.EncryptType == EncryptType.etTripleDES)
{
    if (FHost.HostType == HostType.htClient)
    {
        //----- Get RSA provider!
        RSACryptoServiceProvider serverPublicKey;
        RSACryptoServiceProvider clientPrivateKey =
                                new RSACryptoServiceProvider();

        FCryptoService.OnSymmetricAuthenticate(FHost.HostType,
                                          out serverPublicKey);

        //----- Generates symmetric algorithm!
        SymmetricAlgorithm sa =
          CryptUtils.CreateSymmetricAlgoritm(connection.EncryptType);
        sa.GenerateIV();
        sa.GenerateKey();

        //----- Adjust connection cryptors!
        connection.Encryptor = sa.CreateEncryptor();
        connection.Decryptor = sa.CreateDecryptor();

        //----- Create authenticate structure!
        AuthMessage am = new AuthMessage();
        am.SessionIV = serverPublicKey.Encrypt(sa.IV, false);
        am.SessionKey = serverPublicKey.Encrypt(sa.Key, false);
        am.SourceKey =
          CryptUtils.EncryptDataForAuthenticate(sa,
          Encoding.UTF8.GetBytes(clientPrivateKey.ToXmlString(false)),
          PaddingMode.ISO10126);

        //----- Sign message with am.SourceKey,
        //----- am.SessionKey and signMessage!
        //----- Need to use PaddingMode.PKCS7 in sign!
        MemoryStream m = new MemoryStream();
        m.Write(am.SourceKey, 0, am.SourceKey.Length);
        m.Write(am.SessionKey, 0, am.SessionKey.Length);
        m.Write(signMessage, 0, signMessage.Length);

        am.Sign = clientPrivateKey.SignData(
                  CryptUtils.EncryptDataForAuthenticate(sa,
                  m.ToArray(), PaddingMode.PKCS7),
                  new SHA1CryptoServiceProvider());

        //----- Serialize authentication message!
        XmlSerializer xml = new XmlSerializer(typeof(AuthMessage));
        m.SetLength(0);
        xml.Serialize(m, am);

        //----- Send structure!
        MessageBuffer mb = new MessageBuffer(0);
        mb.PacketBuffer =
          Encoding.Default.GetBytes(Convert.ToBase64String(m.ToArray()));
        connection.Socket.BeginSend(
            mb.PacketBuffer, mb.PacketOffSet,
            mb.PacketRemaining, SocketFlags.None,
            new AsyncCallback(InitializeConnectionSendCallback),
            new CallbackData(connection, mb));

        m.Dispose();
        am.SessionIV.Initialize();
        am.SessionKey.Initialize();
        serverPublicKey.Clear();
        clientPrivateKey.Clear();
    }
...
}

On the client side of the symmetric authentication, the OnSymmetricAuthenticate is called, getting the RSACryptoServiceProvider to encrypt the session key generated by the CryptUtils.CreateSymmetricAlgoritm method. The AuthMessage is filled with the encrypted session key, session IV, and the sign public key. To sign the message, the SourceKey, SessionKey, and signMessage are used, and the resulting hash is assigned to the Sign property.

Server side

protected virtual void InitializeConnection(BaseSocketConnection connection)
{
...
    if (FHost.HostType == HostType.htClient)
    {
    ...
    }
    else
    {
        //----- Create empty authenticate structure!
        MessageBuffer mb = new MessageBuffer(8192);

        //----- Start receive structure!
        connection.Socket.BeginReceive(mb.PacketBuffer, mb.PacketOffSet,
                 mb.PacketRemaining, SocketFlags.None,
                 new AsyncCallback(InitializeConnectionReceiveCallback), ...);
    }
}

private void InitializeConnectionReceiveCallback(IAsyncResult ar)
{
...

bool readSocket = true;
int readBytes = ....EndReceive(ar);

if (readBytes > 0)
{

    readMessage.PacketOffSet += readBytes;
    byte[] message = null;

    try
    {
        message = Convert.FromBase64String(
          Encoding.Default.GetString(readMessage.PacketBuffer,
          0, readMessage.PacketOffSet));
    }
    catch (FormatException)
    {
        //----- Base64 transformation error!
    }

    if ((message != null) &&
       (Encoding.Default.GetString(message).Contains("</AuthMessage>")))
    {

        //----- Get RSA provider!
        RSACryptoServiceProvider serverPrivateKey;
        RSACryptoServiceProvider clientPublicKey =
                    new RSACryptoServiceProvider();

        FCryptoService.OnSymmetricAuthenticate(FHost.HostType,
                                         out serverPrivateKey);

        //----- Deserialize authentication message!
        MemoryStream m = new MemoryStream();
        m.Write(message, 0, message.Length);
        m.Position = 0;

        XmlSerializer xml = new XmlSerializer(typeof(AuthMessage));
        AuthMessage am = (AuthMessage)xml.Deserialize(m);

        //----- Generates symmetric algorithm!
        SymmetricAlgorithm sa =
          CryptUtils.CreateSymmetricAlgoritm(connection.EncryptType);
        sa.Key = serverPrivateKey.Decrypt(am.SessionKey, false);
        sa.IV = serverPrivateKey.Decrypt(am.SessionIV, false);

        //----- Adjust connection cryptors!
        connection.Encryptor = sa.CreateEncryptor();
        connection.Decryptor = sa.CreateDecryptor();

        //----- Verify sign!
        clientPublicKey.FromXmlString(Encoding.UTF8.GetString(
                        CryptUtils.DecryptDataForAuthenticate(sa,
                        am.SourceKey, PaddingMode.ISO10126)));

        m.SetLength(0);
        m.Write(am.SourceKey, 0, am.SourceKey.Length);
        m.Write(am.SessionKey, 0, am.SessionKey.Length);
        m.Write(signMessage, 0, signMessage.Length);

        if (!clientPublicKey.VerifyData(
             CryptUtils.EncryptDataForAuthenticate(sa, m.ToArray(),
             PaddingMode.PKCS7),
             new SHA1CryptoServiceProvider(), am.Sign))
        {
            throw new
              SymmetricAuthenticationException("Symmetric sign error.");
        }

        readSocket = false;


        m.Dispose();
        am.SessionIV.Initialize();
        am.SessionKey.Initialize();
        serverPrivateKey.Clear();
        clientPublicKey.Clear();

        FHost.FireOnConnected(connection);

    }

    if (readSocket)
    {
        ....BeginReceive(readMessage.PacketBuffer,
                         readMessage.PacketOffSet,
                         readMessage.PacketRemaining,
                         SocketFlags.None,
                         new AsyncCallback(
                           InitializeConnectionReceiveCallback), ...);
    }

}

On the server side of the symmetric authentication, a MessageBuffer is used to receive the socket buffer. The read callback method continues to read till a completed AuthMessage is received. With this message, the method calls the OnSymmetricAuthenticate to get the RSACryptoServiceProvider to decrypt the session key, session IV, and the sign public key. With all the keys decrypted, the method verifies the Sign property to ensure that the AuthMessage is accurate, using the SourceKey, SessionKey, and signMessage.

Connection Creator

Although BaseSocketConnectionHost can manage ISocketConnection connections, it cannot create them. This job is made by BaseSocketConnectionCreator which creates and initializes ISocketConnections. The CompressionType and EncryptType properties define, respectively, the compression and the encryption types that will be used in the connection. The CryptoService defines the ICrytoService instance used to initialize the connection, if needed. The Host property defines the host of the BaseSocketConnectionCreator; it can be a server or a client host. The LocalEndPoint defines the socket IP end point used in the connection, and it can have different behavior depending on the type of the creator.

SocketServer and SocketListener

The SocketServer and SocketListener are the classes needed to create a socket server. SocketServer is derived from BaseSocketConnectionHost, and manages ISocketConnections. The SocketListener is derived from BaseSocketConnectionCreator, and listens for incoming connections, accepts a connection, and creates a new ISocketConnection to be used. A SocketServer can have as many SocketListeners attached as required, each one assigned to a local port to listen.

SocketServer constructor and methods

In the SocketServer constructor, the socketService parameter defines the ISocketService instance used by the server. The header parameters define the array of bytes used in the message header exchange. The socketBufferSize adjusts the socket buffer size. The messageBufferSize defines the maximum message size of the service. The idleCheckInterval indicates the interval for idle connections checking, in milliseconds. The idleTimeoutValue defines the timeout, in milliseconds, to be compared to each connection LastAction property.

To add SocketListener items in SocketServer, the method AddListener must be used. The localEndPoint parameter defines the local socket IP endpoint used to listen to connections. The encryptType and compressionType defines, respectively, the encryption and compression methods used in the new accepted connection. The cryptoService defines the ICryptoService used to authenticate the encryption method chosen. The backLog limits the listen queue of the OS socket to the defined number, and acceptThreads sets the calling number of the socket's BeginAccept to increase the accepted performance.

HostThreadPool

This library uses asynchronous socket communication which, in turn, uses the .NET ThreadPool. In the .NET 2.0 ThreadPool, the thread number can be controlled using the SetMaxThreads and SetMinThreads methods, and I think there are a lot of improvements in this class. But, if you don't want to use the .NET class, you can use a managed thread pool called HostThreadPool, very similar to Stephen Toub's ManagedThreadPool. HostThreadPool uses a list of managed threads that keeps increasing as more enqueueing tasks are provided. To use this class instead of the .NET ThreadPool in SocketServer, just set the minThreads and maxThreads constructor parameters to non-zero numbers.

Here are some examples of using SocketServer and SocketListener:

//----- Simple server!
SocketServer server = new SocketServer(new SimpleEchoService());
//----- Simple listener!
server.AddListener(new IPEndPoint(IPAddress.Any, 8087));
server.Start();
//----- Server with header!
SocketServer server = new SocketServer(new SimpleEchoService(),
                      new byte[] { 0xFF, 0xFE, 0xFD });
//----- Listener with simple encryption!
server.AddListener(new IPEndPoint(IPAddress.Any, 8087),
       EncryptType.etBase64, CompressionType.ctNone, null);
server.Start();
//----- Server with header and buffer
//----- sizes, no hostthreadpool and idle check setting!
SocketServer server = new SocketServer(new SimpleEchoService(),
                      new byte[] { 0xFF, 0xFE, 0xFD },
                      2048, 8192, 0, 0, 60000, 30000);
//----- More than one listener each one with different listen port number!
server.AddListener(new IPEndPoint(IPAddress.Any, 8087));
server.AddListener(new IPEndPoint(IPAddress.Any, 8088),
                   EncryptType.etBase64, CompressionType.ctNone, null);
server.AddListener(new IPEndPoint(IPAddress.Any, 8089),
                   EncryptType.etRijndael, CompressionType.ctGZIP,
                   new SimpleEchoCryptService(), 50, 10);
server.AddListener(new IPEndPoint(IPAddress.Any, 8090),
                   EncryptType.etSSL, CompressionType.ctNone,
                   new SimpleEchoCryptService());
server.Start();

SocketClient and SocketConnector

The SocketClient and SocketConnector are the classes needed to create a socket client. SocketClient is derived from BaseSocketConnectionHost and, like SocketServer, manages ISocketConnections. The SocketConnector is derived from BaseSocketConnectionCreator, and it connects with the socket server and creates a new ISocketConnection to be used. A SocketClient can have as many SocketConnectors attached as required, each one connecting to a socket server, and they can be assigned to a local address and a local port to start the connection.

SocketClient constructor and methods

The SocketClient constructor has the same parameter signature as the SocketServer class. To add SocketConnector items in SocketClient, the method AddConnector must be used. The remoteEndPoint parameter defines the remote socket IP endpoint used for the connection. The encryptType and compressionType define, respectively, the encryption and compression methods used in the new connection. The cryptoService defines the ICryptoService used to authenticate the encrypted method chosen. The reconnectAttempts and reconnectAttemptInterval define, respectively, the number of reconnect attempts when using BeginReconnect method and the time interval to reconnect. The localEndPoint defines the local socket IP endpoint used to start the connection process to the remote endpoint.

Here are some examples of using SocketClient and SocketConnector:

//----- Simple client!
SocketClient client = new SocketClient(new SimpleEchoService());
//----- Simple connector!
client.AddConnector(new IPEndPoint(IPAddress.Parse("10.10.1.1"), 8087));
client.Start();
//----- Client with header!
SocketClient client = new SocketClient(new SimpleEchoService(),
                      new byte[] { 0xFF, 0xFE, 0xFD });
//----- Connector with simple encryption!
client.AddConnector(new IPEndPoint(IPAddress.Parse("10.10.1.1"), 8087),
                    EncryptType.etBase64, CompressionType.ctNone, null);
client.Start();
//----- Client with header and buffer sizes,
//----- no hostthreadpool and idle check setting!
SocketClient client = new SocketClient(new SimpleEchoService(),
                      new byte[] { 0xFF, 0xFE, 0xFD },
                      2048, 8192, 0, 0, 60000, 30000);
//----- Connector with encryption and reconnect!
client.AddConnector(new IPEndPoint(IPAddress.Parse("10.10.1.1"), 8087),
                    EncryptType.etSSL, CompressionType.ctGZIP,
                    new SimpleEchoCryptService(),
                    5, 30000);
client.Start();
//----- Client with header and buffer sizes,
//----- using hostthreadpool and idle check setting!
SocketClient client = new SocketClient(new SimpleEchoService(),
                      new byte[] { 0xFF, 0xFE, 0xFD },
                      4096, 8192, 5, 50, 60000, 30000);
//----- Connector with encryption, reconnect and local endpoint!
client.AddConnector(new IPEndPoint(IPAddress.Parse("10.10.1.1"), 8087),
                    EncryptType.etSSL, CompressionType.ctGZIP,
                    new SimpleEchoCryptService(),
                    5, 30000,
                    new IPEndPoint(IPAddress.Parse("10.10.3.1"), 2000));
client.Start();
//----- Simple client!
SocketClient client = new SocketClient(new SimpleEchoService());
//----- More than one connector each one with different remote socket servers!
client.AddConnector(new IPEndPoint(IPAddress.Parse("10.10.1.1"), 8087));
client.AddConnector(new IPEndPoint(IPAddress.Parse("10.10.1.2"), 8088),
                    EncryptType.etBase64, CompressionType.ctNone, null);
client.AddConnector(new IPEndPoint(IPAddress.Parse("10.10.1.3"), 8089),
                    EncryptType.etRijndael, CompressionType.ctGZIP,
                    new SimpleEchoCryptService());
client.AddConnector(new IPEndPoint(IPAddress.Parse("10.10.1.4"), 8090),
                    EncryptType.etSSL, CompressionType.ctNone,
                    new SimpleEchoCryptService(),
                    5, 30000,
                    new IPEndPoint(IPAddress.Parse("10.10.3.1"), 2000));
client.Start();

Echo demo project

There's an Echo demo project available in the article download file, using Console, Windows Forms, and Windows Service hosts and clients, and all them use the same EchoSocketService and EchoCryptService. The demos are divided by their type as follows:

Hosts

  • Console
    1. EchoConsoleClient
    2. EchoConsoleServer
  • Windows Forms
    1. EchoFormClient
    2. EchoFormServer
    3. Echo<code>Form (Forms template)
  • Windows Service
    1. EchoWindowsServiceServer

Services

  • EchoSocketService
  • EchoCryptService

Conclusion

There's a lot going on here, and I think this library can help anyone who wants to write asynchronous sockets with encryption and compression. Any comments will be appreciated.

History

  • May 15, 2006: Initial version
  • May 19, 2006: Some English text corrections (sorry, I'm still learning!), and rechecking the demo sources
  • June 06, 2006: Version 1.2 with the following changes:
    • Minor bugs fixed
    • All "Sended" changed to "Sent" (thanks vmihalj)
    • ReadCanEnqueue now works correctly and with HostThreadPool (thanks PunCha)
    • Added reconnectAttempts and reconnectAttemptInterval to allow client connection reconnects as many times as is needed in a timed interval (thanks Tobias Hertkorn)
  • April 01, 2007: Version 1.3 with the following changes:
    • Fixed rawbuffer = null
    • Fixed BeginAcceptCallback: stops accepting if exception occurs
    • Fixed BeginSendCallback: PacketRemaining bytes should be used
    • Socket config section added in demos
    • New message size (64K)
    • HosThreadPool removed
    • Header changed to Delimiter property with new delimiter options:
      1. dtNone: No message delimiter
      2. dtPacketHeader: Version 1.2 backward
      3. dtMessageTailExcludeOnReceive: Use custom delimiter at end of message (Exclude delimiter on receiving)
      4. dtMessageTailIncludeOnReceive: Use custom delimiter at end of message (Include delimiter on receiving)
    • New connection object properties/methods:
      1. Nagle, Linger and TTL Algorithm options
      2. Host and Creator
    • Encrypt sign message in service class
    • Exception event in service class
    • New Creator name property
  • July 22, 2007: Version 1.4 with the following changes:
    • Connection initialize procedures executing in same thread (not queued in ThreadPool)
    • Connection disconnect now checks windows version and executes the correct disconnect procedure
    • Connection Active checking for Disposed
    • CheckSocketConnections disposed check fixed
    • CryptUtils Flush() method included
    • Client Connection BeginConnect() exception fixed
    • Server Connection BeginSendToAll array buffer fixed
    • New SocketClientSync class for synchronous using (WinForms demo included)
  • September 5, 2007: Version 1.5 with the following changes:
    • SocketClient with proxy authentication (SOCKS5, Basic HTTP)
    • BeginRead bug fixed (messagetail)
    • BeginDisconnect changed (threadpool)
    • BeginSendToAll reviewed (disposed check)
    • New OnSSLClientValidateServerCertificate event to validate server's certificate
    • Idle check interval set to 0 and only created when greater than 0
    • Using Buffer.BlockCopy instead of Array.Copy
    • New Chat Demo

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Andre Azevedo
Software Developer (Senior)
Brazil Brazil
- Living in São Paulo, Brazil
- Developing since 1994 with
* Clipper (Summer '87 and 5.02)
* FoxPro (DOS, 2.6), Visual Foxpro (6, 7, 8, 9)
* Delphi (1, 2, 5, 7, 2007)
* C# (2.0, 4.0)

Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board.
Search this forum  
    Spacing  Noise  Layout  Per page   
Questionoverflow exceptionmembershoaib.adil27-Apr-13 2:59 
System.OverflowException: Arithmetic operation resulted in an overflow.
at xyz.Communication.NetworkEx.SocketsEx.BufferUtils.GetRawBufferWithTai
l(BaseSocketConnection connection, SocketAsyncEventArgs e, Int32 position, Int32
delimiterSize) in Sockets\Source\xyz.Communication.NetworkEx\SocketEx\BufferUtils.vb:line 76
at xyz.Communication.NetworkEx.SocketsEx.BaseSocketConnectionHost.ReadMe
ssageWithTail(BaseSocketConnection connection, SocketAsyncEventArgs e, Int32 rea
dBytes, Boolean& onePacketFound) in Sockets\Source\Datasoft.Communication.NetworkEx\SocketEx\BaseSocketConnectionHost
.vb:line 1115
at xyz.Communication.NetworkEx.SocketsEx.BaseSocketConnectionHost.ReadFr
omConnection(BaseSocketConnection connection, Int32 readBytes) in Sockets\Source\xyz.Communication.NetworkEx\Soc
ketEx\BaseSocketConnectionHost.vb:line 977
at xyz.Communication.NetworkEx.SocketsEx.BaseSocketConnectionHost.BeginR
eadCallbackAsyncP(Object state) in Source\xyz.Communication.NetworkEx\SocketEx\BaseSocketConnectionHost.
vb:line 914
this is the exception i am facing please help me out
GeneralMy vote of 5membersolorg18-Jan-13 9:09 
Concise design & thorough explanation!
QuestionData Table in Asynchronous socket programmingmembersaeedazam78629-Dec-12 23:06 
Hi,
Nice Article.Can you tell me how can i send DataTable from server to client in asynchronous socket programming in c# windows form Application.

QuestionAny upgrade of ALAZ planned? Or even available soon?membermklaey31-May-12 7:21 
Hi Andre Luis,
 
I am using your excellent socket extension in another open source project called "YAT - Yet Another Terminal" (https://sourceforge.net/projects/y-a-terminal/). YAT features an RS-232/422/423/485 terminal optimized for communication with embedded systems. In addition it also supports TCP client/server/AutoSocket and UDP connections for PC based embedded systems simulations, AutoSocket being an automatic client/server detection mechanism.
 
At first I was using ALAZ 1.4/1.5 on .NET 2.0, then updated to ALAZ 2.0 and .NET 3.5. After the update it didn't build right from start. But then I realized that back in 2007 I made significant modifications to ALAZ:
- Adding support for UDP (HostType.htUdp)
- Improving stability by adding exception handling at several places
- Changing SocketsEx\BaseSocketConnectionHost.StopConnections() from blocking to non-blocking
- Some other modification which in retrospect where rather silly
 
I was glad to see that you have improved the exception handling in ALAZ 2.0. So I didn't need to redo these modifications. But I had to redo the support for UDP. Isn't a big thing, still, I am wondering whether you could consider to add UDP support to the original ALAZ library for future versions. In retrospect, I should have given you this input already in 2007, my fault. Better later than never Wink | ;-)
 
UDP
In order to support UDP, I made modifications at the following locations:
- \SocketsEx\SocketConnector
> private ProtocolType FProtocolType;
> Constructor taking the protocol type as additional argument
> BeginConnect() taking the protocol type into account
- \SocketsEx\SocketClient
> using System.Net.Sockets;
> Another constructor taking the protocol type as additional argument
- \SocketsEx\SocketServer
> using System.Net.Sockets;
> Another constructor taking the protocol type as additional argument
- \SocketsEx\SocketClientSync
> private ProtocolType FProtocolType;
> Constructor taking the protocol type as additional argument
> Connect() taking the protocol type into account
- \SocketsEx\BaseSocketConnectionHost
> private ProtocolType FProtocolType;
> public ProtocolType ProtocolType { get; }
> Constructor: Protocol type as additional argument
 
SocketsEx\BaseSocketConnectionHost.StopConnections()
I am not exactly getting why this method is blocking and therefore makes Stop() blocking. The Start() method isn't blocking.
If Stop() is called from a GUI thread and the GUI is attached to the Disconnected event, a dead-lock happens:
- The GUI thread is blocked here
- FireOnDisconnected is blocked when trying to synchronize Invoke() onto the GUI thread
 
However, I am then getting an ObjectDisposedException in CloseConnection() on connection.Socket.Shutdown(SocketShutdown.Send) when stopping:
System.ObjectDisposedException was unhandled
Message="Cannot access disposed object \"System.Net.Sockets.Socket\"."
Source="System"
ObjectName="System.Net.Sockets.Socket"
StackTrace:
at System.Net.Sockets.Socket.Shutdown(SocketShutdown how)
at ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.CloseConnection(BaseSocketConnection connection) in \SocketsEx\BaseSocketConnectionHost.cs:Line 1805.
at ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.BeginDisconnectCallbackAsync(Object sender, SocketAsyncEventArgs e) in \SocketsEx\BaseSocketConnectionHost.cs:Line1489.
at System.Net.Sockets.SocketAsyncEventArgs.OnCompleted(SocketAsyncEventArgs e)
at System.Net.Sockets.SocketAsyncEventArgs.ExecutionCallback(Object ignored)
at System.Threading.ExecutionContext.runTryCode(Object userData)
at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Net.Sockets.SocketAsyncEventArgs.FinishOperationSuccess(SocketError socketError, Int32 bytesTransferred, SocketFlags flags)
at System.Net.Sockets.SocketAsyncEventArgs.CompletionPortCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)
I simply commented-out the connection.Socket.Shutdown(SocketShutdown.Send) call and it works in my case (but probably not in other cases, also see below).
 
Disposal
Probably due to the modifications mentioned above, tt seems that TCP server sockets don't properly shut down when Dispose() is called. As I understand the purpose of Dispose(), a call to this method must always be possible and it must immediately release all resources the object holds. In case of TCP server this seems not the case.
 
Distribution
There are a few files in your distribution which I think shouldn't be in there. I had to remove/untick them before checking them into my SVN.
- *.user
- \Code\Demos\Echo\EchoConsoleServer\Service References
- \Code\Demos\Echo\EchoFormServer\*.tmp
- \Code\Source\ALAZ.SystemEx\bin\
- \Code\Source\ALAZ.SystemEx.NetEx\bin\
 
Some other inputs
- ALAZ AssemblyInfo.cs should have [assembly: CLSCompliant(true)] enabled
> Allows libraries/programs to be CLS compliant
- The "ALAZLibSN.snk" should be within "\Properties"
- The "ALAZLibSN.snk" should be available in both projects
- The "ALAZLibSN.snk" should be referenced/enabled in both projects
- How about adding a static class diagram to the Visual Studio project?
- How about providing BaseSocketConnectionHost/SocketClient/SocketServer constructors that use ctWorkerThread by default?
- How about using System.Threading.Timeout.Infinite to emphasize infinite timeout values?
- How about providing default constants for 1024 * 2 and 1024 * 16?
 
I came across another issue when shutting down a pair of an ALAZ client connected to an ALAZ server within the same application. I ran into a deadlock while shutting down the open connection. It happens in ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnection.Active.get(). Is there a need to lock the field when reading its status? The situation is as follows:
Thread A) at ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.BeginReadCallbackAsyncP trying to call connection.BeginDisconnect()
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnection.Active.get() Line 286
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.FireOnException(ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnection connection =
{ALAZ.SystemEx.NetEx.SocketsEx.ClientSocketConnection}, System.Exception ex = {Cannot evaluate expression because the current thread is in a sleep, wait, or join}) Line 551
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.BeginDisconnect(ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnection connection =
{ALAZ.SystemEx.NetEx.SocketsEx.ClientSocketConnection}) Line 1453
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnection.BeginDisconnect() Line 558
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.BeginReadCallbackAsyncP(object state = {System.Net.Sockets.SocketAsyncEventArgs}) Line 1161
Thread B) at ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.BeginReadCallbackAsyncP trying to get the connection state
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnection.Active.get() Line 286
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.BeginReadCallbackAsyncP(object state = {System.Net.Sockets.SocketAsyncEventArgs}) Line 1146
The Main Thread at ALAZ.SystemEx.NetEx.SocketsEx.SocketServer.Stop()
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnection.Active.get() Line 286
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.BeginDisconnect(ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnection connection =
{ALAZ.SystemEx.NetEx.SocketsEx.ServerSocketConnection}) Line 1435
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnection.BeginDisconnect() Line 558
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.StopConnections() Line 340
> ALAZ.SystemEx.NetEx.dll!ALAZ.SystemEx.NetEx.SocketsEx.SocketServer.Stop() Line 207
 

Also, there seems to be a null reference issue in ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.StopCreators() on line 311 (FWaitCreatorsDisposing.WaitOne). I added a try-catch around the statement.
 

Another issue I found in SocketListener.BeginAcceptCallback on line #229 AcceptAsync(e2):
 
An unhandled System.ObjectDisposedException was thrown while executing this test : The SafeHandle has been closed.
 
at System.Net.Sockets.Socket.AcceptAsync(SocketAsyncEventArgs e)
at ALAZ.SystemEx.NetEx.SocketsEx.SocketListener.BeginAcceptCallback(Object state) in D:\Sandboxes\YAT\_Trunk\ALAZ\Source\ALAZ.SystemEx.NetEx\SocketsEx\SocketListener.cs:Zeile 229.
at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.runTryCode(Object userData)
at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)
 
This issue occured while performing endurance tests using YAT AutoSockets.
 

And the last issue for today, found in BaseSocketConnection.Active:
 
I get another deadlock because I have to synchronize events onto my main/GUI thread. In case of having an application with two sockets connected to each other I get a deadlock upon shut down. Thus I have removed the lock in the get{} property of BaseSocketConnection.Active.
 

UDP
I have removed all UDP stuff and use System.Net.Sockets.UdpClient directly
 

Now, are you still developing ALAZ? Or has this project, as so many others, come to a dead-end?
 
Best regards,
Matthias
AnswerRe: Any upgrade of ALAZ planned? Or even available soon?memberAndre Azevedo31-May-12 8:14 
Hello,
 
The purpose of this library is show how Sockets can be development on .Net; it cannot be called a "project" with upgrades/new versions. Maybe I'll write some update using .Net 4.5 but, till there, you can check SuperSocket Library hosted on codeplex.
 
Regards
________________________________________________________________
Yes, we can!

GeneralRe: Any upgrade of ALAZ planned? Or even available soon?membermklaey12-Oct-12 21:51 
Hello Andre,
 
Thanks for the hint. Over the last couple of days I indeed evaluated SuperSocket, among some other libraries as show n below.
 
However, I found that SuperSocket doesn't really suit the needs of YAT. SuperSocket is designed to be used for higher level applications, such as custom protocols on TCP/IP. YAT doesn't deal with protocols, it just requires a socket library on binary stream level, but more convenient then what System.Net.Sockets provides by default.
 
Other libraries I looked at:
> DotnetAsyncSocket, a "TCP/IP socket networking library for .NET" http://code.google.com/p/dotnetasyncsocket/[^]
> Extasys, an "Asynchronous TCP/UDP socket library" http://code.google.com/p/extasys/[^]
> SocketAsyncServerAndClient, "C# SocketAsyncEventArgs High Performance Socket Code" C# SocketAsyncEventArgs High Performance Socket Code[^]
 
But these libraries have not really proven good enough or suitable for YAT, better than ALAZ, or doesn't seem to be active anymore either. So for moment, I just stick to ALAZ.
QuestionSSL authentication & Client authentication in .netCFmemberrama srinu2-May-12 3:44 
Good article for socket programing. I am working in windows mobile using .netCF, can you provide code or suggestion for implement SSL authentication & Client authentication using .netCF ?
SuggestionClient authentication with certificatesmemberTim_Schneider14-Apr-12 22:01 
Hello,
 
ich found out that the library gives the server no way to check the client certificate - it just checks for signed or not. so if a client sends a valid and signed certificate back to server it will pass the test - whether its right or wrong.
 
i simply use the same ValidateServerCertificate for that - server side.
 
In BaseSocketConnectionHost.cs Line 2170
 
//----- Authneticate SSL!
											SslStream ssl = new SslStream(new NetworkStream(connection.Socket), false, new RemoteCertificateValidationCallback(connection.BaseCreator.ValidateServerCertificateCallback));
                                            ssl.BeginAuthenticateAsServer(cert, clientAuthenticate, System.Security.Authentication.SslProtocols.Default, checkRevocation, new AsyncCallback(SslAuthenticateCallback), new AuthenticateCallbackData(connection, ssl, HostType.htServer));
 
that gives me the ability to check the client cert ... Smile | :)
 
Thanks tim!
QuestionTrabalho em SPmembermbsteireira10-Jan-12 7:58 
Olá, André. Estou precisando de pessoas com seu perfil para um trabalho em SP.
Se puder, entre em contato comigo pelo e-mail mbsteixeira@gmail.com
Grato,
Manoel
Manoel Teixeira
manoel@fonetica.com.br

QuestionThanks wery mutch!!! It helps me implement big file transfermemberUngVas5-Jan-12 22:54 
...in bad network (3G)
GeneralMy vote of 5memberURVISH SUTHAR from Ahmadabad Gujarat, India21-Nov-11 1:40 
gr8

thx!!!
QuestionImplementing a message length headermemberIvan Moosaka25-Oct-11 2:55 
Greetings André!
 
First of all a huge thank you for writing the article and the library. The code is clean and concise, I learned a great deal from it and it really cleared up the subject on quite a few things for me. I've looked around for a similar library and yours is hands down the best one around.
 
I am implementing a message oriented layer on top of TCP that should conform to an event based interface almost identical to NetEx's. I am thinking about reusing your robust and tested code rather than reinventing the wheel but have noticed the header option had been removed some time ago. Spec for my library requires a two byte message length header (a prefix) and since your library seems mature I wouldn't want to mess it up with a quick and dirty "fix".
 
I would like to have your opinion on whether this is a trivial change or a more serious redesign would be required. I'd appreciate any pointers you are willing to offer.
 
Thanks in advance & kind regards from Zagreb, Croatia
Moosaka

QuestionUse of asyncronous socket programming with webserver and GPRS telemetry unitmemberGlen Johnston17-Jan-11 19:47 
Dear Andre,
I'm writing to ask your advice whether yourself, or someone you may recommend, might be able to help us sort out a problem we have come across in trying to write a web-server that receives asyncronous data from a number of GPRS telemetry transmitters.
I will first admit that I am not a web programmer, and have fairly limited programming experience, but we have been working with another programmer who has written a php based webserver, but it's getting stuck on the TCP hold issue. And that is once a telemetry sender has connected its socket with the web server socket, communications with other telemetry units is locked out. So we can only handle one socket connection at a time. Whereas we need to be able to receive connections and data from several sources simultaneously, or randomly, as they are all transmitting their data without any orchestration between them or the server.
 
As I read your excellent article on asyncronous socket programming, it seems that that sort of capability is what we need to address this problem.
Could you please advise if you might be able to help us?
With thanks,
Glen Johnston
Canberra, Australia.
GeneralMy vote of 5memberyakkantiadireddy19-Dec-10 19:41 
Amazing you given entire socket programing for us thanks a lot.
GeneralIOCP based socket servermemberVF2-Sep-10 23:34 
https://sourceforge.net/projects/socketservers
Generalclient side SocketClientSync.Read lost packets when server side sends multiple e.Connection.BeginSendmemberkelvin199719-Mar-10 0:25 
I have tested everyting locally it was fine.
But once I put the server part to the remote server this is what happen.
The server fired 2 messages to the client:
 
for each message in messages
e.Connection.BeginSend(SerializeMessage(msg))
next

 
I tested it locally and it was fine.
Once the server is running remotely, the client only picks up 1 message, and lost the other.
Then I hardcoded the server to wait between each message:
 
for each message in messages
e.Connection.BeginSend(SerializeMessage(msg))
Threading.Thread.Sleep(1000) '// <-- delay a bit!
next

 
Guess what, it worked 1 by 1 with the delay. It is strange. Why can't I fire multiple messages at once?
Please suggest me what to do?
GeneralW2k3 64 server: Exception System.Security.Cryptography.CryptographicException: Object identifier (OID) is unknown.memberkelvin199718-Mar-10 1:32 
Exception System.Security.Cryptography.CryptographicException: Object identifier (OID) is unknown.
-
Worked well under testing environment. However, this is the error I got when I put it on to a windows 2003 x64 server.
Is there a fix to it? Anyone happen to know how? Please help!!! Confused | :confused:
Generaldetail stack trace attachedmemberkelvin199718-Mar-10 1:37 
SocketService::StartServer: Socket Server Started!
SocketService::OnException: Exception Details
Connection Id 1001
Exception System.Security.Cryptography.CryptographicException: Object identifier (OID) is unknown.
at System.Security.Cryptography.X509Certificates.X509Utils._GetAlgIdFromOid(String oid)
at System.Security.Cryptography.X509Certificates.X509Utils.OidToAlgId(String oid)
at System.Security.Cryptography.RSACryptoServiceProvider.VerifyHash(Byte[] rgbHash, String str, Byte[] rgbSignature)
at ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.OnReceived(BaseSocketConnection connection, Byte[] buffer) in C:\Users\Admin\Documents\Visual Studio 2008\Projects\ALAZ.Demo.AsyncSocketServerandClient\Source\ALAZ.SystemEx.NetEx\SocketsEx\BaseSocketConnectionHost.cs:line 2335
Stack Trace at System.Security.Cryptography.X509Certificates.X509Utils._GetAlgIdFromOid(String oid)
at System.Security.Cryptography.X509Certificates.X509Utils.OidToAlgId(String oid)
at System.Security.Cryptography.RSACryptoServiceProvider.VerifyHash(Byte[] rgbHash, String str, Byte[] rgbSignature)
at ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.OnReceived(BaseSocketConnection connection, Byte[] buffer) in C:\Users\Admin\Documents\Visual Studio 2008\Projects\ALAZ.Demo.AsyncSocketServerandClient\Source\ALAZ.SystemEx.NetEx\SocketsEx\BaseSocketConnectionHost.cs:line 2335
SocketService::OnDisconnected: OnDisconnect 1001
SocketService::SendClientDisconnectMessageToAdmins: Notify disconnect 1001
Generalfixedmemberkelvin199718-Mar-10 23:21 
It is a bug with win2k3 64 servers. vs2010 and dotNet 4 fixed the issue.
QuestionOnly 4 Connections available ?memberliaomd8-Mar-10 9:15 
great work by you~
i got a problem running the demo chatclient and chatserver,it seemed that when i runned the server and client on the same computer,the server couldn't hold a connection for the fifth client !i mean the connection won't last long ,it disconnected the moment it connected the server,i didn't event had a chance to send message.but the preceded four connections worked well,i don't why.
AnswerRe: Only 4 Connections available ?memberkiddailey16-Aug-11 10:58 
A simple trace reveals that an exception is happening:
 
ALAZ.SystemEx.NetEx.SocketsEx.MessageLengthException: Message length is greater than Host maximum message length.
at ALAZ.SystemEx.NetEx.SocketsEx.BaseSocketConnectionHost.BeginSend(BaseSocketConnection connection, Byte[] buffer, Boolean sentByServer) in C:\AsyncSocketServerandClient\Code\Source\ALAZ.SystemEx.NetEx\SocketsEx\BaseSocketConnectionHost.cs:line 582
 
So obviously, the problem is that the MessageBufferSize in both the server and client are too small at 512 bytes. The message buffer includes a list of the connected users (to display in the user list panel of the client) in some instances, so once you include a 5th user, the buffer size requires more than 512 bytes.
 
You can fix this by increasing the MessageBufferSize in both the client and server. In my tests, setting it to 1024 allowed me to connect over 20 clients without issues. Note though that as the number of users and/or length of nicknames increases, the size requirement will increase.
 
In ChatClient/frmClient.cs:
client.MessageBufferSize = 1024; // Was 512;
 
In ChatConsoleServer/Main.cs:
chatServer.MessageBufferSize = 1024; // Was 512;
 
In ChatServiceServer/ChatServerService.cs:
chatServer.MessageBufferSize = 1024; // Was 512;

QuestionWhere can i find the old code for .net 2.0 ?membergil_adino6-Mar-10 20:03 
where can i found old version of this great app??
 
Tnx
AnswerRe: Where can i find the old code for .net 2.0 ?membermmoble12-Mar-10 7:14 
I'm also in need of the .NET 2.0 version. I think it's version 1.5 of your code.
 
Thanks!
AnswerRe: Where can i find the old code for .net 2.0 ?memberMartin.Mas17-Jun-10 7:48 
Hello,
 
I am looking also for old 1.5 version. Haven't you made any progress with searching about it?
 
Martin
QuestionHow to create client certificate for client autheticationmemberchinkuanyeh28-Feb-10 21:44 
Hi,
 
Does anybody know how to create client certificate for client authetication?
I tried many times but always got an AuthenticationException: The remote certificate is invalid according to the validation procedure.
 

Matthew
QuestionSocketAsyncEventArgs removed?memberMember 407840622-Dec-09 22:43 
Hi,
 
SocketAsyncEventArgs removed?
 
Any suggestion? Thank you
 
Matthew
GeneralFreezing on BaseSocketConnectionHost.FWaitConnectionsDisposing.WaitOne(Timeout.Infinite, false);memberMember 306669914-Dec-09 6:24 
Im using Windows 7 / C# express 2008.
1. I start the server.
2. Start the cliente ( connection is ok ).
3. If i stop the cliente, its freeze and on debugging, i can see it stops at that line...
 
It occurs also with echo demo. Other demos i dont have tested.
Im trying to write a very simple client/server to manage my lanhouse. Until now, i have used the RakNetDotNet, but i need to use sothing smaller and more simple.
I need to exchange no more than 15 litle messagens with server, almost all is a word like "CloseProgram","Minimize", "Restore", or "StartTime;30:00;aUsername".
The most complex is a screenshot.
Im from Brasil (RS).
GeneralRe: Freezing on BaseSocketConnectionHost.FWaitConnectionsDisposing.WaitOne(Timeout.Infinite, false);memberEfrat211215-May-10 20:21 
Hi,
I am facing the same problem exactly. Working with WinXP and C# 2008
What am I doing wrong??
Thanks
GeneralTransfer filesmemberchinkuanyeh22-Oct-09 20:03 
Hi,
 
What is the best way to implement transfering files with this library.
 
Any suggestion? Thank you Smile | :)
 
Matthew
QuestionRe: Transfer filesmemberSergnet2-Sep-10 4:03 
Good time!
I'm trying to implement the transfer of files in the following form.
To send a class is used like a ChatMessage.
 
The Client
 
In a series of reading a portion of the file and write it in msg.Message.
Next
client.Write (. SerializeMessage(msg.Message))
 
The server
....
 public override void OnReceived(MessageEventArgs e)
 {
     FileTransferMessage msg = DeserializeMessage(e.Buffer);
     ConnectionUserData cnUsrData = (ConnectionUserData)e.Connection.UserData;
 
   ...
   case MessageType.mtMessage:
   MessLen = msg.Message.Length;
   if (MessLen > 0) 
   { 
      fileStW = new FileStream(cFile, FileMode.Append, FileAccess.Write);
      buffRes = Encoding.GetEncoding(1251).GetBytes(msg.Message);
      BufLenght = buffRes.Length;
      fileStW.Write(buffRes, 0, BufLenght); // MessLen
      ((ConnectionUserData)e.Connection.UserData).SizeTran += BufLenght; //MessLen;
      fileStW.Close();
      msg.Message = "Reseived " + ((ConnectionUserData)e.Connection.UserData).SizeTran.ToString() + 
         " bytes - OK";
   }
   else msg.Message = "File Transfer " + cnUsrData.File.NameFile + " is done..";
   e.Connection.AsServerConnection().BeginSend(SerializeMessage(msg));
   .....
 }
There is a problem.
If I run the process to debug it all works. But when working in runtime on the server after three
iterations error "Binary stream does not contain a valid binary header .."
 
Excuse for my english
GeneralRe: Transfer filesmemberSergnet2-Sep-10 4:09 
I will be grateful for the help !
AnswerRe: Transfer filesmemberSergnet3-Sep-10 2:23 
Problem solved!
Posted on the client
Thread.Sleep(100);
after client.Write
QuestionWhy DelimiterType.dtPacketHeader removed?memberchinkuanyeh22-Oct-09 18:22 
Hi,
 
I have noticed that DelimiterType.dtPacketHeader was removed.
But why?
 
Thanks Smile | :)
 
Matthew
AnswerRe: Why DelimiterType.dtPacketHeader removed?memberSindowZ28-Oct-09 18:01 
In an earlier version of the library, the header was removed to have a delimiter. I actually modified it to have a header system instead and it wasn't any more stable in terms of sending messages. The delimiter works just as well as the header, so there shouldn't be a problem for your use.
GeneralRe: Why DelimiterType.dtPacketHeader removed?memberchinkuanyeh28-Oct-09 19:09 
Thanks for your reply , SindowZ.
But how about tranferring files? Is the delimiter suitable for it?
GeneralRe: Why DelimiterType.dtPacketHeader removed?memberSindowZ1-Nov-09 7:17 
Both the delimiter and header would be fine for sending files. The idea behind both is to cut the file into chunks, send that chunk, make sure the chunk is correct and repeat. There should be some segmented file sending projects on here that you could get the idea from and then use this library for.
GeneralRe: Why DelimiterType.dtPacketHeader removed?memberchinkuanyeh1-Nov-09 14:32 
Hi, SindowZ
 
Actually, I am just wondering if part of the file content will be the same with the delimiter (byte[]).
Will everything ok that way? Thx. Smile | :)
GeneralRe: Why DelimiterType.dtPacketHeader removed?memberSindowZ2-Nov-09 21:33 
Yes, with a long/unique enough delimiter the file/parts of the file should be parsed correctly.
Question嘿 Andre Azevedo 兄弟 你好!memberww21xx200912-Oct-09 21:41 
很感谢你把 ALAZ Library 开源 我在这里面学了很多的经验,太感谢你了。 不过我在测试ChatConsoleServer这个项目的时候出现了Process is terminated due to StackOverflowException错误!包括我用ALAZ Library开发的程序也一样,在本地很好的执行,但是一放到外网就会出现那个错误! 希望Andre Azevedo兄弟能帮我解答下 谢谢了! 非常感谢!
AnswerRe: 嘿 Andre Azevedo 兄弟 你好!membereblis8828-Oct-10 15:52 
Boy... I think he cannot understand what you said...you should post it in english...
GeneralRe: 嘿 Andre Azevedo 兄弟 你好!memberww21xx200929-Jun-11 21:03 
Laugh | :laugh: Haha, thank you brother, I think is
GeneralUDP socketmemberinew13-Sep-09 1:43 
Dear Andre Azevedo,
 
Can you post an example about Asynchronous Socket (using UDP socket)?
I need a high performance UDP server.
I read the sample at http://www.codeproject.com/KB/IP/ChatAppAsynchUDPSocks.aspx, but it has bad performance.
 
With your experience, please help me
 
thank you
 
inew

GeneralRe: UDP socketmemberMickaelG16-Nov-09 4:21 
Hi
with a bit late, I guess you have to change the socket type and the protocol type in the SocketConnector and SocketListener class
Replace
FSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
by
FSocket = new Socket(AddressFamily.InterNetwork, SocketType.Dgram, ProtocolType.Udp);
 
I think there are other changes due to UDP protocol.
GeneralSystem.InvalidOperationExceptionmemberDavid Kaplan2-Sep-09 21:10 
Very neat code.
I have created a service which includes this project. In my project, there are currently about 250 GPRS based TCP clients connected. Each client sends a very short 4 byte TCP payload to prevent the cellular PPP connection from closing. These KeepAlive messages are acknowledged by the server. I set the FIdleTimeOutValue so that the CheckSocketConnections() will disconnect a socket that is idle for 10 minutes which is way over the cellular PPP closure time.
 

My server settings are:
               listener.AcceptThreads = 3;
               listener.BackLog = 100;
               SocketBufferSize = 1024;
               MessageBufferSize = 2048;
 
               listener.EncryptType = EncryptType.etNone;
               listener.CompressionType = CompressionType.ctNone;
               DelimiterType = DelimiterType.dtNone;)
 
1) My server is crashing once every few days with the following messages logged beforehand:
 
===========
[9/2/2009 1:22:58 PM] [Prior]:8 [Type]Blush | :O ------------------------------------------------
Exception - System.Net.Sockets.SocketException
Exception Message - The I/O operation has been aborted because of either a thread exit or an application request
Socket Error - 995
------------------------------------------------
 
[9/2/2009 1:22:59 PM] [Prior]:5 [Type]Blush | :O [SERVER DISCONNECTED] [CONNECTIONID]:98 79.149.127.29:1036
[9/2/2009 1:23:00 PM] [Prior]:1 [Type]Blush | :O System.InvalidOperationException: Cannot apply a context that has been marshaled across AppDomains, that was not acquired through a Capture operation or that has already been the argument to a Set call.
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Net.Sockets.SocketAsyncEventArgs.FinishOperationSuccess(SocketError socketError, Int32 bytesTransferred, SocketFlags flags)
   at System.Net.Sockets.SocketAsyncEventArgs.CompletionPortCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
   at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)
===========
 
The windows event log shows:
EventType clr20r3 and system.invalidoperationexception
 
Could it be that one of the CheckSocketConnections() function's BaseSocketConnection items has already been closed when the cnn.BeginDisconnect(); is called?
 
Do you have any ideas on how to track down this problem that randomly happens over a couple of days in a release version?
 
Where should I look?
The service may be hosted on a dual or quad core PC.
Is there any place I may be should add a try catch sequence?
Since this is a Cellular end unit project every time that the server restarts costs more data as GPRS PPP must reconnect and the providers' charge for all data transferred.
 
2) Packed Keep Alive in incoming messages
Sometimes the short incoming 4 byte KeepAlive messages are received packed in groups of 2 or 3.
I have set Nagle to off in the server.
Is there any reason you can think of that the sever would accumulate tiny TCP packets?
I guess the problem is on the cellular GPRS end units or in the cellular provider's backend but since I am writing I thought to ask.
 

 
Thank you very much
 
David Kaplan
GeneralRe: System.InvalidOperationExceptionmemberelicoo6-Oct-09 22:37 
Hi David
 
Did you find the solution for this error?
We encountered the same error on our server.
 
Please help
GeneralRe: System.InvalidOperationExceptionmemberMember 366067028-Apr-10 0:54 
Hi David
 
Did you find the solution for this error?
 
Please help me
GeneralIndexOutOfRangeException in ReadMessageWithTailmemberSindowZ1-Aug-09 22:02 
Hello,
 
Thank you very much for this library. I have been writing a chat program using it and it has performed wondefully. There is one error that has kept on popping up throughout my development though. Sometimes I will get an IndexOutOfRange exception when the client receives a message from the server. This seems to happen most often when I am sending several packets at once to one client but it also happens randomly when receiving messages. Sending one packet to several clients works just fine but sending several packets to one client always breaks it. I have since changed my code so there are no bursts of packets like that but having the thread sleep for at least 300 ms always kept it from crashing.
 
Both the client and the server are inheriting from BaseSocketService. Changing the client to use SocketClientSync resulted in the same error.
 
Do you know what could cause this or how I could fix it? Thank you
GeneralRe: IndexOutOfRangeException in ReadMessageWithTailmembermikey2225-Aug-09 9:22 
This is your lucky day, I think.
 
I have been having trouble like this frequently and spent a few days last week tracking it down. The problem was mostly in the routine ReadMessageWithTail that extracts messages from the receive buffers. The reason it shows up when sending several packets at once, is that the first packet always (?) goes in a single socket-level buffer, but that messages sent 'quickly' may be packed into multiple socket-level buffers and even span buffers. The code that looked for delimiters and extracts the messages from these buffers is broken.
 
Here are the fixes I made, the comments are very helpful I think:
 
BaseSocketConnectionHost.cs:
 
#region ReadFromConnection
 

/// <summary>
/// receive bytes transmitted by BeginSend(). The buffer may contain a single message, or multiple
/// messages, and the end of the buffer may contain the first part of a message to be continued in
/// the next buffer to be received.
/// </summary>
/// <param name="connection"></param>
/// <param name="readBytes"></param>
private void ReadFromConnection(BaseSocketConnection connection, int readBytes)
{
 

bool onePacketFound = false;
int remainingBytes = 0;
SocketAsyncEventArgs e = connection.ReadOV;
 
// readBytes is the actual number of bytes read from the Socket into the buffer
 
// if a partial message was received previously and shifted to the beginning of the buffer, e.Offset
// would have been adjusted to point to the byte following that partial message. processBytes is
// the total length of buffered data assuming a starting offset of zero.
 
int processBytes = readBytes + e.Offset;
 
// call FireOnReceived() for each complete message contained in the buffer
 
switch (connection.DelimiterType)
{
case DelimiterType.dtNone:
//----- Message, one per buffer, no delimiter!
remainingBytes = ReadMessageWithNoDelimiter(connection, e, processBytes);
break;
 
case DelimiterType.dtMessageTailExcludeOnReceive:
case DelimiterType.dtMessageTailIncludeOnReceive:
//----- Message(s) packed into buffers, with tail delimiters!
remainingBytes = ReadMessageWithTail(connection, e, processBytes, ref onePacketFound);
break;
}
 
// remainingBytes is the number of unprocessed bytes from the buffer which should be saved
// for processing when the next buffer is received
 
if (remainingBytes == 0)
{
// the entire message has been read, no further messages are contained in the buffer
 
// reset buffer for next receipt of data
 
e.SetBuffer(0, e.Buffer.Length);
}
else
{
// messages which span buffers
 
// remainingBytes is the number of bytes at the end of the current buffer, to use with the next buffer
// to rebuild the message. Reallocate the socket buffer large enough to hold this unprocessed data
// PLUS the next incoming buffer. e.Count ensures that the next buffer will be stored immediately
// following the unprocessed data from the current buffer.
 
if (!onePacketFound)
{
// no message delimiter was found in the buffer - as in the case dtNone
// is it possible for a delimited message to span multiple buffers??? if so, this code probably won't work
 
// int nextoffset = e.Buffer.Length - remainingBytes;
int nextoffset = processBytes - remainingBytes;
 
// SetBuffer():
// The offset and count parameters can't be negative numbers.
// The combination of the offset and count parameters must be in bounds of the buffer array in the Buffer property.
// This method sets the Count property to the count parameter and the Offset property to the offset parameter.
 
try
{
e.SetBuffer(nextoffset, remainingBytes);
}
catch (Exception ex)
{
// WriteEventLog("ReadFromConnection", "!onePacketFound .SetBuffer exception !!!!!!!!!!!! " + ex.Message);
}
 
}
else
{
// a partial message remains in the buffer, to be continued in the next buffer
// save this message fragment to process together with the next buffer
// maximum size for one complete message is FMessageBufferSize
// messages span at most two buffers - start in the first buffer, are completed in the second
 
byte[] readMessage = connection.BaseHost.BufferManager.TakeBuffer(FMessageBufferSize);
 
// crash if e.Offset = 0x9ff + remainingBytes = 0x01
 
// it seems to occur if large messages are sent "fast" due to the buffering algorithm
// on the sending side.
 
// by catching this exception, the previous behaviour (trapping immediately OUT of this function
// and terminating the read) is prevented - since the value in e.Offset will prevent proper handling
// of future messages, probably the connection should be Disconnected at this point!
try
{
// copy remainder of message to the side buffer
// e.Offset points to the next unprocessed data byte
// remainingBytes is the number of unprocessed data bytes
Buffer.BlockCopy(e.Buffer, e.Offset, readMessage, 0, remainingBytes);
 
// point the socket buffer to the new buffer containing the beginning of a message
// so that the next socket buffer will be appended to it
 
try
{
// free the socket's original buffer
connection.BaseHost.BufferManager.ReturnBuffer(e.Buffer);
 
int remainingBytesToFill = readMessage.Length - remainingBytes;
 
e.SetBuffer(null, 0, 0);
e.SetBuffer(readMessage, remainingBytes, remainingBytesToFill);
}
catch (Exception ex)
{
// WriteEventLog("ReadFromConnection", "onePacketFound .SetBuffer exception !!!!!!!!!!!! " + ex.Message);
}
 
}
catch (Exception ex)
{
// WriteEventLog("ReadFromConnection", "onePacketFound BlockCopy exception !!!!!!!!!!!! " + ex.Message);
}
}
}
 
if (connection.Active)
{
 
// ====================================================================================
// this is where the next Socket read is requested
// ====================================================================================
 
//----- Read!
bool completedAsync = true;
 
if (connection.Stream != null)
{
connection.Stream.BeginRead(e.Buffer, 0, e.Count, new AsyncCallback(BeginReadCallbackSSL), connection);
}
else
{
// if the incoming message is longer than MessageBufferSize, e.BytesTransferred is zero
// but completedAsync returns true. Clients must never specify MessageBufferSize larger
// than the server.
 
completedAsync = connection.Socket.ReceiveAsync(e);
}
 
if (!completedAsync)
{
BeginReadCallbackAsync(this, e);
}
 
}
 
}
 
#endregion
 

 
#region ReadMessageWithTail
 
/// <summary>
/// extracts one or more messages from a buffer, and calls FireOnReceived() for each complete message found.
/// Partial messages are left for the next buffer received from the Socket
/// </summary>
/// <param name="connection"></param>
/// <param name="e">current descriptor and data buffer, containing one or more messages separated by delimiters</param>
/// <param name="dataBytes">number of valid data bytes in e.Buffer, beginning at offset=0</param>
/// <param name="MessageFound">output: TRUE if the at least one complete packet was completed and FireOnReceived() called.
/// If the return value is zero, it means there were no message completions found - otherwise it means that some of the bytes
/// at the end of the buffer are part of a future message and should be saved for next time. </param>
/// <returns>number of data bytes unprocessed in e.Buffer - which should be saved and prepended to the next data buffer.
/// If greater than zero, e.Offset points to the next unprocessed byte.</returns>
private int ReadMessageWithTail(BaseSocketConnection connection, SocketAsyncEventArgs e, int dataBytes, ref bool MessageFound)
{
// default return values
 
MessageFound = false;
int unprocessedBytes = dataBytes;
int messageOffset = 0; // offset of first byte in a message
int messageLength = 0; // message length including the delimiter
int delimiterOffset = 0; // offset of end-of-message delimiter in the buffer
 
while ((messageOffset < dataBytes) && (delimiterOffset >= 0))
{
// find all the messages in the buffer
// each series of message bytes is followed by the message delimeter
// if there is no delimiter following the last message identified, the unprocessed bytes are part of the next message in a future buffer
 
while ((delimiterOffset >= messageOffset) && (messageOffset < dataBytes))
{
// find the next complete end-of-message delimiter in the buffer
 
delimiterOffset = Array.IndexOf<byte>(e.Buffer, connection.Delimiter[0], delimiterOffset, unprocessedBytes); // zero-based, returns (-1) if not found
 
if (delimiterOffset < messageOffset)
break; // not found! Bytes from messageOffset to (dataBytes-1) remain unprocessed
 
messageLength = delimiterOffset - messageOffset + 1;
 
// match remaining characters of delimiter
// if the complete delimiter is not found here, it may occur later in the buffer
// so keep scanning until the entire buffer has been examined
// the entire delimiter must be contained in the buffer and not span to the next one !!
 
for (int i = 1; (i < connection.Delimiter.Length); i++)
{
if (((messageOffset + messageLength) >= dataBytes) || (e.Buffer[messageOffset + messageLength] != connection.Delimiter[i]))
{
messageLength = -1;
break;
}
 
messageLength++;
}
 
if (messageLength <= 0)
break;
 
// message plus a complete delimiter series was found
 
MessageFound = true;
unprocessedBytes -= messageLength;
 
// messageOffset points to the first byte of the message, which is messageLength bytes long including the delimiter
// delimiterOffset points to the first byte of the delimiter
 
// process the message buffer
 
byte[] messageBuffer = BufferUtils.GetRawBufferWithTail(connection, e, messageOffset, messageLength, connection.Delimiter.Length);
 
messageBuffer = CryptUtils.DecryptData(connection, messageBuffer, FMessageBufferSize);
 
FireOnReceived(connection, messageBuffer);
 
// advance pointers for next scan
 
messageOffset += messageLength;
delimiterOffset = messageOffset;
messageLength = 0;
 
} // end-while
} // end-while
 
if (unprocessedBytes > 0)
{
// from messageOffset to [dataBytes-1] are unprocessed
 
// sets the value for e.Offset so that the calling routine can know the offset to the next message
// which begins in this buffer

e.SetBuffer(messageOffset, unprocessedBytes);
}
 
return (unprocessedBytes);
}
 
#endregion
 

 

BufferUtils.cs:
 

 
#region GetRawBufferWithTail
 
/// <summary>
/// returns byte array containing the encoded message - assuming that the buffer contains the delimiter string,
/// e.Buffer contains the raw bytes, and e.Offset points to the first byte to be copied. There is no check for
/// trying to BlockCopy beyond the end of the e.Buffer
/// </summary>
/// <param name="connection"></param>
/// <param name="e"></param>
/// <param name="offset">offset at which to start copying</param>
/// <param name="length">number of bytes to copy, including the trailing delimiter</param>
/// <param name="delimiterSize">number of bytes of the delimiter</param>
/// <returns></returns>
public static byte[] GetRawBufferWithTail(BaseSocketConnection connection, SocketAsyncEventArgs e, int offset, int length, int delimiterSize)
{
 
//----- Get Raw Buffer with Tail!
byte[] result = null;
 
if (connection.DelimiterType == DelimiterType.dtMessageTailIncludeOnReceive)
{
// include the delimiter in the output buffer
 
result = new byte[length];
}
else
{
// exclude the delimiter from the output buffer
 
result = new byte[length - delimiterSize];
}
 

Buffer.BlockCopy(e.Buffer, offset, result, 0, result.Length);
 
return result;
 
}
 
#endregion
 
}
GeneralRe: IndexOutOfRangeException in ReadMessageWithTailmemberSindowZ6-Aug-09 23:12 
Thanks a lot for the code! It looks like it should work just fine but when I plug it into the three regions and run my program again OnReceived is not firing. It does detect the connection to the server. Were these changes made using the 1.5 or 2.0 version and did you make any other changes as to how the received messages are handled? I am currently using the latest version with a class based off of the SocketServer in the chat example.
GeneralRe: IndexOutOfRangeException in ReadMessageWithTailmembermikey2227-Aug-09 5:37 
I am using the Framework 3.5 version

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Permalink | Advertise | Privacy | Mobile
Web02 | 2.6.130617.1 | Last Updated 29 Apr 2009
Article Copyright 2006 by Andre Azevedo
Everything else Copyright © CodeProject, 1999-2013
Terms of Use
Layout: fixed | fluid