Click here to Skip to main content
Rate this: bad
good
Please Sign up or sign in to vote.
I have a Client - Server solution that uses TCP to communicate. Everything was just fine until some client try to connect throe VPN. After debugging a similar condition I have notice that Client doesn’t get all sended bytes. For example:
Sending (packet index – packet number – first byte in packet):
111
222
333
444
Receiving:
111 
222
3144634534521230
433 
544 
My question is: how can I remove this garbage packages? And where they come from?
Here a simple code for sending and receiving bytes:
 
Sending Function:
    Public Function SendMessage(ByVal Client As TcpClient, ByVal Type As MessagesType, Optional ByVal Message As Object = "", Optional ByVal ID As ULong = 0) As Boolean
        If Client.Connected Then
            Dim networkStream As NetworkStream = Client.GetStream()
            Try
                Dim Start As Date = Now
                Do
                    If networkStream.CanWrite Then
                        Dim sendBytes As [Byte]()
                        If Type = MessagesType.SendFile And TypeOf Message Is FileNewSending Then
                            Dim mess As FileNewSending = Message
                            Dim fs As New FileStream(mess.FullPath, FileMode.Open)
                            Dim objReader As New BinaryReader(fs)
                            Dim sendFile As [Byte]() = objReader.ReadBytes(fs.Length)
                            fs.Close()
                            sendBytes = ObjectToByteArray(New TCPData(Type, New FileSending(mess.TargetDir, mess.Name, System.IO.Path.GetExtension(mess.FullPath), sendFile), ID))
                        Else
                            sendBytes = ObjectToByteArray(New TCPData(Type, Message, ID))
                        End If
                        Dim writer As New BinaryWriter(networkStream, Encoding.BigEndianUnicode)
                        Dim size As UInt64 = sendBytes.Length
                        writer.Write(size)
                        Dim packetIndex As UInt64 = 0
                        For i As Integer = 0 To sendBytes.Length - 1 Step Client.SendBufferSize
                            writer.Write(packetIndex)
                            If size < i + Client.SendBufferSize Then
                                writer.Write(sendBytes, i, size - i)
                            Else
                                writer.Write(sendBytes, i, Client.SendBufferSize)
                            End If
                            packetIndex += 1
                        Next i
                        writer.Flush()
                        Return True
                    End If
                Loop While (Now - Start).TotalSeconds < WaitForResponceTimeOut
                Return False
            Catch ex As IOException
                Return False
            Catch ex As Exception
                Log("Network Error", ex.ToString)
                Return False
            End Try
        End If
        Return False
    End Function
 
Receiving function:
Public Function ReciveMessage(ByVal Client As TcpClient, ByRef Type As MessagesType, ByRef Message As Object, ByRef ID As ULong) As Boolean
        If Client.Connected Then
            Try
                Dim networkStream As NetworkStream = Client.GetStream()
                If networkStream.DataAvailable Then
                    Dim bytes() As Byte = Nothing
                    Dim ReciveBufferLength As UInt64 = 0
                    Dim TotalReciaveLength As UInt64 = 0
                    Dim readStarted As Date = Now
                    Dim lIndex As UInt64 = 0
                    Dim packetIndex As UInt64 = 0
                    Do
                        If networkStream.DataAvailable Then
                            readStarted = Now
                            Dim reader As New BinaryReader(networkStream, Encoding.BigEndianUnicode)
                            If TotalReciaveLength = 0 Then
                                ReciveBufferLength = reader.ReadUInt64()
                                lIndex = reader.ReadUInt64()
                                ReDim bytes(ReciveBufferLength - 1)
                                If ReciveBufferLength > Client.ReceiveBufferSize Then
                                    TotalReciaveLength = Client.ReceiveBufferSize
                                Else
                                    TotalReciaveLength = ReciveBufferLength
                                End If
                                reader.Read(bytes, 0, TotalReciaveLength)
                                packetIndex += 1
                            Else
                                lIndex = reader.ReadUInt64()
                                If packetIndex = lIndex Then
                                    If ReciveBufferLength > TotalReciaveLength + Client.ReceiveBufferSize Then
                                        reader.Read(bytes, TotalReciaveLength, Client.ReceiveBufferSize)
                                        TotalReciaveLength += Client.ReceiveBufferSize
                                    Else
                                        reader.Read(bytes, TotalReciaveLength, ReciveBufferLength - TotalReciaveLength)
                                        TotalReciaveLength = ReciveBufferLength
                                    End If
                                    packetIndex += 1
                                End If
                            End If
                        Else
                            System.Threading.Thread.Sleep(100)
                        End If
                    Loop While (networkStream.DataAvailable And ReciveBufferLength = 0) Or (ReciveBufferLength > 0 And TotalReciaveLength < ReciveBufferLength And (Now - readStarted).Seconds < WaitForReciaveTimeOut)
                    Dim msg As TCPData = ByteArrayToObject(bytes)
                    If msg Is Nothing Then
                        Return False
                    Else
                        Type = msg.Type
                        Message = msg.Data
                        ID = msg.ID
                        Return True
                    End If
                End If
            Catch ex As OverflowException
                Return False
            Catch ex As Exception
                Log("Network Error", ex.ToString)
                Return False
            End Try
        End If
        Return False
    End Function
 
Client initialization:
Client = New TcpClient(ConnectIP, ListnerPort)
                Client.SendBufferSize = 1024 * 2
                Client.ReceiveBufferSize = 1024 * 2
                Client.NoDelay = False
 
To be clear - I use low level communication. "Sending" function convert all "messages" to byte array thrue:
ObjectToByteArray(New TCPData(Type, Message, ID))
TCPData is a simple serializable class:
<Serializable()> Public Class TCPData
    Public Property ID As ULong
    Public Property Type As Integer
    Public Property Data As Object
    Sub New()
    End Sub
    Sub New(ByVal _Type As Integer, ByVal _Data As Object)
        Type = _Type
        Data = _Data
        ID = 0
    End Sub
    Sub New(ByVal _Type As Integer, ByVal _Data As Object, ByVal _ID As ULong)
        ID = _ID
        Type = _Type
        Data = _Data
    End Sub
End Class
And function "ObjectToByteArray":
Public Function ObjectToByteArray(ByVal _Object As Object, Optional ByVal Compression As Boolean = True) As Byte()
    Try
        Dim _MemoryStream As New MemoryStream()
        Dim _BinaryFormatter As New System.Runtime.Serialization.Formatters.Binary.BinaryFormatter()
        _BinaryFormatter.AssemblyFormat = Formatters.FormatterAssemblyStyle.Simple
        _BinaryFormatter.Serialize(_MemoryStream, _Object)
        If UseCompress And Compression Then
            Return Compress(_MemoryStream.ToArray())
        Else
            Return _MemoryStream.ToArray()
        End If
    Catch _Exception As Exception
        Console.WriteLine("Exception caught in process: {0}", _Exception.ToString())
    End Try
    Return Nothing
End Function
Compress function just "zip" a byte array:
Private Function Compress(ByVal Data() As Byte) As Byte()
    Dim msCompressed As New MemoryStream()
    Dim size As UInt64 = Data.Length
    Dim writer As New BinaryWriter(msCompressed, System.Text.Encoding.ASCII)
    writer.Write(size)
    Dim zosCompressed As New GZipOutputStream(msCompressed)
    zosCompressed.Write(Data, 0, Data.Length)
    zosCompressed.Close()
    Return msCompressed.ToArray()
End Function
At the receiving end the process is reversed.
I have tried many ways to send bytes:
1. Write whole byte array
writer.Write(sendBytes, 0, sendBytes.length)
2. Tried to adjust SendBufferSize/ClientBufferSize
But nothing helped me.
I look forward to your further advice.
Posted 8-Apr-11 0:44am
Edited 10-Apr-11 20:20pm
v2

1 solution

Rate this: bad
good
Please Sign up or sign in to vote.

Solution 1

I think you did not get fast response because your problem looks more difficult then an average one. Let's start…
 
It may be a shift in message boundaries. You're trying to use too low-level approach (not really gaining performance) where you don't have distinct message boundaries (message in the sense of your application protocol) and calculate the message boundary dynamically using the condition I see in the loop. It's too easy to make a mistake here. You also use pessimistic and defensive style of coding where optimistic and offensive style is much better (but then you have to deal with exceptions which is in fact much better: exception instead of silent error).
 
Another suspect is the loop condition using WaitForResponceTimeOut. It resembles spin cycle. You should prefer explicit timeout where you need it. Again, I'm not trying to analyze all your logic which would take too much time and would be pointless. Instead, you should change your approach at least slightly. See below.
 
So, how to overcome the problem? I refuse the idea to fix it on detailed level. To resolve it, you really need to use one or two steps to higher level. Let's start with the level of TpcListener/TcpClient. This level is fine. Instead of writing/reading to/from network string, work at the level of messages. Look at what you call a packet on top of your Question. Make is a Packet class and make is serializeable. Use binary serializer for performance. Serialize to network stream, deserialize from the network stream. This approach is optimistic. If you have a problem, you will get an exception. Explicit thing, no silent error. You will need to use the class System.Runtime.Serialization.Formatters.Binary.BinaryFormatter. If you're not yet well familiar with serialization, start here: http://msdn.microsoft.com/en-us/library/ms233843.aspx[^].
 
If you need a higher level, you can use classic remoting or self-hosted WCF foundation (with Data Contract). If can be slower. Check up if the performance is good enough for you.
 
Just in case, check my past Answers about using threading with networking:
Multple clients from same port Number[^].
Not all my points are relevant to your project, but many are.
 
Your follow-up Questions are welcome.
 
Good luck,
—SA
  Permalink  
v2
Comments
Espen Harlinn at 10-Apr-11 6:49am
   
Good advice, my 5 :)
SAKryukov at 10-Apr-11 20:31pm
   
Thank you, Espen.
--SA
Alexander Chubarev at 11-Apr-11 1:28am
   
I debug it in one machine and on two. I see a problems if both sides run on the same computer when sending a big message (greater then 4Mb).
I will try create the binary formatter once in the run-time and reuse today and run debug with recording to System.Log. Then post a result.
Alexander Chubarev at 11-Apr-11 11:54am
   
Finaly I found a solution:
 
It turned out that ReceiveBuffer not always was fully loaded. Sometimes it was 1024/1024 bytes, but sometimes not (occur most frequently when transferring large messages). So I implement variable LocalReciaveLength that reads number of bytes actually being sent.
 
thank you for your help. Can you tell me how to close this topic correctly?
SAKryukov at 11-Apr-11 14:27pm
   
Great.
If you think my Answer is helpful, formally accept it by pressing "Accept" button.
Kindly add a comment to this comment to notify me when and if you've finally closed the issue.
--SA
Alexander Chubarev at 12-Apr-11 6:04am
   
Thank you. Issue is closed
SAKryukov at 12-Apr-11 15:39pm
   
Great. Thank you for accepting this Answer and good luck.
Call again,
--SA

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
0 OriginalGriff 230
1 PIEBALDconsult 150
2 DamithSL 125
3 BillWoodruff 108
4 Garth J Lancaster 90
0 OriginalGriff 5,790
1 DamithSL 4,601
2 Maciej Los 4,012
3 Kornfeld Eliyahu Peter 3,480
4 Sergey Alexandrovich Kryukov 3,195


Advertise | Privacy | Mobile
Web01 | 2.8.141220.1 | Last Updated 11 Apr 2011
Copyright © CodeProject, 1999-2014
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100