Click here to Skip to main content
Rate this: bad
good
Please Sign up or sign in to vote.
See more: VB.NET
Hello friends, i am in the middle of the dark now and i don't know what to do again. I am writing a proxy application that receives HTTP request from browser, forwards it to the server, read the response from server and send it back to the browser.
 
I wrote the program and everything works fine. But i discovered that whenever i receive a response from the server that has the "Transfer-Encoding: chunked" as part of the HTTP header response, the packet datas i read from server are incomplete. So, i do a google search to know what this chunked transfer encoding really means. I read many articles and i know what this chunked transfer really is. So, i create a sub to handle the reading of any header response that contains the "Transfer-Encoding: chunked" but still, my logic is not accurate enough because the packet datas i read from the server are still incomplete. So pls i need help on how i can read "CHUNKED" transfer encoding response from the server. Here is the header response i get from the server:
 
HTTP/1.1 200 OK
Date: Fri, 23 Nov 2012 08:45:52 GMT
Server: Apache
Transfer-Encoding: chunked
Content-Type: text/html
Connection: close
 
Here is the sub i created to handle the reading of any chunk response from the server:
 
Sub chunkResponseReader(ByVal incoming As TcpClient, ByVal outgoing As TcpClient, ByVal client As NetworkStream, ByVal server As NetworkStream)
        While True
            Try
                Dim res_hdr As String = ""
                Dim agw As Integer
                Dim php(0) As Byte
                Dim chunkSize As Integer
                Dim buffer() As Byte
                While True
                    agw = server.Read(php, 0, php.Length)
                    res_hdr = res_hdr & Encoding.ASCII.GetString(php, 0, agw)
                    If res_hdr.IndexOf(vbNewLine) <> -1 Then
                        Exit While
                    End If
                    If agw = 0 Then
                        outgoing.Close()
                        incoming.Close()
                        Exit Sub
                    End If
                    received += agw
                End While
                totalTransfer = sent + received
                agw = 0
                chunkSize = CInt(convertHex(res_hdr.Substring(0, res_hdr.IndexOf(vbNewLine))))
                Dim chunkRemain As Integer = 0
                'Dim chunkBuff As Integer
                ReDim buffer(chunkSize + 2)
                While True
                    agw = server.Read(buffer, 0, buffer.Length)
                    If agw = 0 Then
                        incoming.Close()
                        outgoing.Close()
                        Exit Sub
                    End If
                    client.Write(buffer, 0, agw)
                    received += agw
                    chunkRemain = (chunkSize + 2) - agw
                    chunkSize = chunkRemain
                    If chunkRemain <= 0 Then
                        Exit While
                    End If
                End While
                msg("Chunk :: DONE!")
            Catch ex As Exception
                msg("Chunk Response Reader Sub :: " & ex.Message)
            End Try
        End While
    End Sub
 
Here is my convertHex function:
 
 Private Function convertHex(ByVal hex As String) As Object
        Try
            Return Integer.Parse(hex, Globalization.NumberStyles.HexNumber)
        Catch ex As Exception
            Return False
        End Try
    End Function
 
Pls help
Posted 22-Nov-12 22:41pm
Edited 22-Nov-12 23:12pm
v4

1 solution

Rate this: bad
good
Please Sign up or sign in to vote.

Solution 1

@all i later get to solve this problem myself after careful studying and analyzing the chunked data. Here is my chunkResponseReader sub maybe it may be of great help to another person.
 
Sub chunkResponseReader(ByVal incoming As TcpClient, ByVal outgoing As TcpClient, ByVal client As NetworkStream, ByVal server As NetworkStream)
        While True
            Try
                Dim res_hdr As String = ""
                Dim agw As Integer
                Dim php(0) As Byte
                Dim chunkSize As Integer
                Dim buffer() As Byte
 
                '################################
                '#      READ CHUNK SIZE         #
                '################################
                While True
                    agw = server.Read(php, 0, php.Length)
                    res_hdr = res_hdr & Encoding.ASCII.GetString(php, 0, agw)
                    If res_hdr.IndexOf(vbCrLf) <> -1 Then
                        Exit While
                    End If
                    If agw = 0 Then
                        outgoing.Close()
                        incoming.Close()
                        Exit Sub
                    End If
                    received += agw
                End While
                totalTransfer = sent + received
                agw = 0
                chunkSize = CInt(convertHex(res_hdr.Substring(0, res_hdr.IndexOf(vbCrLf))))
 

                
 
                Dim chunkRemain As Integer = 0
                'Dim chunkBuff As Integer
                ReDim buffer(chunkSize - 1)
                Dim res_hd As String
                While True
                    agw = server.Read(buffer, 0, buffer.Length)
                    If agw = 0 Then
                        incoming.Close()
                        outgoing.Close()
                        Exit Sub
                    End If
                    client.Write(buffer, 0, agw)
                    res_hd = Encoding.ASCII.GetString(buffer, 0, agw)
 

                    
 
                    received += agw
                    chunkRemain = (chunkSize + 0) - agw
                    chunkSize = chunkRemain
 
                    If chunkRemain <= 0 Then
                        'read away the eof and CrLf
                        Dim chunkEOF As String = ""
                        Dim agws As Integer = 0
                        While True
                            agws = server.Read(php, 0, php.Length)
                            chunkEOF = chunkEOF & Encoding.ASCII.GetString(php, 0, agws)
                            If chunkEOF.IndexOf(vbCrLf) <> -1 Then
                                Exit While
                            End If
                            received += agws
                        End While
                        Exit While
                    Else
                        ReDim buffer(chunkRemain - 1)
                    End If
 
                End While
                msg("Chunk :: DONE!")
            Catch ex As Exception
                msg("Chunk Response Reader Sub :: " & ex.Message)
                incoming.Close()
                outgoing.Close()
                Exit While
            End Try
        End While
    End Sub
 
My convertHex function still remains the same.
Thanks
  Permalink  

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
0 OriginalGriff 515
1 Maciej Los 290
2 Richard MacCutchan 265
3 BillWoodruff 235
4 Suraj Sahoo | Coding Passion 175
0 OriginalGriff 8,804
1 Sergey Alexandrovich Kryukov 7,457
2 DamithSL 5,689
3 Maciej Los 5,279
4 Manas Bhardwaj 4,986


Advertise | Privacy | Mobile
Web03 | 2.8.1411028.1 | Last Updated 24 Nov 2012
Copyright © CodeProject, 1999-2014
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100