Click here to Skip to main content
13,148,211 members (57,410 online)
Rate this:
Please Sign up or sign in to vote.
See more:
I'm building a windows application that will connect to a website using private proxies and do stuff on that site. I want to use the new HTTPClient to do that. I have the next scenario for the login part:

I GET the login page. The login page set a cookie (I need to store that).
In the code of the login page I find some dynamic parameters that I scrape using Regex.
I create the POST url using the parameters that I scraped and an user and a password.
I use the cookie that the login page stored and I POST to the website (the created URL).
I receive the content of the response and if the login is done correctly I see in the code a redirect url (the http web request will not follow this redirect!) and acookie is stored.
I scrape the redirect url and I use the cookies stored with the POST and do a GET request to the redirect url. Another final cookie is stored (overwritten the last one)

I have tried to do this with httpwebrequest but after the POST I get the response and when I load that response in a browser I see "your browser has the cookies functionality turned off" so I obviously do not know how to store my cookies and use them with the next request then store the new cookies and use them with the last request ...

Now I try to do all over again with HTTPClient... does anyone can help me with an example using proxy (with proxy user and proxy password)for all 3 requests an with cookie functionality ??
Please HELP!!!

Edit: after some sleep and more research I have found out why I could not find any example for httpclient in httpclient is implemented only in C#, C++ and JS... so please help me with httpwebrequest (my initial approach )
Strange - in C# I have HTTPClient by default but now I managed to install it for using NuGet ... so the question remains: Does anyone know how to use it effectively?

This is my code - the version that return: "cookies functionality disabled, please enable cookies"

Imports System.Net
Imports System.Text
Imports System.IO
Imports System.Runtime.Serialization.Formatters.Binary
Imports System.Text.RegularExpressions
Imports System.Web
Namespace Functions
    Public Class Functions
        Public xuser As String
        Public xpassword As String
        Public xproxy As String
        Public xport As String
        Public xproxypassword As String
        Public xproxyuser As String
        Public xphonenumber As String
        Public logincookies As CookieContainer
        Dim UA As String = "Mozilla/5.0 (Windows NT 6.2; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0"

        Public Function login(ByVal user As String, ByVal password As String, ByVal proxy As String, ByVal port As String, ByVal proxyuser As String, ByVal proxypassword As String, ByVal phonenumber As String)
            xuser = user
            xpassword = password
            xproxy = proxy
            xport = port
            xproxypassword = proxypassword
            xproxyuser = proxyuser
            xphonenumber = phonenumber
            logincookies = New CookieContainer
            'start the login phase
            'initiate the GET procedure for the login page
            'set the login url
            Dim loginpage As String = "https://theloginpageURL"
            'the GET request
            Dim request As HttpWebRequest = DirectCast(WebRequest.Create(loginpage), HttpWebRequest)
            request.UserAgent = UA
            request.Referer = "http://thepagethatreferedmeURL"
            request.KeepAlive = True
            'set the cookies for the request to the public cookie container
            request.CookieContainer = New CookieContainer()
            request.CookieContainer = logincookies
            'get the response from the server
            Dim response As HttpWebResponse = DirectCast(request.GetResponse(), HttpWebResponse)
            'start the stream reader to read the response
            Dim reader As New StreamReader(response.GetResponseStream())
            Dim loginpagecontent As String = reader.ReadToEnd
            'read the login page and get the parameters
            Dim pageGetLogin As String = loginpagecontent
            'Get the parameters
            Dim param1 As String = "this is a parameter value"
            Dim param2 As Object
            Dim param2fin As String
            param2 = Regex.Match(pageGetLogin, "(?<=param2""\s* value="")[\w\W]*?(?="")")
            param2fin = param2.ToString
            Dim param3 As String = "this is a parameter value"
            Dim param4 As String = "this is a parameter value"
            'creating the POST request
            'declaring the encoding type
            Dim encoding As New UTF8Encoding
            'creating the post data
            'creating the Request
            Dim postLoginRequest As HttpWebRequest = DirectCast(WebRequest.Create("" & param1 & "¶m2=" & param2fin & "¶m3=" & param3 & "¶m4=" & param4 & "&Email=" & Email & "&Passwd=" & Passwd & "&signIn=" & signIn), HttpWebRequest)
            'postLoginRequest.CookieContainer = logincookies
            'postLoginRequest.CookieContainer = cookieContainer
            postLoginRequest.Method = "POST"
            postLoginRequest.KeepAlive = True
            postLoginRequest.AllowAutoRedirect = True
            postLoginRequest.MaximumAutomaticRedirections = 50
            postLoginRequest.ContentType = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
            postLoginRequest.Referer = "https://theloginpageURL"
            postLoginRequest.UserAgent = UA
            postLoginRequest.ContentLength = 0
            postLoginRequest.AutomaticDecompression = DecompressionMethods.GZip
            postLoginRequest.CookieContainer = New CookieContainer
            postLoginRequest.CookieContainer = logincookies
            'getting the response to the POST request
            Dim postLoginResponse As HttpWebResponse
            postLoginResponse = DirectCast(postLoginRequest.GetResponse(), HttpWebResponse)
            Dim postLoginReader As New StreamReader(postLoginResponse.GetResponseStream())
            Dim loginResponsePageContent As String = postLoginReader.ReadToEnd

            'show the parameters to check if they are find corectly - to comment out later
            MessageBox.Show(param1 & "--" & param2 & "--" & user & "--" & xport & "--" & xproxyuser & "--" & xproxypassword)

            'serialize the cookies to bin format and save them to the disk
            'set the name for the cookie file
            Dim filename As String = "d:/" & "cookie.dat"
            'delete the old file if it exists
            If (File.Exists(filename)) Then
            End If
            'serialize and write the file
            Dim stream As FileStream = File.Create(filename)
            Dim formatter As New BinaryFormatter()
            formatter.Serialize(stream, response.Cookies)

            'Return the functions values
            'Return loginpagecontent
            Return loginResponsePageContent
        End Function

    End Class
End Namespace

I have replaced all the URL's in the code and most of the commented code that I used in later trials. I removed the parameters also and I replaced them with similar ones (also Iformated the post url to match the real one in aspect)

I call this class from a button on a form and I supply the response variable to a web browser component just to see how the response look (I'm to tired to read the code in a text editor by now ) )

The cookie that is saved is the first cookie that the login page set so I know that the first cookie is saved in the container.
The GET part is working as it should and I receive the response and I parse it and obtain all the parameters, I can verify that I have them by showing them in a
The POST it seem to be ok but I receive the "cookies disabled" message on the response html...
Please show me where I,m doing things wrong and how to fix it...
I can't post the real URL's and the entire cod ... I think an experienced programmer will spot the mistakes from this code

Thank You!!

PS - this code does not have the next GET as I can't receive a good response from the POST so I can't find the redirect URL and make the next GET request
Posted 7-Jul-13 10:26am
Updated 8-Jul-13 0:45am

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
Top Experts
Last 24hrsThis month

Advertise | Privacy |
Web03 | 2.8.170924.2 | Last Updated 8 Jul 2013
Copyright © CodeProject, 1999-2017
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100