HTTPWebRequest Cookies and authorization tokens

See more:

VB

.NET

.NET4.5

cookie

, +

My question: **What exactly is causing the program's request to be denied while the IE request is accepted?**

Earlier today I was working on logging into E926 with an HTTPWebRequest and though all the other sites I tried worked just fine, I couldn't get this one working..



I changed the code having realized that the 'auth token' changed every time a new browser session begun, and I created code to strip the NEW auth token from the site, however I had noticed that the auth token seemed to behave strangely in its format.

Examples:

4x1ypV6//nLWFtwb4krjWWXXT8eI117ebUuPKk3+Tzc=  ''Original token
4x1ypV6%2F%2FnLWFtwb4krjWWXXT8eI117ebUuPKk3%2BTzc  ''After being sent through the header


fldoYzSDltOv7PuhgdyuxckCJvbP2b2OWjbWXtSXZMQ=  ''Original token
fldoYzSDltOv7PuhgdyuxckCJvbP2b2OWjbWXtSXZMQ  ''After being sent through the header


8Gd+s/uBDL0j4gC6Zc2oV23uvBxc+R6bn2ZBteMi0mg=  ''Original token
8Gd%2Bs%2FuBDL0j4gC6Zc2oV23uvBxc%2BR6bn2ZBteMi0mg  ''After being sent through the header

I imagine this is due to the encoding, but I wasn't sure what encoding language they may be using, the following links show the headers I get from an IE request, and my program's request.

IE: https://docs.google.com/document/d/1UPf2vzPF11Q-i_3xSkZmdN1dfqFZX4ydyAZhVeQlh94/edit?usp=sharing

Program: https://docs.google.com/document/d/1ltZo3JZLt4sZiwZBltfqqParb3H86qi62I0KZN5t_VE/edit?usp=sharing

Finally, this is the actual code I am using:

Sub Client()
    Dim webRequest As HttpWebRequest
    Dim responseReader As StreamReader
    Dim responseData As String
    Dim postData As String = "%3D&url=&user%5Bname%5D=USERNAME&user%5Bpassword%5D=PASSWORD&user%5Broaming%5D=0"
    Dim cookies As CookieContainer = New CookieContainer()
    Dim requestWriter As StreamWriter
    Dim strUrl As String = Nothing
    Dim strRequestedHTML As String


    'get login  page with cookies
    strUrl = "https://e926.net/user/login"
    webRequest = HttpWebRequest.Create(strUrl)
    webRequest.Headers.Add("DNT", "1")
    webRequest.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; ru; rv:1.9.2.3) Gecko/20100401 Firefox/4.0 (.NET CLR 3.5.30729)"
    webRequest.AllowAutoRedirect = True
    webRequest.CookieContainer = cookies

    responseReader = New StreamReader(webRequest.GetResponse.GetResponseStream())
    responseData = responseReader.ReadToEnd()
    responseReader.Close()
    strRequestedHTML = responseData

    Dim doc As New HtmlDocument()
    doc.LoadHtml(strRequestedHTML)

    For Each node As HtmlNode In htmlparse.DocumentNode.SelectNodes("//input")
        Dim className = node.GetAttributeValue("name", "")
        If className.Contains("authenticity_token") Then
            Dim test As String = node.Attributes("value").Value
            test = test.Replace("=", "")
            postData = "authenticity_token=" & test & postData
        End If
    Next


    My.Computer.Clipboard.SetText(strRequestedHTML)
    ''MsgBox(postData) ''For debugging purposes

    'recieve non-authenticated cookie
    webRequest.GetResponse().Close()


    'post form  data to page
    strUrl = "https://e926.net/user/authenticate"
    webRequest = HttpWebRequest.Create(strUrl)
    webRequest.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; ru; rv:1.9.2.3) Gecko/20100401 Firefox/4.0 (.NET CLR 3.5.30729)"
    webRequest.AllowAutoRedirect = True
    webRequest.Method = WebRequestMethods.Http.Post
    webRequest.ContentType = "application/x-www-form-urlencoded"
    webRequest.CookieContainer = cookies
    webRequest.ContentLength = postData.Length

    requestWriter = New StreamWriter(webRequest.GetRequestStream)
    requestWriter.Write(postData)
    requestWriter.Close()

    'recieve authenticated cookie
    webRequest.GetResponse().Close()

    'now we get the authenticated page
    webRequest = HttpWebRequest.Create("https://e926.net/post")
    webRequest.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; ru; rv:1.9.2.3) Gecko/20100401 Firefox/4.0 (.NET CLR 3.5.30729)"
    webRequest.CookieContainer = cookies
    responseReader = New StreamReader(webRequest.GetResponse.GetResponseStream())
    responseData = responseReader.ReadToEnd()
    responseReader.Close()
    strRequestedHTML = responseData
    My.Computer.Clipboard.SetText(strRequestedHTML)


End Sub