Hello community,
In advance, please excuse my lack of knowledge regarding the following topic, I have tried to familiarise myself with the topic using both google and the search function.
A friend of mine asked whether I could get data from a specific website for him, I answered "yes, I can try". I feel like I am pretty close and only need a final push. So let's go:
www.regelleistung.net -> this is the website in question
Menu item : Data for Control Reserve - in the following form he would normally specify what kind of data he is interested in and would then click the submit button .
Someone gave me the TAMPER information for the post request and I came up with the following code:
1. The request requires a jsession cookie which I retrieve + from TAMPER I saw that I require several other items of interest : __fp , _sourcePage and CSRFToken.
Because I do not know how to get those information without making an actual request - I just perform a testrequest and populate an array, which I will use for the actual request
BASEURL = @"https://www.regelleistung.net/ip/action/abrufwert";
public string[] ReceiveCookiesAndHiddenData()
{
string[] tempHidden = new string[4];
HttpWebRequest tempRequest = (HttpWebRequest)WebRequest.Create(BASEURL);
using (HttpWebResponse tempResponse = (HttpWebResponse)tempRequest.GetResponse())
{
HtmlDocument doc = new HtmlDocument();
doc.Load(tempResponse.GetResponseStream());
tempHidden[0] = doc.DocumentNode.SelectNodes("//input[@type='hidden' and @name='_sourcePage']")[0].Attributes["Value"].Value;
tempHidden[1] = doc.DocumentNode.SelectNodes("//input[@type='hidden' and @name='__fp']")[0].Attributes["Value"].Value;
tempHidden[2] = doc.DocumentNode.SelectNodes("//input[@type='hidden' and @name='CSRFToken']")[0].Attributes["Value"].Value;
tempHidden[3] = tempResponse.Headers["Set-Cookie"].Split(new Char[] { ';' })[0];
}
return tempHidden;
}
2. Now I have to make the actual request. which requires the following information (according to Tamper):
http://pl.vc/3f5z0[
^]
public void MakeActualRequest()
{
hiddenData = ReceiveCookiesAndHiddenData();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(BASEURL);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.UserAgent = "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:31.0) Gecko/20100101 Firefox/31.0";
request.Accept = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
request.Referer = "https://www.regelleistung.net/ip/action/abrufwert";
request.Headers.Add("Set-Cookie",hiddenData[3]);
NameValueCollection outgoingQueryString = HttpUtility.ParseQueryString(String.Empty);
outgoingQueryString.Add("von", "05.08.2014");
outgoingQueryString.Add("uenbld", "STXs440zwks=");
outgoingQueryString.Add("produkt", "MRL");
outgoingQueryString.Add("work", "anzeigen");
outgoingQueryString.Add("_sourcePage", hiddenData[0]);
outgoingQueryString.Add("__fp", hiddenData[1]);
outgoingQueryString.Add("CSRFToken", hiddenData[2]);
string postdata = outgoingQueryString.ToString();
byte[] data = Encoding.ASCII.GetBytes(postdata);
request.ContentLength = data.Length;
using(Stream requestStream = request.GetRequestStream())
{
requestStream.Write(data,0,data.Length);
}
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
using(StreamReader reader = new StreamReader(responseStream, Encoding.Default))
{
string content = reader.ReadToEnd();
}
}
I receive a WebException "Remote server error :(403)" in this line of code:
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
I would be thankful for help. I know this is like the 1000000 time such a question has been posted, but I read all those threads and still cannot figure it out.
Thank you all!