Click here to Skip to main content
Click here to Skip to main content
Technical Blog

Tagged as

Programmatically make(fake ;)) HTTP Requests using WebRequest, WebResponse and WebClient

, 17 Jul 2014 CPOL
Rate this:
Please Sign up or sign in to vote.
Its without a doubt that Microsoft .NET provides us with wonderful APIs to interact with a variety of interfaces/entities. One such API/library exists in the System.Net namespace. WebRequest and WebResponse are two such libraries which help us in programatically making HTTP Requests and analyse the

It's without a doubt that Microsoft .NET provides us with wonderful APIs to interact with a variety of interfaces/entities.
One such API/library exists in the System.Net namespace. WebRequest and WebResponse are two such libraries which help us in programatically making HTTP Requests and analyse the Responses in the code itself.

Its actually pretty straightforweard making GET and POST Requests and analysing the corresponding responses from C# code.

I’ll try to discuss the most common methods to accomplish the same.

HTTP GET Request

An HTTP Get Request is simple to make using WebRequest class. An example of one of the requests is as under :

public static string HttpGet(string URI)

{

System.Net.WebRequest req = System.Net.WebRequest.Create(URI);

//req.Proxy = new System.Net.WebProxy(ProxyString, true); //true means no proxy

System.Net.WebResponse resp = req.GetResponse();

System.IO.StreamReadersr = new System.IO.StreamReader(resp.GetResponseStream());

return sr.ReadToEnd().Trim();

}

Just passing the URL for the HTTP Request returns the response in string format. Ok, that was simple; now lets make an HTTP Post call from c#.

HTTP POST Request

Making and HTTPPost request is pretty straight forward, if you know what an “HTTP Post” is. An HTTP Post is basically any request which posts some form data (aggregated from User input).

Usually this form data is sent as a concatenated string of name=value& pairs.

Apart from the Form Data which is sent as part of the HTTP request body, there are other parameters as well which make up a POST request, such as cookies and HTTP header data.

The components which make an HTTP request can be easily identified using tools such as Fiddler. In case you don’t know, fiddler is a web debugging software which can be used to intercept and analyze HTTP traffic originating from our system.

I’ll cover a few details on how to use Fiddler in the last section of this Post. But I guess the below function would hold you in good stead to get you started.

public static string HttpPost(string URI, string Parameters)
{
System.Net.WebRequest req = System.Net.WebRequest.Create(URI);
  //req.Proxy = new System.Net.WebProxy(ProxyString, true);
//Add these, as we’re doing a POST
req.ContentType = “application/x-www-form-urlencoded”;
req.Method = “POST“;
//req.Timeout = 200000;

/*This is optional*/

(req as HttpWebRequest).Referer = @”https://someURL/”;
(req as HttpWebRequest).UserAgent = @”Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.76 Safari/537.36″;
(req as HttpWebRequest).Headers.Add(“Origin”, @”https://someURL/”);

/*This is optional*/

//We need to count how many bytes we’re sending. Post’ed Faked Forms should be name=value&
byte[] bytes = System.Text.Encoding.ASCII.GetBytes(Parameters);
req.ContentLength = bytes.Length;
System.IO.Stream os = req.GetRequestStream();
os.Write(bytes, 0, bytes.Length); //Push it out there
os.Close();
System.Net.WebResponse resp;
resp = req.GetResponse();
System.IO.StreamReader sr = new System.IO.StreamReader(resp.GetResponseStream());
return sr.ReadToEnd().Trim();
}
Download File Request

Sometimes we might just want to download a file from a URL. This usually a simple GET request and involves use of another function of the library System.Net. Below is a simple implementation of the same.

WebClientclient = new WebClient();

client.DownloadFile(URL, Path.Combine(&ldquo;<directory Path>&rdquo;, &ldquo;FileName.<extension>&rdquo;));

The above is a very simple implementation of a file download scenario. We can also make an asynchronous file download request using the above method, which might be a more practical scenario.

HTTP Response Analysis

Fiddler is an excellent utility for analysis HTTP traffic originating from your system. As I have always been saying, its always a good idea to get under the layer of abstraction MS has blinded us with and actually be able to see how the “web” works. You’ll be able to fully exploit the libraries provided by MS in System,Net only if you have a clear understanding of the HTTP traffic.

I’ll try to introduce (strictly for newbies :P) you to a very handy tool (fiddler) for accomplishing the same. (I know it may sound like Hand-holding to some, BUT hey, who doesn’t need it every now and then.)

First, download and install Fiddler from the link Fiddler

Open up Fiddler. You’ll be presented with a screen as below :

B1

The Left pane is where you’ll be able to see the overview all the HTTP Traffic. The Right pane shows the details.

Lets, open up http://chinmoymohanty.com on our browser. On inspecting in Fiddler, you’ll find the below output.

B2

The Left pane shows all the HTTP Requests which might have been made when we hit the website URL. Clicking on one of the rows brings out the Request (top-right pane) and Response (bottom-right pane). Clicking on the “Raw” tab would show the actual HTTP Request/Response (You might need to decode the response if it was encoded; don’t worry fiddler will alert you about that).

B3

As is quite easy to understand here, using data from the Fiddler analysis, you could easily construct/fake and HTTP Post/Get call.Beware, in some cases the server might be expecting encoded data, so it might not work in some rare scenarios.

Hope, this post was informative enough and would provide the needed kick for beginner programmers to code/automate HTP calls.

Please shout your views in the comments section.

The post Programmatically make(fake Wink | ;) ) HTTP Requests using WebRequest, WebResponse and WebClient appeared first on Chinmoy Mohanty's Blog.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Chinmoy Mohanty
Software Developer
India India
I am a Developer working across .NET Technologies.
Passionate about WEB technologies more than anything.
Fascinated about the idea of how the WEB world and the WinForm world are slowly merging into each other, with a very thin line to differentiate them.
 
I am an active blogger and I blog mostly about Technology. You can find my BLOG here.
 
Also, I am active on various techical forums like StackOverflow

Additionally, You can find my Presentations on SlideShare
Follow on   Google+   LinkedIn

Comments and Discussions

 
-- There are no messages in this forum --
| Advertise | Privacy | Mobile
Web02 | 2.8.141022.1 | Last Updated 17 Jul 2014
Article Copyright 2014 by Chinmoy Mohanty
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid