Click here to Skip to main content
15,867,686 members
Articles / Programming Languages / C#
Article

HTTPeep - an HTTP inspector

Rate me:
Please Sign up or sign in to vote.
4.87/5 (41 votes)
14 Mar 20055 min read 135.4K   1.8K   121   35
Handy C# utility to perform custom HTTP requests and view the raw HTTP response.

Screenshot

Introduction

This tool allows you to inspect (or "peep at") the RAW HTTP response of a URL. You can build the request using the settings form (including POST variables, user agent etc.). I initially made this tool a couple of years ago, and it continued to be extremely useful, so I improved it over time, and resolved the main bugs. This app is a work in progress, but if I wait till I'm "finished", this code will never get posted - you know how it goes with side-projects. I hope that in the mean time people will find this useful as is.

The Interface

The user interface is fairly self-explanatory - specify the URL ("http://" is optional), and optionally the Request Settings. Available settings are:

  • User agent
  • Referer
  • Range (i.e., byte range - if you only want to retrieve a partial file. Note: this only works if the web server supports it - which most do). E.g.: 0 - 29 will download the first 30 bytes of a file.
  • Option to use HTTP 1.0 instead of the default HTTP 1.1
  • POST variables - manually add key/value entries, or specify RAW post string. (The latter is extremely useful for sending XML to a .NET web service, shown in the screenshot above.)
  • Custom headers - specify one per line using standard HTTP syntax (e.g., "Set-Cookie: one=blah").
  • Accept (i.e., content-types - most browsers send this so the server knows what content-types the client accepts).
  • Follow redirects - if not checked then HTTP redirect responses will not be followed, and you will see the actual redirect response. (Note: if it is checked and a redirect occurs, a message window will be displayed that shows you the end URL that responded.)

Once the response has been fully downloaded, HTTPeep will show you:

  • Response HTTP status code & description.
  • HTTP raw headers - right click to copy or save.
  • Response content - right click to copy or save. If the response content-type is text, it will display here unless it is longer than the MaxLength property of the TextBox (currently set to 100,000 characters). So you will need to right-click and save to a file to view the content. If the response content-type is non-text, it also won't display here, and again you can right-click to save the content as a file.
  • Bytes downloaded / time / average bytes per second in the status bar.

Edit POST variables

Screenshot: edit POST variables

By default, the Raw checkbox is unchecked and you can add POST variables one by one. To edit an existing item, just double click it. To remove an item, select it and hit Delete key. You can save or load the POST variables to a text file. The format of the text file is intentionally simple so that you can easily create one using a text editor:

[User]
one
[Pass]
two
[Key]
value value
value

value

(Note: I haven't provided for an escape of "[" - so will be a problem if your POST variable starts with this.)

Internals

From a technical point of view, there is nothing fancy under the hood. I am simply configuring the properties of a System.Net.HttpWebRequest object, and calling the GetResponse() method to return a HttpWebResponse object. Note that a CookieContainer is set, and cookies are retained while the app remains open.

The response content is saved to a unique temporary file:

C#
TmpFilename = Path.GetTempPath() + "http_content" + requestStart.Ticks + ".tmp";

The code to download the content stream from the HttpWebResponse object into a file looks like this:

C#
Stream rcvStream = resp.GetResponseStream();
byte[] respBytes = new byte[256];
int byteCount;

FileStream fs = new FileStream(TmpFilename, FileMode.Create, FileAccess.Write);
do
{
   byteCount = rcvStream.Read(respBytes, 0, 256);
   fs.Write(respBytes, 0, byteCount);
} while (byteCount > 0);

HTTPeep.exe.config

The appSettings keys in the .config file are self-explanatory. All but the first affect the Settings form:

  • DefaultURL (for the main form)
  • DefaultUserAgent
  • DefaultAccept
  • DefaultReferer
  • DefaultCustomHeaders
  • UserAgentMenu - pipe-delimited list of values (name|string|name|string...). The current UA strings are IE 6, Firefox 1.0, Nokia 6230:

    Screenshot: select user agent

The annoying TextBox BEEP!

Even though Ctrl-C, Ctrl-V, Ctrl-X work by to copy, paste, cut on the standard .NET TextBox control, Ctrl-A does not select all as you might expect. Instead you get an annoying beep.

So you'd expect a simple solution would work - i.e., code a KeyDown or KeyUp event handler and set e.Handled to true so Windows knows you've handled the key-combination. Hmmm... that doesn't work. Guess what? Only setting e.Handled in the KeyPress event handler prevents this beep. Unfortunately, KeyPressEventArgs doesn't give you all the additional info that KeyEventArgs does so forget putting your code in the KeyPress event handler. But I wouldn't bother writing all this if I didn't have a solution: just set a flag in the KeyDown event handler and set e.Handled in your KeyPress event handler (which is called after KeyDown) if the flag is set! Code as follows:

C#
private void TxtUrl_KeyDown(object sender, System.Windows.Forms.KeyEventArgs e)
{
  keyHandled = false;

  if (e.KeyValue == 13)
  {
    BtnGo.PerformClick();
    keyHandled = true;
  }
  else if (e.Control && e.KeyCode == Keys.A)
  {
    ((TextBox)sender).SelectAll();
    ((TextBox)sender).Focus();
    keyHandled = true;
  }
}

private void TxtUrl_KeyPress(object sender, System.Windows.Forms.KeyPressEventArgs e)
{
  if (keyHandled)
    e.Handled = true;
}

Sending XML to a .NET webservice

To get an XML response from a webservice, you can use HTTPeep to POST some XML to your .asmx URL:

  1. In your web browser, go to your .asmx URL.
  2. Click on one of the available operations.
  3. From the SOAP section, copy the SOAPAction line and paste that into the "Custom Headers" textbox on HTTPeep's settings form.
  4. From the SOAP section, copy the XML for the request, modify the appropriate values, and paste that into the Raw POST textbox on HTTPeep's settings form (make sure "Raw" is checked).
  5. Paste the .asmx URL into HTTPeep's main form, and click GO. You should then get an XML response (as shown in the main screenshot at the top).

Future improvements / ideas

I have no idea when I'll get around to doing any of these:

  • Perform the downloading in a separate thread and provide a Cancel option.
  • Redo the user interface - break it up using tabs.
  • Don't show "actual returned URL" in a dialog.
  • Change default settings from within the app.
  • Provide interface to view the contents of the CookieContainer object.
  • Keep a history of previous URLs.
  • Provide a way to parse stuff out of the returned HTML - e.g., all form fields, all images etc.
  • Let you do a HTTP HEAD request.
  • Let you run a regular expression on the response content.
  • Provide a way to generate the C# code to make that request.

Feel free to make changes to suit your own requirements.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


Written By
Australia Australia
Ash is a C# developer (MCAD) with a background developing e-commerce and content management solutions. His current role includes working with VOIP systems, integration and maintenance of business and billing apps. His personal projects include the ScrollingGrid web control to enable cross-browser freeze-header 2-way scrolling of DataGrids. His other interests include travel, cinema, Squash, photography, Muay Thai.

Comments and Discussions

 
GeneralError: ConnectFailure Pin
Lex Blagus23-Jun-10 16:15
Lex Blagus23-Jun-10 16:15 
GeneralRe: Error: ConnectFailure Pin
Lex Blagus25-Jun-10 5:36
Lex Blagus25-Jun-10 5:36 
QuestionTime out issue Pin
vinothkumarvj13-Nov-09 4:35
vinothkumarvj13-Nov-09 4:35 
GeneralThx dude Pin
sharpalloy22-Jan-09 2:14
sharpalloy22-Jan-09 2:14 
GeneralProxy usage Pin
fgblanch11-Mar-08 4:35
fgblanch11-Mar-08 4:35 
Questionhow about captive portal? Pin
Aung Sithu27-Aug-07 6:06
Aung Sithu27-Aug-07 6:06 
Generalhave problem Pin
TAREQ F ABUZUHRI30-May-07 5:33
TAREQ F ABUZUHRI30-May-07 5:33 
hello

i use httpwebrequest & httpwebresponse in this page

http://www.codeproject.com/cs/internet/httpeep.asp


i test this code in many site it work verrry good
but when i test this code with this site

https://isp.paltel.net/login.cfm

i have problem ???

please help me plzzzzzz

where the problem and how can solve this problem ????

---
the see this error message

--------------------------------------------------------------------
TrustFailure - The underlying connection was colsed : could not estabilsh trust relashionship for SSL/TLS secure channel
--------------------------------------------------------------------


Palestine

Generalthoroughly impressed Pin
jadedgeek4-May-07 0:11
jadedgeek4-May-07 0:11 
GeneralRe: thoroughly impressed Pin
Ashley van Gerven4-May-07 1:41
Ashley van Gerven4-May-07 1:41 
Generalcrawler Pin
rajmohan8315-Apr-07 20:47
rajmohan8315-Apr-07 20:47 
Questiontime Pin
madaiya16-Feb-07 6:28
madaiya16-Feb-07 6:28 
GeneralWow, good work... Pin
Ralf Heyde8-Dec-06 8:39
Ralf Heyde8-Dec-06 8:39 
GeneralSweet dude, this rocks Pin
asdf123sad123asdasd123123123asds123123asd21-Nov-06 16:35
asdf123sad123asdasd123123123asds123123asd21-Nov-06 16:35 
GeneralHTTP Headers Pin
sabbaghov007027-Apr-06 21:17
sabbaghov007027-Apr-06 21:17 
GeneralSet username and password Pin
adelheid9-Mar-06 4:53
adelheid9-Mar-06 4:53 
GeneralRe: Set username and password Pin
Ashley van Gerven10-Mar-06 4:07
Ashley van Gerven10-Mar-06 4:07 
Questionwhat additional info? Pin
Anonymous19-Sep-05 18:31
Anonymous19-Sep-05 18:31 
AnswerRe: what additional info? Pin
Ashley van Gerven25-Feb-06 9:05
Ashley van Gerven25-Feb-06 9:05 
Generalhttps, SSL, Certificates Pin
Elfman_NE6-Sep-05 8:38
Elfman_NE6-Sep-05 8:38 
GeneralRe: https, SSL, Certificates Pin
Ashley van Gerven11-Sep-05 1:15
Ashley van Gerven11-Sep-05 1:15 
Generalpls. help me Proxy Error!! Pin
rmu90922-Aug-05 5:32
rmu90922-Aug-05 5:32 
GeneralRe: pls. help me Proxy Error!! Pin
Ashley van Gerven22-Aug-05 14:06
Ashley van Gerven22-Aug-05 14:06 
GeneralRe: pls. help me Proxy Error!! Pin
rmu90925-Aug-05 5:16
rmu90925-Aug-05 5:16 
Generalquestions... help plz Pin
enjoycrack19-Jul-05 22:59
enjoycrack19-Jul-05 22:59 
GeneralRe: questions... help plz Pin
Ashley van Gerven20-Jul-05 14:20
Ashley van Gerven20-Jul-05 14:20 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.