Click here to Skip to main content
Click here to Skip to main content

Socket-based HTTP Client with Bandwidth Limit

, 10 Aug 2010
Rate this:
Please Sign up or sign in to vote.
A simple HTTP client implementation based on sockets with ability to limit upload/download speed

Introduction

Communication over HTTP is a very common task. Most of the .NET programmers are familiar with classes like WebRequest, WebResponse or WebClient which are very useful, but sometimes one needs features they don't offer. In these cases, we could not have another option but to write our own HTTP client implementation based on TCP sockets. This article is an example of how to do so. The reader should know the basics of HTTP protocol.

Background

My first reason for writing my own HTTP client library was the need to upload large files to the server and show the current upload speed. It didn't seem to be very difficult, but I found out standard classes load all output data into memory before send despite the face that the WebRequest class offers an output stream for POST data. HTTP protocol is not really complex, so why not just open a socket on port 80 and start communication?

I didn't want to rewrite all my application but just to do some experiments first to ensure my idea is sustainable. I needed to create an abstract interface, which enables me to return to standard solution at any time. These are my classes:

  • HttpConnection - represents an abstract connection to the web server which can handle only one request at a time
  • HttpSocketConnection - new client implementation based on TCP sockets
  • HttpWebRequestConnection - implementation using standard WebRequest and WebResponse classes
  • HttpMessage - container for HTTP request or response including headers and data stream

These classes are all you need to run simple HTTP requests with data uploading/downloading in both ways. I also prepared some code which enables control bandwidth used by network communication. This can be accomplished simply by slowing down read streams - first for the POST data and second for server response, so I created a stream proxy class slowing down read/write operations for a given stream held inside the proxy.

  • WatchedStream - Serves as a proxy for given stream calling events before every read/write operation
  • WatchedStreamEventArgs - Event arguments for stream transfer events
  • HttpAdvancedStream - Uses the events from base class to insert some wait time
  • BandWidthManager - Responsible for measuring time and computing transferred data to keep the desired speed

Class diagram 1

If you would like to use the library to send POST requests, you could use some framework to create POST body. Of course I have created one, and it can prepare request body in two common formats: simple URL encoded and multi-part format.

  • HttpPostBodyBuilder - abstract class for POST body builders
  • HttpPostBodyBuilder.Multipart - Prepares a request body in standard multipart format which allows you upload files
  • HttpPostBodyBuilder.UrlEncoded - Prepares a request body in standard URL format
  • MergingStream - This stream reads data from multiple given streams, which allows to prepare an entire request body without the need to load all necessary data into memory
  • HttpUtility - URL encoding logic extracted from System.Web.dll. This allows you to avoid referencing this library and keep application under .NET Framework Client Profile.

Class diagram 2

Using the Code

I have prepared and attached a client/server sample solution for you to test and better understand the library. There is an ASP.NET web site which allows you to upload image, then it does some graphics operations and sends the result back to the browser. To invoke the service, you can either use your standard browser or the WinForms client emulating browser behavior.

Screenshot

This block of the code is crucial:

this.NotifyState("Converting file...", 0);

// choose connection
HttpConnection http;
switch (httpMethod)
{
	case 0:
		http = new HttpSocketConnection();
		break;
	case 1:
		http = new HttpWebRequestConnection();
		break;
	default:
		throw new NotSupportedException();
}

// prepare request
var url = "http://localhost:12345/Page.aspx";
var postBody = new HttpPostBodyBuilder.Multipart();
var fileStream = new FileStream
		(this.openFileDialog.FileName, FileMode.Open, FileAccess.Read);
var advStream = new BandwidthControlledStream(fileStream);
lock (this.bandWidthSync)
{
	this.uploadSpeed = advStream.ReadSpeed;
	this.uploadSpeed.BytesPerSecond = 1024 * (int)this.upSpeedBox.Value;
}
postBody.AddData(
	"imageFile",
	advStream,
	Path.GetFileName(this.openFileDialog.FileName),
	GetMimeType(this.openFileDialog.FileName)
	);
var bodyStream = postBody.PrepareData();
bodyStream.Position = 0;
var req = new HttpMessage.Request(bodyStream, "POST");
req.ContentLength = bodyStream.Length;
req.ContentType = postBody.GetContentType();
req.Headers["Referer"] = url;

// send request
advStream.BeforeRead +=
	(s, e) => this.NotifyState("Uploading...", e.Position, bodyStream.Length);
var response = http.Send(url, req);

// get response
var readStream = new BandwidthControlledStream(response.Stream);
lock (this.bandWidthSync)
{
	this.downloadSpeed = readStream.ReadSpeed;
	this.downloadSpeed.BytesPerSecond = 1024*(int) this.downSpeedBox.Value;
}
readStream.BeforeRead +=
	(s, e) => this.NotifyState("Downloading...", e.Position, response.ContentLength);
this.convertedFile = ReadAll(readStream, (int) response.ContentLength);
this.NotifyState("Done", 100, true);

As you can see, the code snippet is not quite short, but it does a lot of work:

  • Chooses method to upload data (WebRequests/sockets)
  • Prepares request body with file to upload (file is loaded continuously during sending)
  • Prepares speed-limited streams (you can also choose not to use them, or change speed during transfer)
  • Uses BeforeRead events to inform user about progress
  • Shows how to set request headers

Limitations

Please don't consider this library as full replacement of HttpWebRequest and other classes. This is only a simple solution and cannot be used in any scenario because of its limits:

  • No support for HTTPS
  • No support for proxy
  • No caching
  • Does not keep opened socket
  • Only few HTTP statuses are treated correctly (100, 206, 302 and of course 200)
  • Only seek-capable streams can be used as source for upload

History

  • 1.0 - Initial version

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Petr Bříza
Software Developer (Senior)
Czech Republic Czech Republic
Freelance .NET developer from Ostrava, Czech republic. Mission of my life is to find a question for the answer 42, but in the meantime I try to use .NET and surrounding technologies in behalf of mankind. I hope I bring more good than damage Smile | :)

Comments and Discussions

 
QuestionHow-to-script-a-global-bandwidth-limiter-in-Window? PinmemberMember 1096144221-Jul-14 22:02 
QuestionThank you for taking the time to publish this! PinmemberMember 155025619-Mar-13 5:20 
QuestionGreat job, but about download? PinmemberAlexsandro_xpt16-Apr-12 8:08 
There no code example to that, looking for that, I see method called Send and BeginSend, both get request object parameter with body, it's sound not necessary to GET request method.
 

I'm wrong? I like your job, I'm trying use bandwidth limit to download. Good!!!
 

Thanks!
GeneralMy vote of 5 PinmemberDiamonddrake12-Aug-10 5:28 
GeneralMy vote of 5 PinmemberPetr Pechovic10-Aug-10 21:41 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web04 | 2.8.140721.1 | Last Updated 10 Aug 2010
Article Copyright 2010 by Petr Bříza
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid