Threaded WebDownload class with Progress Call-backs

• Download demo project - 24 Kb

Preface

Ever wanted your application to download something from the Internet? Here is an implementation of a C# class that performs an asynchronous download of a file (over HTTP) providing call-backs for both progress and to indicate when the download is complete.

Introduction

Recently I needed my application to be able to retrieve a graphic from a web-server. During the download the rest of the application should still operate with full functionality. Typically a placeholder graphic being used until the download had completed. As soon as the download is complete, the newly downloaded graphic would be used rather than the placeholder.

Because the interface can look a little weird until all the downloads are complete, the user really needs to be aware that this is something that will be rectified after the download is complete.

Framework Provided Solutions

Since .Net operates within a rich framework, there is a good chance that Microsoft have already provided a number of different classes to help:

The WebRequest and HttpWebResponse classes allows a really simple download function to be implemented in just a few lines of code. Indeed, the following code worked for a while:

public Image GetImage(string urlFile)
{
    WebRequest wreq = WebRequest.Create( urlFile );
    HttpWebResponse httpResponse = (HttpWebResponse) wreq.GetResponse();
    Stream stream = httpResponse.GetResponseStream();
    return Image.FromStream(stream);
}

The problem with the above code was that with a large download or a slow web-site, the GetImage function above both locks the main application thread of my program and provides no download progress to the user. Obviously, there were still some options:

• Use a new thread to perform the download, thus leaving my main thread free to work;
• Use the Asynchronous stream mechanisms;

Existing Implementations

I found a couple of VB.NET examples that implemented the Asynchronous download of the file, however they still had some problems:

1. After starting the asynchronous download, they had to wait at a ManualResetEvent to await the download to complete;
2. They did not display any download progress;
3. The used the StringBuilder class to store the download.

The final point was my biggest problem, with the conversion from bytes to chars would render the download pointless, trimming out important information from the download making construction of the image impossible.

My Requirements

1. Shall not block operation with application (including its functionality, not simply the user-interface);
2. Shall be capable of providing some progress indication;
3. Shall notify when the download is complete;
4. Shall preserve all of the downloaded information;
5. Shall cope with unknown file sizes (Note, all information is stored in memory, so there is a limit to file sizes);

The Solution

Shown below are the classes implemented to satisfy my requirements above.

The DownloadThread class simply wraps my download thread and provides a logical place to store the DownloadCompleteHandler. It is also responsible for instantiating the WebDownload class and calling its Download method.

The DownloadInfo class is a simple class storing the download information, this information can be passed around within the asynchronous calls, providing access to the overall download (not just the snip that the call-back is informed).

The WebDownload class wraps the asynchronous download functionality with two call-backs,

• Download is the main function, it initiates the download and then waits for the download to complete. Depending on what type of buffers are used, this function may also construct a byte[] return type from the working buffer;
• ResponseCallback is called by the .Net Framework when the initial response from the web-server has been received. At this point, it is possible to look at the HTTP headers and determine if the web-server has specified the size of the file. If so, a byte array (byte[]) is created of the correct size. This is flagged in the DownloadInfo class with useFastBuffers set to true. If the file size is not known, slow buffers are used (which is a System.Collection.ArrayList allowing the contents to grow during the download);
• ReadCallback is called whenever the read-buffer (set to 1k) fills up, at this point, the master-buffer gains this new read-buffer and the progress call-back is called.

Usage

The included tester application simply uses the following to start a download.

if ( this.downloadUrlTextBox.Text != "" )
{
  this.outputGroupBox.Enabled = true;

  // Initialize all form information
  this.bytesDownloadedTextBox.Text = "";
  this.totalBytesTextBox.Text = "";
  this.progressBar.Minimum = 0;
  this.progressBar.Maximum = 0;
  this.progressBar.Value = 0;

  // Create the download thread (class) and populate with filename and callbacks
  DownloadThread dl = new DownloadThread();
  dl.DownloadUrl = this.downloadUrlTextBox.Text;
  dl.CompleteCallback += new DownloadCompleteHandler( DownloadCompleteCallback );
  dl.ProgressCallback += new DownloadProgressHandler( DownloadProgressCallback );

  // Start the download thread....
  System.Threading.Thread t = new System.Threading.Thread( 
    new System.Threading.ThreadStart(
      dl.Download ));
  t.Start();
}

The tester application also changes its appearance slightly when the progress call-back indicates that the total file size is unknown. It hides the progress bar and places "Total File Size Not Known" within the Total Bytes display. The Bytes Downloaded display still counts up with the progress so far.

Known Issues

An OutOfMemoryException is raised when very large files are downloaded (40Mb caused it in my case, although this corresponded with my system running out of virtual memory).

Conclusion

Currently, this class exists within quite a safe environment. It is hoped that by sharing the code here, not only will others benefit from my efforts but also, with wider testing, any bugs can be discovered sooner.

Disclaimer

These classes have been developed for a prototype application only, if you use them, it is your responsibility to test them against your requirements.