Click here to Skip to main content
Click here to Skip to main content

VB.NET Background File Downloader

By , 20 May 2009
 
fileDownloader-downloading.gif

Introduction

This class enables you to easily download multiple files in the background (via a separate thread), and will provide information about the amount of downloaded data, the percentage of completion, and the download speed. On top of this, you can cancel downloading, pause it, and of course also resume. Note that there is also a C# implementation of this class available. 

Background 

I started working on this class after someone on a programming help forum asked how to best download files in the background. I got a lot of ideas and based part of my code on the Downloading Files in .NET with all Information article, by Carmine_XX. This class adds multiple file support, logic/layout abstraction and is designed to be easy to use.

Using the Code 

Once you added the class to your project, you should be able to access it via the namespace Bn.Classes. You can of course adapt the namespace to your own preferences or delete it altogether.  

The first thing you need to do when using this class is (logically) create a new instance, and then add the files you want to download. You'll also need to set the local directory to which you want to download. This is pretty straight forward.

' Creating a new instance of a FileDownloader
Private WithEvents downloader As New FileDownloader
' A simple implementation of setting the directory path, 
' adding files from a textbox and starting the download
Private Sub btnStart_Click(ByVal sender As System.Object, _
	ByVal e As System.EventArgs) Handles btnStart.Click
    Dim openFolderDialog As New FolderBrowserDialog
    With downloader
        If openFolderDialog.ShowDialog() = Windows.Forms.DialogResult.OK Then
            .Files.Clear()
            .LocalDirectory = openFolderDialog.SelectedPath
            For Each path As String In txtFilesToDownload.Lines
           ' The FileInfo structure will parse your path, 
	  ' and split it to the path itself and the file name, 
	  ' which will both be available for you
                .Files.Add(New FileDownloader.FileInfo(path))
            Next
            .Start()
        End If
    End With
End Sub

The code needed to then pause, resume or cancel the downloads couldn't be simpler:

Private Sub btnPauze_Click(ByVal sender As System.Object, _
		ByVal e As System.EventArgs) Handles btnPauze.Click
    ' Pause the downloader
    downloader.Pause()
End Sub
Private Sub btnResume_Click(ByVal sender As System.Object, _
		ByVal e As System.EventArgs) Handles btnResume.Click
    ' Resume the downloader
    downloader.Resume()
End Sub 
Private Sub btnStop_Click(ByVal sender As System.Object, _
		ByVal e As System.EventArgs) Handles btnStop.Click
    ' Stop the downloader
    ' Note: This will not be instantaneous - the current requests need to 
    ' be closed down, and the downloaded files need to be deleted
    downloader.Stop()
End Sub 

The downloader also provides a few properties to indicate its current state: IsBusy, IsPaused, CanStart, CanStop, CanPause and CanResume (all booleans). Here you have an example of how to use these to set your interface:

' This event is fired every time the paused or busy state is changed, 
' and used here to set the controls of the interface
Private Sub downloader_StateChanged() _
    Handles downloader.StateChanged 'This is equal to: Handles 
                       'downloader.IsBusyChanged, downloader.IsPausedChanged
    ' Setting the buttons
    btnStart.Enabled = downloader.CanStart
    btnStop.Enabled = downloader.CanStop
    btnPauze.Enabled = downloader.CanPause
    btnResume.Enabled = downloader.CanResume

    ' Enabling or disabling the setting controls
    txtFilesToDownload.ReadOnly = downloader.IsBusy
    cbUseProgress.Enabled = Not downloader.IsBusy
End Sub	

This is the demo code to display the progress information:

' Occurs every time a block of data has been downloaded, 
' and can be used to display the progress with
' Note that you can also create a timer, and display the progress 
' every certain interval
' Also note that the progress properties return a size in bytes, 
' which is not really user friendly to display
' The FileDownloader class provides shared functions to format these 
' byte amounts to a more readable format, either in binary or decimal notation
Private Sub downloader_ProgressChanged() Handles downloader.ProgressChanged
    pBarFileProgress.Value = CInt(downloader.CurrentFilePercentage)
    lblFileProgressDetails.Text = String.Format("Downloaded {0} of {1} ({2}%)", _
		FileDownloader.FormatSizeBinary(downloader.CurrentFileProgress), _
		FileDownloader.FormatSizeBinary(downloader.CurrentFileSize), _
		downloader.CurrentFilePercentage) & String.Format(" - {0}/s", _
		FileDownloader.FormatSizeBinary(downloader.DownloadSpeed))
    If downloader.SupportsProgress Then
        pBarTotalProgress.Value = CInt(downloader.TotalPercentage)
        lblTotalProgressDetails.Text = _
		String.Format("Downloaded {0} of {1} ({2}%)", _
		FileDownloader.FormatSizeBinary(downloader.TotalProgress), _
		FileDownloader.FormatSizeBinary(downloader.TotalSize), _
		downloader.TotalPercentage)
    End If
End Sub 

Another noteworthy snippet of code is how to set the SupportsProgress property.

' Setting the SupportsProgress property - 
' if set to false, no total progress data will be available!
Private Sub cbUseProgress_CheckedChanged(ByVal sender As System.Object, _
		ByVal e As System.EventArgs) Handles cbUseProgress.CheckedChanged
    downloader.SupportsProgress = cbUseProgress.Checked
End Sub 

When the SupportProgress property is set to true, the file sizes will be calculated before any download is started. This can take a while, definitely when you have a large number of files. The FileDownloader class fires an event every time it starts checking the size of a file, which can be used to display the progress.

' Show the progress of file size calculation
Private Sub downloader_CalculationFileSize(ByVal sender As Object, _
	ByVal fileNumber As Int32) Handles downloader.CalculatingFileSize
    lblStatus.Text = String.Format("Calculating file sizes - _
		file {0} of {1}", fileNumber, downloader.Files.Count)
End Sub

Points of Interest  

When I started working on this class, I assumed it would be ready after working on it for a day. I ran into some problems though, involving a wrong assumption I had about multithreading, and the fact that you can only have two open HttpWebResponses at the same time. Before I created this class, I was oblivious to the fact that objects get locked and can't always be set via all threads without the proper extra code. I also never used the ReportProgress event of the BackGroundWorker. After creating this class, I did some more reading on multithreading to get a better grip on it, and I can recommend the following series of articles: Beginners Guide To Threading In .NET Part 1 of n. I hope this article will both help other people out, and receive some critic feedback on how to improve the class itself.

I'm hoping to implement some more features soon, including cancellation without deleting the files and the option to resume downloading afterwards, and the ability to download multiple files simultaneously, on separate threads. 

History

  • May 20th 2009: Published version 1.0.3 and updated this article 
  • April 30th 2009: Published version 1.0.2 on my forums (added PackageSize and StopWatchCyclesAmount properties) 
  • April 22nd 2009: Published the first version of this article 
  • April 21st 2009: Published the class
    • For the code this is based upon (can be seen as an older version), see this article.

References

  • Dutch support for this class can be found here.
  • If I have only slightly updated code, I won't update this article and only publish it here.

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)

About the Author

Jeroen De Dauw
Software Developer
Belgium Belgium
Member
I am a free and open source software enthusiast and freelance software developer with multiple years of experience in both web and desktop development. Currently my primarily focus is on MediaWiki and Semantic MediaWiki work. I'm in the all time top 10 MediaWiki comitters and am one of the WikiWorks consultants. You can contact me at jeroendedauw at gmail for development jobs and questions related to my work.
 
More info can be found on my website [0] and my blog [1]. You can also follow me on twitter [2] and identi.ca [3].
 
[0] http://www.jeroendedauw.com/
[1] http://blog.bn2vs.com/
[2] https://twitter.com/#!/JeroenDeDauw
[3] http://identi.ca/jeroendedauw

Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board.
Search this forum  
    Spacing  Noise  Layout  Per page   
Questionhow can download a file from password protected host ?memberlight2536025 Mar '13 - 12:53 
how can download a file from password protected host in this method ?
when i try to download a file from username and password protected host my app say download complated Frown | :(
 
sry for my bad eng
QuestionInfinity %memberPaul720 Feb '13 - 16:34 
Sometimes the variable CurrentFilePercentage reports Infinity. Can this be fixed? It seems to happen randomly on different downloads.
QuestionProblem with the ProgressChanged EventmemberPaul78 Feb '13 - 10:27 
It seems like there is only certain code I can put in the ProgressChanged event. If I update a progressbar, that works, but if I try to update a string in a listview it causes a target of invocation exception. Is this a thread safety problem? What can I do? I need to be able to update status inside a listview control.
AnswerRe: Problem with the ProgressChanged EventmemberPaul720 Feb '13 - 16:32 
Found the problem was in my code. Sorry.
BugHuge problem with THE SPEEDmemberMember 847446029 Sep '12 - 13:03 
what i have noticed in the downloadFile Sub in FileDownloader.vb that in
this line :
currentPackageSize = webResp.GetResponseStream().Read(readBytes, 0, Me.PackageSize)
it calls GetResponseStream every time on the while loop for example :
in 646 kb file it calls it about 1292 times, so i figured a way to call this only 1 time!
which makes the downloading process faster about 66% percent more than the original code, here it is :
Dim ResStream As IO.Stream =  webResp.GetResponseStream()
Dim readBytes(2047) As Byte
            Do
 
                currentPackageSize = ResStream.Read(readBytes, 0, 2048)
 
                If currentPackageSize = 0 Then Exit Do
 
                
                writeStream.Write(readBytes, 0, bytesread)
            Loop
 
try it out guys and tell me what you think.
GeneralRe: Huge problem with THE SPEEDmemberwebflashing11 Oct '12 - 14:57 
Could you elaborate a little more, for people like me that are not really experts on the matter, how would the code look like after your change?
 
Because I tried to comment that line, and play around with yours and I wasn't succesful with it.
 
Thanks very much Smile | :)
GeneralRe: Huge problem with THE SPEED [modified]memberMember 847446018 Oct '12 - 6:49 
Do the following and it should work fine:
1- go to FileDownloader.vb
2- go to downloadFile sub
3- and then just copy and paste this:
 
 Private Sub downloadFile(ByVal fileNr As Int32)
            m_currentFileSize = 0
            fireEventFromBgw([Event].FileDownloadAttempting)
 
            Dim file As FileInfo = Me.Files(fileNr)
            Dim size As Int64 = 0
            Me.PackageSize = 2048
            Dim readBytes(Me.PackageSize - 1) As Byte
            Dim currentPackageSize As Int32
            Dim writer As New FileStream(Me.LocalDirectory & "\" & file.Name, IO.FileMode.Create)
            Dim speedTimer As New Stopwatch
            Dim readings As Int32 = 0
            Dim exc As Exception
 
            Dim webReq As HttpWebRequest
            Dim webResp As HttpWebResponse
 
            Try
                webReq = CType(Net.WebRequest.Create(Me.Files(fileNr).Path), HttpWebRequest)
                webResp = CType(webReq.GetResponse, HttpWebResponse)
 
                size = webResp.ContentLength
            Catch ex As Exception
                exc = ex
            End Try
 
            m_currentFileSize = size
            fireEventFromBgw([Event].FileDownloadStarted)
 
            If exc IsNot Nothing Then
                bgwDownloader.ReportProgress(InvokeType.FileDownloadFailedRaiser, exc)
            Else
                m_currentFileProgress = 0
				Dim ResStream As IO.Stream =  webResp.GetResponseStream()
                While m_currentFileProgress < size
                    If bgwDownloader.CancellationPending Then
                        speedTimer.Stop()
                        writer.Close()
                        webResp.Close()
                        Exit Sub
                    End If
                    trigger.WaitOne()
 
                    speedTimer.Start()
 
                    currentPackageSize = ResStream.Read(readBytes, 0, Me.PackageSize)
                    m_currentFileProgress += currentPackageSize
                    m_totalProgress += currentPackageSize
                    fireEventFromBgw([Event].ProgressChanged)
 
                    writer.Write(readBytes, 0, currentPackageSize)
                    readings += 1
 
                    If readings >= Me.StopWatchCyclesAmount Then
                        m_currentSpeed = CInt(Me.PackageSize * StopWatchCyclesAmount * 1000 / (speedTimer.ElapsedMilliseconds + 1))
                        speedTimer.Reset()
                        readings = 0
                    End If
                End While
 
                speedTimer.Stop()
                writer.Close()
                webResp.Close()
                fireEventFromBgw([Event].FileDownloadSucceeded)
            End If
            fireEventFromBgw([Event].FileDownloadStopped)
        End Sub
 
4- Compile and run!
 
tell me the results Cool | :cool:

modified 18 Oct '12 - 17:26.

GeneralRe: Huge problem with THE SPEEDmemberrevno19 Jan '13 - 3:49 
Yes it works good! Thanks!
QuestionSimpleDownloadFile.suomemberTrizarian11 Sep '12 - 14:13 
SimpleDownloadFile.suo will not convert?
Generalnot workingmemberstefanz9227 Jun '12 - 2:26 
it only creates the files but its not downloading am i doing something wrongCry | :((
 
i now know the problem its not downloading .txt files wy?
GeneralRe: not workingmemberFusionOz19 Aug '12 - 16:13 
I had the same problem. How i fixed it was to use the following code in my Form_Load event
 
    
Private Sub Setup_Form_Two_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
 _Downloader.SupportsProgress = True
End Sub
 
Hope that helps
QuestionWith File Sizememberudnsofyan22 Feb '12 - 19:15 
this is good articel what i found....
but how i can add information with file size in listbox, not just filename...
 
sorry, my english not well..
Questionif use datagridview?memberfeatrick14 Nov '11 - 1:25 
how can i download each file if all the link stored in datagridview?
GeneralMy vote of 5memberАslam Iqbal18 Jan '11 - 0:48 
useful article
QuestionAdding a classmemberJames Milligan16 Aug '10 - 22:25 
I know I'm going to look incredibly thick here, but oh well:
 
How do you add a class to a project?
 
I've tried adding a reference, but none of the files in the Bn folder will work. I've tried adding the three files in the Bn folders to my project, no luck. I've done something like this before, and I thought you just added a reference...
 
I'm using VB Express 2010, which is where I'm thinking some of the errors are coming in.
 
Many thanks! I actually started using your other downloader, but it stops at 20MB, the file I'm trying to get it to download is 480MB OMG | :OMG:
 
James
AnswerRe: Adding a classmemberJames Milligan16 Aug '10 - 22:28 
Actually I think I might have done it (doh).
 
I've copied the code from the FileDownloader.vb file in Bn/Classes to a new, empty class file in my project. Seems to have worked, no errors, just some warnings etc.
 
If I need to something else let me know!
 
James
AnswerRe: Adding a classmemberFusionOz19 Aug '12 - 14:50 
All you need to do is the following
 
in Solution Explorer right click the project name, then scroll down and highlight 'Add', this will open a new menu where you need to select 'Existing Item'.
 
This will allow you to select the Class you have downloaded making it available in your project
GeneralGood articlememberLloyd Atkinson10 Aug '10 - 23:54 
5 Smile | :)



GeneralGreat but eating my CPU...memberpimb210 Jul '09 - 4:20 
I love this as I'm trying to make an update module for my application, but it's really slowing down my computer. It might be that my CPU is just slow, but what about this at your own computer?
 
Thanks.
GeneralRe: Great but eating my CPU...memberbn2vs14 Jul '09 - 12:26 
Hey,
 
I'm not noticing any strain on my system while downloading. Do you notice similar effects when downloading at the same speed to the same location via another application? Also, can you check if you have problems when you use the demo application? (That will rule out wrong implementation as a cause.)
 
Although I haven't had any problems, that doesn't say much. I had a 6GiB memory leak a few months ago without noticing for over an hour (the disadvantage of having a good machine I suppose).
 
Cheers
BN
 
GSoC 2009 student for SMW!
---
My little forums: http://code.bn2vs.com
---
70 72 6F 67 72 61 6D 6D 69 6E 67 20 34 20 6C 69 66 65!

QuestionWhat if i Want to download from mediafire ??membergk874 Jul '09 - 9:08 
heyy
great workk , what if i want to download from mediafire and add a password
 
thx in advance
AnswerRe: What if i Want to download from mediafire ??memberbn2vs14 Jul '09 - 12:22 
Hey,
 
I'm not familiar with mediafire, but you can't download any password protected files with this class, unless you first gain access in some way. This class does not support the later part (and ATM I don't see how it could, and what would be the use anyway).
 
Cheers
 
BN
 
GSoC 2009 student for SMW!
---
My little forums: http://code.bn2vs.com
---
70 72 6F 67 72 61 6D 6D 69 6E 67 20 34 20 6C 69 66 65!

QuestionHow organize the tool workmemberAmirtich25 May '09 - 4:40 
I've posted code about credential, and I have a few question to you.
I want to use your class in my application and i need to do some operation with downloaded files.
I must process files one after another.
1/ Is it a good idea to use downloader.Pause for process file in downloader_FileDownloadSucceeded procedure? Is it prevent to download the next one?
 
sub downloader_FileDownloadSucceeded (....)
downloader.Pause
.. do something
downloader.Resume or downloader.Stop
 
end sub
 
2/ How to catch error 404 if no file found on server? other errors...
I have file with length= 0 after error
 
3/ How to download file to Stream instead of file
 

4/ how to get file type from CurrentFile object (is it text/html or jpg or png)
AnswerRe: How organize the tool workmemberbn2vs25 May '09 - 21:54 
Hey,
 
1. If you use the Pause method, the current download will be put on hold, and nice this version does not feature the ability to download multiple files at once, this will also prevent the next download from being executed. Multiple simultaneousness downloads and the ability to cancel and resume individual downloads are planned features. I won't have time to work on it in the next few months though, so I encourage you to try write them yourself and publish the code Smile | :)
 
2, 3, 4. The FileDownloader class does simply not support these features, but it are good suggestions for what can be added. Like I mentioned before, feel free to contribute Smile | :)
 
Cheers
BN
 
GSoC 2009 student for SMW!
---
My little forums: http://code.bn2vs.com
---
70 72 6F 67 72 61 6D 6D 69 6E 67 20 34 20 6C 69 66 65!

GeneralRe: How organize the tool workmemberAmirtich26 May '09 - 1:59 
a few additions
 
Private m_currentContentType As String
'*************
' Sub downloadFile
m_currentContentType = ""
Dim type As String = ""
type = webResp.ContentType
 

' if error delete file with length=0
if
If exc IsNot Nothing Then
'added
speedTimer.Stop()
writer.Close()
If webResp IsNot Nothing Then webResp.Close()
cleanUpFiles(m_fileNr, 1)
bgwDownloader.ReportProgress(InvokeType.FileDownloadFailedRaiser, exc)

else
endif
 

'*****************
 
''' <summary>Gets the ContentType of the current file</summary>
Public ReadOnly Property CurrentContentType() As String
Get
Return m_currentContentType
End Get
End Property
 
'***********************
'FileInfo structure new NEW()
'so we can change name of downloaded file, and more - it directory
' for example Path = "C:\temp" like BasePath or ""
'sName = "0\11\333\myfile.txt"
Public Sub New(ByVal path As String, ByVal sName As String)
Me.Path = path
Me.Name = sName
End Sub
'after that everywhere i use IO.Path.Combine
Dim fullPath As String = IO.Path.Combine(Me.LocalDirectory, Me.Files(fileNr).Name)
 
' in filedownload sub
'added
If Not Directory.Exists(IO.Path.GetDirectoryName(IO.Path.Combine(Me.LocalDirectory, file.Name))) _
Then Directory.CreateDirectory(IO.Path.GetDirectoryName(IO.Path.Combine(Me.LocalDirectory, file.Name)))
 
'your code
Dim writer As New FileStream(IO.Path.Combine(Me.LocalDirectory, file.Name), IO.FileMode.Create)

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Permalink | Advertise | Privacy | Mobile
Web02 | 2.6.130523.1 | Last Updated 20 May 2009
Article Copyright 2009 by Jeroen De Dauw
Everything else Copyright © CodeProject, 1999-2013
Terms of Use
Layout: fixed | fluid