Click here to Skip to main content
Click here to Skip to main content

.NET WebScheduler for scheduled downloading of web sites and files through HTTP and FTP

By , 28 Oct 2005
 

Introduction

Ever wanted to create a web application that sends reminder e-mails every week? Or that cleans up a remote images or articles folder? Or to backup a certain FTP folder every night or week? This document describes a ScheduledTask class written by David van Leerdam for use in (for example) a Windows Service that you can run to accomplish these tasks for you.

Background

When you're hosting your own applications, scheduling events for your web server is not a real challenge. But it is an issue when your (for example, ASP.NET or PHP) application is hosted by a third party which does not allow running of home grown timers, schedulers, Windows Services etc. A possible solution is to run the (web) scheduler on a remote machine, for example, a machine at home or at the office, which is always online. This is also great for remotely creating backups of websites through FTP.

Using the code

The solution consists of two parts:

  1. An application, a Windows Service (or whatever) that references the WebScheduler assembly (see downloads).
  2. The WebScheduler assembly containing the ScheduledTask class.

For part 1, I assume you will be creating a Windows Service (see downloads for sample code). All the code my Windows Service contains is:

Protected Overrides Sub OnStart(ByVal args() As String)
    Try
        sites = ScheduledTask.GetAll()
        For i As Integer = 0 To sites.Count - 1
            Dim st As ScheduledTask = sites(i)
            st.Start()
        Next
    Catch ex As Exception
        EventLog.WriteEntry("WebScheduler", _
          "An exception occurred. " & ex.ToString())
        Return
    End Try
End Sub

Protected Overrides Sub OnStop()
    Try
        For i As Integer = 0 To sites.Count - 1
            Dim st As ScheduledTask = sites(i)
            st.Cancel()
        Next
    Catch ex As Exception
        EventLog.WriteEntry("WebScheduler", _
          "An exception occurred. " & ex.ToString())
        Return
    End Try
End Sub

The result of the GetAll() method is an ArrayList containing all the tasks defined in the config file.

Tasks are defined using a config file (currently hard coded to reference c:\WebScheduler.config). This example contains a task definition for creating a backup of mydomain.nl every night at 0:45 A.M.:

<?xml version="1.0" encoding="utf-8" ?>
<config>
    <tasks>
     <task id="ftp.mydomain.nl"
        url="ftp://ftp.mydomain.nl/"
        scheduledFor="everyDay"
        startTime="00:45:00"
        saveTo="c:\backup\websites\mydomain.nl\"
        recursive="true">
            <credentials
            username="davidl"
            password="p4ssw0rd" />
     </task>
    </tasks>
</config>

Note that any configured path to a folder (i.e. not a file) needs to end with '\'. This is applicable to both local and remote paths.

Points of Interest

A few changes are required to make this component more reliable:

  • What happens when the config file is updated while a task is being executed is to be researched.
  • Further attempts should be made when a request or a complete task fails.

All suggestions, bug reports, code updates, refactorings etc. are welcome. Please leave me a message at the bottom of this page so that I can update the article.

Consuming application

I will keep a list of applications that consume this component or parts of it. Please leave me a message so that I can list your application!

  • My own SchedulingService application, see downloads.

History

  • 19 October 2005: Updated the article with version 0.2 of the code. Some major changes include a better FTP component (version 1.2.2 of edtFTP.NET), config format adjustments, and bug fixes. Minor adjustments where made in terms of performance and refactoring.
  • 1 February 2005: Posted the article with version 0.1 of the code.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

About the Author

David van Leerdam
Web Developer
Netherlands Netherlands
Member
I am a .NET developer for a large European IT consultancy company.

Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board.
Search this forum  
    Spacing  Noise  Layout  Per page   
Questionhow do i set this up on my local machine?memberFlipsideDevelopments14 Sep '08 - 23:25 
please help!!!!
 
new to vb.net
QuestionHow to implement in my application.memberUmesh87383 Mar '08 - 19:45 
Hi,
I had developed my own application for FTP . I want to add scheduling facility in my application.
Can you help me in this issue ? How can i use your windows service in my application.
 

 
thanks,
Umesh
Generalcopy of new versionmemberninja_scroll22 Aug '07 - 23:56 
Hi,
 
I would appreciate it if you could send me a copy of the new version source code at
 
roads.get.longer (at) gmail.com
 
Thank you in advanced
G.
Generalcould u pls send me the latest updatememberravi kanthi24 Jun '07 - 18:35 
hi..it's nice work done by u..
 
could u pls send me the updated copy to my mail id
 
viswa_prabhakar@yahoo.co.in
 

with regards
 
prabhakar
GeneralAn update on the article/project's progress [modified]memberDavid van Leerdam8 Sep '06 - 3:38 
Dear reader,
 
I started working on the WebScheduler component and Service in early 2005, because I needed it in my occupation.
 
After the 0.2 release, that has been online for while now, I have worked on a much better, greatly improved, release that I could not finish.
 
Meanwhile, I switched jobs and since then I have much less spare time and am no longer in need for the component/Service. This made that I have since then not touched the project. Some, but not all, of my improvement concepts are already implemented in the version on my development machine. Since I do not have the time to complete the article and source code, I will summarize what my planned improvements were. If the functionality in this article and/or my changes is what you need, you might want to code the changes yourself (in that case, notify me, I really like to hear about it and maybe I can place your updated code).
 
Let me know if you need one of the listed changes, I might have implemented it already completely or partly on my development machine!
 
The greatest benefit of the suggested changes is the decoupling of unrelated responsibilities, so that a task can be run in, for example, a single threaded context and can also be run ad-hoc instead of scheduled.
 
Following is the list of my planned updates:
 
[Design Updates]
Renaming ScheduledTask to WebTask
Moving all configuration functionality from ScheduledTask to a new WebTaskFactory class (thus making the task unaware of any configuration)
Moving all scheduling functionality from ScheduledTask to a new WebTaskScheduler class (thus making the task unaware of any scheduling)
Moving all thread housekeeping from ScheduledTask to the new WebTaskScheduler class (thus making the task unaware of any threading)
Renaming the 'url' config attribute to 'source' and the 'saveTo' config attribute to 'target' to better facilitate new features like upload
Introduce (a new base) custom Exception type(s) and replace all ApplicationException throws with it
 
[General Important Code Improvements]
Ensuring that all classes are thread safe
Delegation of logging to a 3rd party library with runtime configurable listeners
Making any updates required for - or possible by using the most recent edtFTP.NET
Updating the task classes' code with proper exception throwing
Updating the FtpTask code with much, much smarter FTP error handling and retry mechanisms
 
[New Features]
Introduction of events (Started, Finished, Error etc.) in the new WebTaskScheduler class. These are very important because else these events are hard to handle nicely in a multithreaded implementation
Implementation of at least an Task_Error event handler in the new WebTaskScheduler class
Implementation of uploading (Ftp only), see Design Updates
 
Kind regards,
 
David van Leerdam
 

 
-- modified at 10:22 Tuesday 12th September, 2006
General[Message Removed]memberMojtaba Vali26 Oct '05 - 10:10 
Spam message removed
GeneralRe: opensourcememberDavid van Leerdam26 Oct '05 - 20:08 
Hello Mojtaba,
 
I put the example code of my Windows Service in the article. The Windows Service references the WebScheduler classes. Does this satisfy you? If not, please contact me again.
 
Note that this version 0.1 still contains some bugs. I have refreshed the article and source code, but Code Project have not uploaded it yet. If you want I can e-mail you the new version.
 
Kind regards,
 
David van Leerdam
GeneralUnable to Download source codememberPrakash Kalakoti1 Jun '05 - 22:24 
Hi i am trying to download your source code.. but i think the zip file is currepted .. i am not able to extract it .. can you fix it.
 






GeneralRequestmemberPhaniraz14 Feb '05 - 19:46 
Hi Mr.David,
 
This really nice example for the learners of .Net Windows services and its advantages.
And more over this is wat i'm looking for my Application... But in my case i have 2 scenarios like
1. Uploading Files from HeadOffice to LocalSites
2. Uploading Files from LocalSites to HeadOffice
 
And IIS running at HO will be having a Public IP and v can upload file using FTP but where as at LocalSites IIS will be running on a machine which is not having Public IP.
So i can use FTP for Uploading Files from HO to LocalSites.
 
So is there any way to Upload Files from HeadOffice to LocalSites without using FTP
 
Please gimme some suggestions its very urgent for my Application.
 
Thanks in advance.
 
Phani.D
GeneralRe: RequestsussAnonymous14 Feb '05 - 21:49 
Hello Phaniraz,
 
If HeadOffice and LocalSites are on the same network, you could refer to the NetBIOS name of LocalSites from HeadOffice. Or use a dynamic DNS service (www.dyndns.org for example).
 
Uploading (instead of downloading, what this project achieves) would be a better solution though. In this case you might want to update the project with an uploading feature. Nice suggestion for the future.
 
Hope this helped a bit.
 
Kind regards,
 
David van Leerdam
GeneralRe: RequestsussAnonymous15 Feb '05 - 0:42 
Hi..
First of all let me say thanks for ur immediate reply.
Wat u explained is perfectly correct when Site user has some information abt the latest updations happened at HO.
Then only he can DownLoad files.
But here in my case it is very difficult to notify the site user that HO user has made some changes.
 
And one thing is that, HeadOffice and LocalSites are not on the same network.
So v have to make use of Internet.

GeneralRe: RequestsussAnonymous15 Feb '05 - 4:40 
Hi Phani.D ,
 
I'm not sure if I'm getting your problem completely.
 
When you want to transfer data from a known source, to an unknown target, you might want to run a WebScheduler service on the target machine. If not, you might want to change the code, or use another solution.
 
Sorry if this doesn't work out!
 
Kind regards,
 
David van Leerdam

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Permalink | Advertise | Privacy | Mobile
Web04 | 2.6.130516.1 | Last Updated 28 Oct 2005
Article Copyright 2005 by David van Leerdam
Everything else Copyright © CodeProject, 1999-2013
Terms of Use
Layout: fixed | fluid