Click here to Skip to main content
Click here to Skip to main content

Perl Script for uploading modified files to a FTP-Server

, 5 Nov 2000 CPOL
Rate this:
Please Sign up or sign in to vote.
Simple Perl Script that uses a local MS-Access database to store modification dates of files and uploads modified files to a FTP-Server
<!-- Link to source file download -->
  • Download source files - 17 Kb
  • <!-- Add the rest of your HTML here -->

    Introduction

    (This article assumes that you are familar with Perl and ADO).

    When uploading files from a local PC to a remote FTP-server, I ran into the problem of having to upload a lot of files, where only a few actually changed.

    So I decided to compare the timestamps of the local files with those of the remote files. Theoretically a file needs to be uploaded only if

    1. the local modification date is newer than the remote modification date or
    2. the remote file doesn't exists at all.

    In practice, the problem is that the FTP-Server has another timebase (e.g. its clock is some seconds ahed or behind the local clock), so the comparison could fail.

    Upload Algorithm

    The solution presented in this article handles this by storing the modification dates of the files to upload in a local database and does the comparison with these timestamps.

    In a nutshell the "algorithm" works , for each file to upload, like this:

    • Does an entry for the file exist in the database?
    • If no, upload it, create new recordset for this file, storing its path and modification date.
    • If yes, compare the actually modification date of the file with the stored modification date in the database.
    • If actual modification date is newer than the stored one, upload the file, store its modification date in the database.
    • If actual modification date is older than the stored one, do nothing.

    As a drawback, the very first time the script is started, the database is empty, and therefore it once uploads all files, even if they would already exists on the remote FTP-server.

    Starting from the second time, the script is executed, it works as described above.

    Script installation

    You must have installed the Windows version of Perl from ActiveState. I always use the latest version, which is ActivePerl 5.6 as of writing this article. 

    In addition, I'm using some modules that are not in the standard distribution of Perl. These modules can be obtained from CPAN at www.cpan.org.

    The following extra modules need to be installed:

    Each module contains a readme file which explains how to install.

    Script configuration

    When Perl and all modules are installed, you need to configure the script.

    The following values must be configured:

    Value Description Example
    $ftp_server The address (IP or name) of the remote FTP-server "ftp.codeproject.com"
    $ftp_dir The subdirectory on the remote FTP-server. Use no slash at the end "/download/files"
    $ftp_uid The username (user-ID) of the account to log on to the FTP-server "chris"
    $ftp_pw The password of the account to log on to the FTP-server "maunder"
    @src_dir An array of directories to upload. Each given directory is recursively processed (i.e. subdirectories are processed, too). ("c:\mywebsite")
    @wc_exclude An array of substring to match to exclude files or directories. For each found file (i.e. its complete path), a substring-match is executed, and then only if no item is found, this file or directory is processed.  ("_vti",".mdb","\\bak","\\data","server.inc")
    $db_connstr The ADO connection string to connect to the database. You can use here any valid expression, e.g. an OLE-DB string or an ODBC string. "Provider=Microsoft.Jet.OLEDB.4.0;Persist Security Info=False;Data Source=\\\\thisismyserver\\dir\\dir\\dir\\upload.mdb"
    $db_uid The username of the account to connect to the database. Usually this field is empty.  ""
    $db_pw The password of the account to connect to the database. Usually this field is empty. ""

    Script execution

    When you configured everything, simple execute the script by double-clicking it in the explorer or by typing "perl upload.pl" at the command prompt.

    The script writes outpout to stdout, as well as to a logfile called "upload.log".

    Epilog

    If you have any suggestions, found errors or want to comment on that article or script, please write a comment to at the end of this article and I will answer it.

    Please keep in mind that I'm doing Perl for only some few month right now. So there are probably a lot of "no really beautiful" code-fragments in my script. So if you have any constructive comments regarding my coding style or how the way I use Perl, please tell me!

    License

    This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

    Share

    About the Author

    Uwe Keim
    Chief Technology Officer Zeta Producer Desktop CMS
    Germany Germany
    Uwe does programming since 1989 with experiences in Assembler, C++, MFC and lots of web- and database stuff and now uses ASP.NET and C# extensively, too. He has also teached programming to students at the local university.
     
    In his free time, he does climbing, running and mountain biking. Recently he became a father of a cute boy.
     
    Some cool, free software from us:
     
    Free Test Management Software - Intuitive, competitive, Test Plans. Download now!  
    Homepage erstellen - Intuitive, very easy to use. Download now!  
    Send large Files online for free by Email
    Offline-Homepage-Baukasten
    Follow on   Twitter   Google+   LinkedIn

    Comments and Discussions

     
    GeneralMy vote of 5 Pingrouparif111114213-Jan-13 11:48 
    QuestionPerl Script for downloading added/modified files from the FTP server Pinmembergeeks831-Dec-12 7:40 
    GeneralMy vote of 5 PinmemberMangesh Biradar11-Jun-12 0:03 
    GeneralHi Pinmembervinodh61917-Sep-10 10:03 
    Generalthxxxx Pinmemberjadoohere8-Jul-06 3:17 
    GeneralFTP get file in ASCII using ActivePerl (v5.8) PinmemberDominicTan22-Dec-04 16:21 
    GeneralA bit over the top PinmemberJon Hulatt17-Oct-01 7:12 
    GeneralRe: A bit over the top Pinsussnot your Perl17-Jun-03 8:36 
    GeneralRe: A bit over the top PinsitebuilderUwe Keim17-Jun-03 20:17 
    GeneralPotential problem with modification times PinmemberJonathan Gilligan7-Nov-00 22:26 
    QuestionWhy not just compute the time difference? PinmemberAnonymous7-Nov-00 9:10 
    AnswerRe: Why not just compute the time difference? PinmemberUwe Keim7-Nov-00 20:37 

    General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

    Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

    | Advertise | Privacy | Terms of Use | Mobile
    Web01 | 2.8.141223.1 | Last Updated 6 Nov 2000
    Article Copyright 2000 by Uwe Keim
    Everything else Copyright © CodeProject, 1999-2014
    Layout: fixed | fluid