A Windows FTP Tool - CodeProject

Download FTPTool - 19.92 KB

Introduction

I use a Windows 2008 server from a hosting company to support IT operations for several local non-profit schools. The server is hosting several SQL Server 2005 databases, among other things. Each morning at 3:00AM, a program I wrote will shutdown the database server, copy database files to a backup folder (the files will be renamed by adding a date string), and then restart the database server. The whole process takes about one minute to finish. This way, I can save a copy of data for each day, just in case a server crash will corrupt my data. Since there is limited storage on the Windows 2008 server, my program will also delete files in the backup folder that are more than 7 days old.

What if a more serious disaster happens? Say, the hard disk on the server gets toasted? What if I need to look at data snapshot that is more than 7 days old? There may be better solutions, however, I think all I need is a program that will automatically download files from the backup folder on the remote server to my home machine via FTP.

Please note that what I provided in this article is not an FTP client library that you can use and extend to build your own programs. It is a tool ready for use, all you have to do is modify the configuration file.

The FTP Tool

My FTP tool is a .NET console application, it does one thing and one thing only. That is, downloading/uploading contents of a folder (ok, two things). You can schedule it to run at your convenience. Here is how it works in downloading mode, it works similarly in uploading mode:

Read remote folder path and local folder path from configuration file.
Get list of files and subfolders of the remote folder.
For each file, download it from the remote folder to the local folder.
For each subfolder, create a subfolder with the same name on the local folder and go to step 2 (recursively download files/folders of the remote subfolder).

Here is a sample of the configuration file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key ="FTPUserName" value ="MyUserName"/>
    <add key ="FTPPassword" value ="MyPassword"/>
    <add key ="FTPURL" value ="ftp://MyServer.com"/>
    <add key ="FTPRemoteFolder" value ="%2fMyRemoteFolder"/>
    <add key ="FTPLocalFolder" value ="C:\MyLocalFolder"/>
    <add key ="FTPMode" value ="download"/>
    <add key ="MaxWorkerThreadCount" value ="16"/>
    <add key ="MaxRetryCount" value ="3"/>
    <add key ="CleanupDays" value ="0"/>
    <add key ="TraceDirectory" value ="c:\logs\ftp"/>
    <add key ="TraceFileName" value ="FTPTool"/>
    <add key ="TraceLevel" value ="4"/>
  </appSettings>
</configuration>

The value of FTPMode determines whether it will download a remote folder to local machine or upload a local folder to remote server.

The value of FTPRemoteFolder specifies the path of the remote folder on the server. String %2f indicates the path is relative to the FTP root directory. Without %2f the path will be relative to the FTP user's home directory.

The value of MaxWorkerThreadCount is the maximum number of worker threads in the thread pool of the current process. Each file is downloaded by one of these threads.

The value of MaxRetryCount is the maximum number of times each file will be retried in case of error. The default value is 3. That is, each file will be retried at most 3 times.

If you don't want to keep all files downloaded to your local folder forever, you can specify a positive value for CleanupDays, say 60, which will cause all files in your local folder that are more than 60 days old to be deleted automatically!

Note

The program is designed to run without human intervention. Any error will be written to the log file defined in the configuration file. In case of error, such as loss of network connection, the FTP commands will be retried until all files and subfolders have been downloaded / uploaded successfully or until each file has been retried the maximum number of times.
If a file on remote server already exists on local machine, it will not be downloaded by this program unless the version on remote server is newer (it checks/compares time stamps). The same applies to upload mode.

The Code

Here is the code for downloading in the main method. As you can see, it just calls the DownloadFolder method which does the main work. In case of error, DownloadFolder will retry FTP commands. There is also a corresponding UploadFolder method to do the work in uploading mode.

Tools.Trace.WriteTrace(4, "FTPTool starting ...");
ThreadPool.SetMaxThreads(MaxWorkerThreadCount, 8);
// Start downloading remote folder
DownloadFolder(FTPRemoteFolder, FTPLocalFolder);
// Wait for all worker threads to finish
while (WorkItemCount > 0) Thread.Sleep(3 * IdlePause);
Tools.Trace.WriteTrace(4, "... FTPTool ended");

The DownloadFolder method uses .NET FtpWebRequest class to get file list from remote server and to download files. Here is the code to get file list:

FtpWebRequest req = (FtpWebRequest)FtpWebRequest.Create(FTPURL + "/" + RemoteFolder);
req.Credentials = new NetworkCredential(FTPUserName, FTPPassword);
req.EnableSsl = false;
req.UsePassive = false;
req.UseBinary = true;
req.KeepAlive = false;
req.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
StreamReader reader = new StreamReader(req.GetResponse().GetResponseStream());
string[] pRemoteFiles = reader.ReadToEnd().Split("\n".ToCharArray());
reader.Close();

For each subfolder, the DownloadFolder method will be called recursively. For each file that has not been downloaded already, DownloadFolder will use one of the worker threads to download a file to the local folder. Here is the code to download a file, the code to upload a file is similar:

FtpWebRequest req = (FtpWebRequest)FtpWebRequest.Create(FTPURL + "/" + 
			RemoteFolder + "/" + sRemoteFile);
req.Credentials = new NetworkCredential(FTPUserName, FTPPassword);
req.EnableSsl = false;
req.UsePassive = false;
req.UseBinary = true;
req.KeepAlive = false;
req.Method = WebRequestMethods.Ftp.DownloadFile;
BinaryReader breader = new BinaryReader(req.GetResponse().GetResponseStream());
byte[] pDataBuffer = new byte[_FileSize];
int nPos = 0;
while (true)
{
    int nSize = breader.Read(pDataBuffer, nPos, _FileSize - nPos);
    if (nSize <= 0) break;
    nPos += nSize;
    if (nPos >= _FileSize) break;
}
breader.Close();
if (nPos != _FileSize) throw new Exception("Wrong file size");
BinaryWriter bwriter = new BinaryWriter(new FileStream
		(LocalFolder + "\\" + sRemoteFile, FileMode.Create));
bwriter.Write(pDataBuffer, 0, nPos);
bwriter.Close();

Limitations

Since there is no standard output format for listing directory detail (it depends on the FTP server), the program may break if running against a server that uses a unsupported output format. Currently, I have tested this program against Windows 2003, Windows 2008, and Windows 7. The downloading mode has been tested against a few Unix/Linux servers.

When using the tool on a server in a background task, please remember to adjust your firewall settings to allow this tool to connect to remote machine, otherwise it will keep throwing exceptions.

History

01/27/2011: Initial version posted
01/28/2011: Updated code; Modified article text (added code and explanations)
01/30/2011: Added support for Unix/Linux FTP servers
01/31/2011: Used worker threads to increase downloading speed
02/01/2011: Updated source code to fix a bug
02/02/2011: Fixed a problem in Trace.dll (on Windows 7 64Bit edition)
02/11/2011: Added uploading feature
02/25/2011: Modified code to replace existing file with newer version (check date/time stamp) during download/upload, the previous version would not replace an existing file
03/30/2011: Added max retry count. In case of error, each file will be retried up to the maximum number of times. The previous version will retry forever.