Partial FTP Downloader

• Download application - 253 KB
• Download source code - 267.28 KB

Why Another FTP Downloader?

There are millions of FTP programs. Why write another one?
The answer is simple: I searched for a tool for my special needs, and I couldn't find one.

I'm sharing a Dreambox with a friend. The dreambox is a digital DVB satellite receiver that runs Linux with a built in harddisk.
The dreambox has a Web interface that can be used to program timer recordings via Internet from anywhere in the world.
It also has an FTP server via which you can download the films from the harddisk once they are recorded.
The Dreambox can be completely remote-controlled via Internet. (Obviously you need DynDns and an open FTP and HTTP port in your router)
So if you live in Brazil and have a friend in Germany who has a dreambox, you can remotely record the German television program and later download it.
You can also use a dreambox to share a Pay-TV subscription with a friend.

Normally if you want to record a film which starts at 20:00 and ends at 22:00, you will program a timer from 21:55 till 22:30 to be sure that you don't miss anything if the transmission starts delayed. But a DVB stream creates between 20 and 50 Megabytes per minute (depending on the TV channel's bitrate). So it would be a great advantage if you can download only the interesting part. To download only the film and exclude the news and advertising I needed a preview function. My friend complained that I was occupying all his DSL upstream to load the films. So additionally a bandwidth control and a download scheduler were required.
All this is realized in this project.

Features

• Purpose: Download files from an FTP server.
• Multiple split files on the server are automatically put together to one file when downloaded.
• Partial downloads are possible: You can specify a start and an end position between which the file is downloaded.
• The preview function allows to download little chunks in fixed distances, so you get a lot of screen shots of the film.
• All program settings are stored in an XML file, so if a download was aborted you can resume by simply hitting the "Start" button, even after shutting down the computer. The password is stored encrypted and can only be decrypted by the same user who encrypted it.

  

• Bandwidth control: A maximum transfer rate can be defined.
• Download scheduler: You can specify a start time and end time between which the program downloads, for example during the night and during the day but not in the evening (23:00 - 18:00).
• Additionally the program can be started with a commandline parameter so an external scheduler like for example PTBSync from ElmüSoft can shut down your PC and wake it up automatically from hibernation and then start the FtpDownloader at a certain time with the parameter "/Run".
  You can also put a shortcut with the parameter /Run into your Autostart folder in start menu to continue a download when your Windows starts up.
• When you minimize the window, the program sits in the IconTray and the tooltip shows the progress.
  The color of the icon changes to green while a download is running and to red after an error:

  (The clock with seconds and date display is created by PTBSync)

• If an Internet or server error occurs during a download, the program automatically keeps on trying to reconnect / download every minute.
• A detailed logging (resizable) shows all errors and success messages and the current download progress.
  If compiled in Debug mode, you additionally see more details in the Trace debug output.

  

• Some FTP servers (like the dreambox) send Unicode filenames (e.g. in Russian, Greek, Japanese) UTF8 encoded.
  You cannot read anything if your FTP program does not support UTF encoding.

  With the checkbox "UTF8" you can toggle the display of filenames:

Previews

The Dreambox stores the films in the DVB satellite / cable stream format with the file extension *.TS (Transport Stream) as a split file:

• Tibet.ts
• Tibet.ts.001
• Tibet.ts.002
• Tibet.ts.003
• etc.

You can easily adapt the generation of split-filenames to your needs: (in the file Defaults.cs)

public static string BuildSplitFileName(string s_File, int s32_Index)
{
   if (s32_Index == 0)
       return s_File;

   return s_File + "." + s32_Index.ToString("D3");
}

To use the Preview function, you simply enter a value into the "Preview" textbox. If you enter "100" here, a preview is downloaded every 100 MB. Later, you find in your download directory:

• Preview_Tibet_00000.ts
• Preview_Tibet_00100.ts
• Preview_Tibet_00200.ts
• Preview_Tibet_00300.ts
• ........
• Preview_Tibet_02500.ts

If you need more previews to decide where the film starts or ends, simply modify the Start, End and Preview values and hit the 'Start' button again.
So in the second cycle, you can enter a finer scale to find the start of the film:
Start: 100 MB, End: 300 MB, Preview: 25 MB.
If a preview already exists in your download folder, it will not be downloaded a second time.

Each preview chunk has a length of 500 KB. This assures that it contains at least one I-Frame which is required to decode the data.
After downloading, the previews open them in a tool like Project-X and see what they contain:

Partial Download

After the above procedure, you know that you have to download for example, from 125 MB until 2350 MB and enter this as Start and End values.
Then you can process the downloaded TS file with Project-X, then cut and multiplex it with a tool like Cuttermaran and finally burn it on DVD.

Resuming Downloads

If you abort a running download and start it again later by clicking "Start" it will automatically resume correctly. If you have set the checkbox "Autostart FtpDownloader with Windows," any interrupted download will resume automatically even after shutting down your PC meanwhile.

Problems may only occur if you manually manipulate existing downloads without understanding how resume works:

FtpDownloader calculates the position where to resume a broken download on the server from the value in the field "Start (MB)" plus the filesize that is already downloaded.

IMPORTANT:
To resume a broken download you must NEVER delete an entry from the listview and then enter it again with the same local filename but a different value in the field "Start (MB)"! Nor must you ever manipulate the XML file manually. This would result in a corrupt file.

It is also impossible to download two different files from the server into the same local file. You cannot merge different files together!

If you have three different recordings and want to put them together as one movie, download them as three separate files and then use Video Cutting software like Cuttermaran to put them together.

To avoid these problems never specify an already existing file in the field "Local Filename" when adding new downloads to the list.

Source Code

You will find a very clean and well structured C# source code with plenty of comments, written by a very experienced programmer.

The code is reusable, so you can easily reuse it in your own projects:

• The FTP class allows (partial) FTP downloads of (splitted) files.

  This class includes four workarounds for the following bugs in Wininet.dll:

  1. The internet timeout settings are ignored (Q176420),
  2. After hours of downloading InternetCloseHandle() hangs or produces a timeout error after 5 minutes when closing a FTP filehandle,
  3. In the FTP directory listing a few UTF8 encoded characters of the filename are crippled.
  4. Sometimes InternetReadFile() returns 0 Bytes although there is more to read.
• The very versatile, fast and threadsafe LogView class to display coloured log information.
• The universal ThreadInvoker class which starts any function synchronously or asynchronously in the context of a new thread.
• The secure PasswordTextbox class which enrypts a password so that it can only be decrypted by the same user and also protects it from API spies.
• The XML class which offers ultra-easy loading and saving of settings into GUI controls.
• The Measure class which measures a timespan with a REAL exactness of 1 millisecond. (used for download speed calculation)
• A ComboBoxEx and a ListViewEx class which allow to display other text than they store. (e.g. store UTF8 and show Unicode)

P.S.:
On my homepage, you find PTBSync and more information about the Dreambox and tools for cutting DVB films.

History

• 25^th June, 2008: Initial version
• 14^th August, 2008: Update image, source, application, and Source Code Section