It seems to me that you 've always wanted to create some backup/restore feature with incremental processing of each files. Here is a small library that demonstrates the power of IRdcLibrary COM API, ready to be used.
COM, Win32 and C++, as usual. I don't tolerate more beginners than me here :P.
Using the code
The DIFF library can read/write from files and memory. For reading data, you use Microsoft's IRdcFileReader interface. For your convienience, I provide 2 helper classes in the code that can provide the IRdcFileReader interface:
MemoryRdcFileReader, which can read from memory. The memory must persist while this interface exists (i.e. the interface does not copy the memory).
MemoryRdcFileReader(const char* dd,unsigned long long size);
FileRdcFileReader, which can read from an existing HANDLE. The handle must have been opened with GENERIC_READ and must be seekable.
For writing data, my functions require an instance of a DIFFWRITER class. I also provide a FileDiffWriter (which can provide a DIFFWRITER from a HANDLE which must have been opened with GENERIC_WRITE) and a MemoryDiffWriter which writes to a vector<char>. These helper classes provide the member function GetReader(IRdcFileReader**) that returns a MemoryRdcFileReader or a FileRdcFileReader.
Let's say you have a file named 1.txt in c:\test , and you have backed it in c:\test2. Now let's say that you have modified 1.txt in the original c:\test directory, so c:\test\1.txt is newer. Your incremental backup steps now are (error checking is removed for simplicity):
This will save in the files the RDC signature for both old 1.txt and new 1.txt.
Your next step is now to generate a diff between 2 files. My library provides you with 2 options: Either you create a self-contained diff, or a standalone-diff.
A self-contained diff allows you to reconstruct the updated file from the source file offline.
A standalone diff is a lot smaller, but it needs parts of the remote file. So if you are to use standalone diffs, you have to implement a IRdcFileReader yourself that downloads the requested portions of the updated file from a server. Since this needs a server, we will focus on self-contained diffs here.
Now, you want to create the diff between the old and the new file.
This generates a self-contained diff (if you pass NULL instead of r2 in the 3rd parameter it generates a standalone diff). Instead of uploading now your new file c:\test\1.txt, you can upload this diff file which is smaller. How smaller it is depends on how much the files differ.
Now say that you have the original (master) backup of 1.txt and you want to restore the newer 1.txt. You need the original 1.txt and a self-contained diff file:
And that's all. Your newer 1.txt file has been reconstructed by the original file and the diff. If the diff is a standalone diff, you would have to pass a reader to DIFF::Reconstruct's 3rd parameter which would be asked to provide portions of the file.
Points of Interest
- The API requires Vista or newer.
- The library runs ONLY in multithreaded COM environement (CoInitializeEx(0, COINIT_MULTITHREADED);)
- The constructor throws an exception if the COM object cannot be created.
- There's a similarity API I must investigate more.
- The library uses smart pointers, both COM ATL and C++.
21 - MAR - 2016: Implemented as .h file, with some bugfixes.
21 - OCT - 2015: A few bugfixes and some experimental multiple-sig stuff.
06 - JAN - 2015: First Release (and Happy NY:))