Eyes on NTFS

• Download demo project - 107.48 KB
• Download source - 20.07 KB

Introduction

The Change Journal is a database that contains a list of every change made to the files or directories on an NTFS 5.0 volume, that is, the Change Journal is a log file.

The structure of Change Journal's record includes USN, Reason, Filename, ParentFileReferenceNumber and TimeStamp, etc.
Let me explain them to you:

• USN is ID
• Reason is why the record is written, say, 'create'
• Filename is merely name without path
• ParentFileReferenceNumber is a foreign key to file's path
• TimeStamp is the time when event happens

With this information, we can know what happened to a file, the only problem is where to find the path of the file. There is no simple/easy way to get there as far as I know. Let's face it, NTFS is not a RDBMS, we can't use one 'select' sentence from relevant tables to get all we need. Although, in concept, ParentFileReferenceNumber is a foreign key. Well, what Window2000 provides us to get a FileReferenceNumber is using an opened handle to the file, the API is GetFileInformationByHandle. We have to search the whole directory tree to find what we want! -- That's why I feel tired for hard drive every time I push the 'Path detail' button.

The other thing we should know is that the Change Journal is not available automatically. Somebody should activate/create it before using it.

To archive functions on Change Journal, the key API is DeviceIoControl. To fulfill a retrieval of the Change Journal records, the basic steps are: 

1. Open the Change Journal file handle using API CreateFile 
2. Call DeviceIoControl using FSCTL_QUERY_USN_JOURNAL as the second parameter 
3. Call DeviceIoControl using FSCTL_READ_USN_JOURNAL as the second parameter

Step (1) opens the handle and the handle is also a parameter needed by step (2); Step (2) gets the general information about the Change Journal, the key is UsnJournalID and FirstUsn. UsnJournalID is Session ID, which is needed by every call of Step (3). FirstUsn is needed by first time call of Step (3). Step (3) is called repetitively until we get all the records. Every call of Step (3) requires a StartUsn which indicates from which record we are searching and it could be fetched from the previous call, is it simple for your loop?

The demo code includes a class CChangeJournal and its consumer which is a dialog. CChangeJournal is used to operate the Change Journal, including creating and disabling(deleting) it.

Note: I learned all this from MSDN, especially from the article: 'Keeping an Eye on Your NTFS Drives: the Windows 2000 Change Journal Explained ' by Jeffrey Cooperstein and Jeffrey Richter. In addition, a lot of the code in this demo is from their demo. I simplified the code by ignoring the OVERLAPPED I/O.

Background

Sometimes, you may have done a series of operations on files and probably one of them is wrong, but after a short time, you couldn't remember where the file is and you don't want to roll back all the operations by using 'Undo' provided by the Explorer, because most operations are what you want. At such times, this tool might help.
Anyway, some real time anti-virus software may use this technique.

Using the Code

CChangeJournal provides some functions according to the steps I mentioned above. Let me explain with the code below:

CChangeJournal cj('c');              	// 'c' represents "C:" ;
cj.open();                              	// open file handle    [step 1]

cj.query(...);                           	// get general information, 
				    	// key is UsnJournalID [step 2]

CJRecords recs;                        	// CJRecords is a vector of part 
				    	// structure of Change Journal

cj.getRecords(startUsn,..,recs,...);     	// get records we need [step 3]

// the other things
cj.create();                             	// create Change Journal

cj.disable(UsnJournalID);                	// delete Change Journal need a parameter 
				    	// from (step 2: query)

cj.setDriver('D');                       	// change Drive

cj.setMemoryBlock(bytes);                	// how large you want to allocate for 
				    	// reading records. it's called
                                         	// with default 1M size, you can change 
				    	// if you like. 

Points of Interest

There are several things I want to mention:

1. There is another way to fetch records by using FSCTL_ ENUM_USN_DATA as the parameter of DeviceIoControl, find out from MSDN.
2. In my demo, within the dialog(consumer), I set 'nextUsn' not the real next Usn, while the current last Usn, so I get one duplicate record every time (except page 1), but it prevents calling being blocked. This is because the query will be blocked if the Next usn hasn't been created. Unfortunately, I couldn't stop it being blocked when there is nothing in the database when the Change Journal is just being created!
3. Firstly when I tested my demo, I could find all the paths associated with the ParentFileReferenceNumber except the root directory, in my case, say "C:". Then I tried "C:\", o.k., it started working. It bothers me so far. Could you tell me why when you find out, thanks a lot.
4. What if a directory is renamed? It doesn't change its FileReferenceNumber; If we delete a directory, of course we can't find it any more, because we have no way to get its opened handle. Then what if we recreate a directory with the same name? The FileReferenceNumber changes! So, if we really want to keep an eye on NTFS, we should maintain a database of our own to trace every change we make -- just as the two Jeffreys do in their demo.
5. What's the rule of a new FileReferenceNumber? I couldn't find it.

History

• Article created on 9^th September 2005