I've long been interested in promoting the use of the NTFS USN Journal in Windows applications. And while .Net doesn't provide access to the USN Journal, I've written the PInvoke code to make it easy to use from C#. I've written a NtfsUsnJournal class in C# that exposes the functionality of the USN Journal but masks most of the complexity required to use it. I've also written a WPF application I call StCroixSkipper's USN Journal Explore that lets you select a volume, create or delete the USN Journal on that volume, save the USN Journal state, query the USN Journal and view the changes (USN Journal entries) to the volume. Since accessing the USN Journal is much like reading directly from the NTFS Master File Table, I also let you enumerate all of the directories on the volume and display them in a ListBox.
I'd like to share the current version with folks to get a discussion going. I'd like to see ideas on how the USN Journal could be used in applications to speed up things like identifying all the files of a given type on a volume or finding all of the identical files on a volume or finding all the files who attributes meet certain criteria.
I'd also like to solicit comments on the code. Out of this process I'd like to bring together a community of folks who have an interest in sharing thier ideas and code relative to the USN Journal.
Remember that to use or access the USN Journal you'll need admin rights. So right click on the application and select the "Run as administrator" menu option. If you run without admin rights, you'll get an 'Access denied' message.
Also, the 'View Changes' button reads the USN Journal entries from the saved state to the current state. So if you have no saved state, you'll get 'No saved state' message.
Here is the code and a copy of the executable attached as an exe file and then the entire project with source code in a zip file. I'll try to download these to make sure I can unzip them.
A little background on the USN Journal:
Jeffrey Cooperstein and Jeffrey Richter have co-authored a great article Keeping an Eye on Your NTFS Drives: the Windows 2000 Change Journal Explained If you are interested in the USN Journal, this article is well worth your time.
Each entry in the USN Journal is assigned a 64 bit number which is the location with in the actual USN Journal file of the entry. USN Journal entries can be different sizes since at least the filenames are different lengths.
Each file or directory on a volume has a unique 64 bit File Reference Number. Files can have the same names excluding the path but no two files or directories can have the same File Reference Number.
Here is the format of a USN Journal entry.
/// <summary>
/// Contains the USN Record Length(32bits), USN(64bits), File Reference Number(64bits),
/// Parent File Reference Number(64bits), Reason Code(32bits), File Attributes(32bits),
/// File Name Length(32bits), the File Name Offset(32bits) and the File Name.
/// </summary>
public class USN_RECORD
{
private const int FR_OFFSET = 8;
private const int PFR_OFFSET = 16;
private const int USN_OFFSET = 24;
private const int REASON_OFFSET = 40;
public const int FA_OFFSET = 52;
private const int FNL_OFFSET = 56;
private const int FN_OFFSET = 58;
public UInt32 RecordLength;
public UInt64 FileReferenceNumber;
public UInt64 ParentFileReferenceNumber;
public Int64 Usn;
public UInt32 Reason;
public UInt32 FileAttributes;
public Int32 FileNameLength;
public Int32 FileNameOffset;
public string FileName = string.Empty;
/// <summary>
/// USN Record Constructor
/// </summary>
/// <param name="p">Buffer of bytes representing the USN Record</param>
public USN_RECORD(IntPtr p)
{
this.RecordLength = (UInt32)Marshal.ReadInt32(p);
this.FileReferenceNumber = (UInt64)Marshal.ReadInt64(p, FR_OFFSET);
this.ParentFileReferenceNumber = (UInt64)Marshal.ReadInt64(p, PFR_OFFSET);
this.Usn = Marshal.ReadInt64(p, USN_OFFSET);
this.Reason = (UInt32)Marshal.ReadInt32(p, REASON_OFFSET);
this.FileAttributes = (UInt32)Marshal.ReadInt32(p, FA_OFFSET);
this.FileNameLength = Marshal.ReadInt16(p, FNL_OFFSET);
this.FileNameOffset = Marshal.ReadInt16(p, FN_OFFSET);
FileName = Marshal.PtrToStringUni(new IntPtr(p.ToInt32() + this.FileNameOffset), this.FileNameLength / sizeof(char));
}
}
In the USN_RECORD class you can easily see what the entry contains. You read a buffer from the USN Journal and parse thru the buffer to find and marshal the values from the buffer into the fields of the USN Journal entry.
So we now have access to the record length, the file reference number, the parent's file reference number (which is the file reference number of the directory which contains the file), the reason why an entry was generated for this
file or directory, the files attributes, the length of the files name in bytes and the offset into the entry of the file name.
As it so happens, the Master File Table is made up of entries that are of the same format so this one class works for both the USN Journal and enumerating all of the files and directories on a volume using the Master File Table. It takes about one tenth of the time to enumerate all of the files and directories on a volume by reading the Master File Table on an active drive than the traditional FindFirst() FindNext() approach.
