Internet Explorer Favorites, deconstructed

Download Favorites2XBEL.zip - visual studio 2008 solution - 20 KB

Download Favorites2XBELvs2005.zip - visual studio 2005 solution - 20 KB

Introduction

This article contains the information I have gathered on how and where Internet Explorer stores its bookmarks, or 'Favorites' in Microsoft terms.

Background

In my quest to write the ultimate bookmark synchronizer I ran into a problem: Microsoft doesn't offer an API to access the explorer bookmarks structure. While a large part of the bookmark information is available in easy to read formats, the order in which the bookmarks appear in the favorites menu is stored in a binary format in the registry. A search on the internet reveals a few people that have decoded (part of) this structure, and I wanted to add my findings here.

How to start

IE favorites are stored in a directory structure below the 'favorites' folder. This folder can be found by reading the 'Favorites' value in the registry key HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders. Alternatively, in .NET you can call Environment.GetFolderPath(Environment.SpecialFolder.Favorites) to retrieve the path. The favorites folder contains '.URL' files, optionally arranged in a directory structure that reflects the menu structure of your IE favorites.

URL files

URL files (extension: .url) are good old .INI files that contain the target address of a favorite. The name of the url file without the extension is how the bookmark appears in the favorites menu. Here is the content of a bookmark to the codeproject site:

[DEFAULT]
BASEURL=http://www.codeproject.com/
[InternetShortcut]
URL=http://www.codeproject.com/
IDList=
[{000214A0-0000-0000-C000-000000000046}]
Prop3=19,2

As you can see, the codeproject address is stored twice in this URL. However, only the second one is used when visiting a favorite site. The first one (key BASEURL, section DEFAULT) merely stores the original address of the page when it was added as a favorite. Any subsequent edits of the favorite will only modify the key URL, section InternetShortCut.

Ordering

Now for the tricky part: IE stores the menu order of the URLs and folders in a totally different way, namely the registry. This adventure starts at the registry-key HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MenuOrder\Favorites. This key contains a binary type (REG_BINARY) value 'Order', which is a kind of table that contains ordered records that point to URL files or folders. For every favorites sub-folder on the filesystem there's also a corresponding registry key below the "..\MenuOrder\Favorites" key, each of which contains another 'Order' table. The following screenshots clearly demonstrate the mirroring of the favorites' filesystem structure and the registry structure.

favorites filesystem structure

favorites registry structure

Order table format

An order table is a binary format table. Its basic structure is as follows:

MenuOrderTable = MenuOrderTableHeader,  { MenuOrderRecord }
MenuOrderRecord = MenuOrderRecordHeader, SubRecord0, Filler
SubRecord0 = SubRecord0Header, "short folder or url filename and various other information", SubRecord1
SubRecord1 = SubRecord1Header, "long folder or url filename and various other information"

_{(note that {..} means: repeat zero or more times)}
The exact binary format is given in the following tables:

MenuOrderTable
Field length	Field type	Value	Description
4	UInt32	0x00000008	Header ID, always 8
4	UInt32	0x00000002	Header ID, always 2
4	UInt32	x	Total length of the payload that follows, including the 4 bytes of this field
4	UInt32	0x00000001	Unknown, always 1
4	UInt32	NumberOfRecords	The number of MenuOrder records that follow
NumberOfRecords * n	MenuOrderRecord	See below	'NumberOfRecords' times the MenuOrderRecord structure

MenuOrderRecord
Field length	Field type	Value	Description
4	UInt32	x	Total length of this record upto and including the filler bytes. Also includes the 4 bytes of this field
4	Int32	ordernumber	Number that determines the display order of this entry, or -5 when the entry is unordered
variable	SubRecord0	See below	SubRecord that contains the type, length, name of the menuorder entry
6	byte[6]	0	filler, all zero

SubRecord0
Field length	Field type	Value	Description
2	UInt16	x	Total length of this record, including the 2 bytes of this field
2	UInt16		Flags: Bit 0 : 1 => it's a folder, 0 otherwise Bit 1 : 1 => it's an URL, 0 otherwise Bit 2 : 1 => short 8.3 filename is stored in unicode format, 0 => short name is stored using ASCII
4	UInt32	?	Unknown
4	UInt32	?	Might be the 'last visited' date, is updated everytime you visit this link. Unknown encoding.
2	UInt16	0x20, 0x10 or 0x11	Flags: Bit 0 : 1 => the long filename in SubRecord1 is followed by a string that points to a resource string Bit 4 : 1 => it's a folder, 0 otherwise Bit 5 : 1 => it's an URL, 0 otherwise
variable (even number)	zero-terminated ascii or unicode string, depending on the Flags		Zero-terminated 8.3 style folder name or url name. This field is padded with zeros to contain an even number of bytes.
variable	SubRecord1	See below	SubRecord that contains the long unicode name of this menuorder entry

SubRecord1
Field length	Field type	Value	Description
2	UInt16	x	Total length of this record, including the 2 bytes of this field
36 on vista 18 on XP/2003	byte[]	?	Unknown purpose
variable	zero-terminated unicode string	variable	long name of the folder or URL
variable	zero-terminated unicode string	variable	optional field: examine bit 0 of the second 'flags' field in SubRecord0 to see if this field is present or not. This string points to a resource string located in a DLL, and is formatted like this: @shell32.dll,-12693
2	UInt16	variable	location (byte-index) of SubRecord1 within SubRecord0

How to use

When reading the above information, you are probably only interested in the following bits of information:

is it a folder or URL
the ordering number
the long filename

Caveat

Please note that this information is the result of reverse engineering.. I have only verified these structures on XP and Vista. You should also be aware of the structure differences between Vista and XP: on Vista, the 'unknown' byte array in SubRecord1 has a different size than on XP.

The code

I have condensed the information above into a class called FavoritesNode that allows you to load and examine the internet favorites of the current user. FavoritesNode implements IList<FavoritesNode>, so it's actually a tree of sorts. Every node in this tree is either a 'folder' or 'url'.

How to use

The following example shows how you would load the favorites and print the top-level bookmarks:

    ...
    using CodeProject.IEFavDecon;
    ...
    FavoritesNode node = FavoritesNode.LoadRoot();

    foreach(FavoritesNode child in node)
    {
        if(child.NodeType == FavoritesNodeType.Url)
        {
            Console.WriteLine("Name: {0}, Url: {0}", child.DisplayName, child.Url);
        }
    }

XBEL: XML Bookmark Exchange Language

The project in the attached zip does something slightly more useful: it loads your favorites and writes them to an XBEL formatted XML file (which is the bookmark format that Microsoft should have been using by now). Once you have your bookmarks in XML format its easy to transform them into webpages, import or export them, merge, sort, etcetera.
As an example, the main program file contains a commented-out section that shows how you can apply a stylesheet to transform the generated XML bookmarks into a nice indented HTML file.