Internet Explorer Favorites, deconstructed

Download Favorites2XBEL.zip - visual studio 2008 solution - 20 KB

Download Favorites2XBELvs2005.zip - visual studio 2005 solution - 20 KB

Introduction

This article contains the information I have gathered on how and where Internet Explorer stores its bookmarks, or 'Favorites' in Microsoft terms.

Background

In my quest to write the ultimate bookmark synchronizer I ran into a problem: Microsoft doesn't offer an API to access the explorer bookmarks structure. While a large part of the bookmark information is available in easy to read formats, the order in which the bookmarks appear in the favorites menu is stored in a binary format in the registry. A search on the internet reveals a few people that have decoded (part of) this structure, and I wanted to add my findings here.

How to start

IE favorites are stored in a directory structure below the 'favorites' folder. This folder can be found by reading the 'Favorites' value in the registry key HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders. Alternatively, in .NET you can call Environment.GetFolderPath(Environment.SpecialFolder.Favorites) to retrieve the path. The favorites folder contains '.URL' files, optionally arranged in a directory structure that reflects the menu structure of your IE favorites.

URL files

URL files (extension: .url) are good old .INI files that contain the target address of a favorite. The name of the url file without the extension is how the bookmark appears in the favorites menu. Here is the content of a bookmark to the codeproject site:

[DEFAULT]
BASEURL=http://www.codeproject.com/
[InternetShortcut]
URL=http://www.codeproject.com/
IDList=
[{000214A0-0000-0000-C000-000000000046}]
Prop3=19,2

As you can see, the codeproject address is stored twice in this URL. However, only the second one is used when visiting a favorite site. The first one (key BASEURL, section DEFAULT) merely stores the original address of the page when it was added as a favorite. Any subsequent edits of the favorite will only modify the key URL, section InternetShortCut.

Ordering

Now for the tricky part: IE stores the menu order of the URLs and folders in a totally different way, namely the registry. This adventure starts at the registry-key HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\MenuOrder\Favorites. This key contains a binary type (REG_BINARY) value 'Order', which is a kind of table that contains ordered records that point to URL files or folders. For every favorites sub-folder on the filesystem there's also a corresponding registry key below the "..\MenuOrder\Favorites" key, each of which contains another 'Order' table. The following screenshots clearly demonstrate the mirroring of the favorites' filesystem structure and the registry structure.

Order table format

An order table is a binary format table. Its basic structure is as follows:

MenuOrderTable = MenuOrderTableHeader,  { MenuOrderRecord }
MenuOrderRecord = MenuOrderRecordHeader, SubRecord0, Filler
SubRecord0 = SubRecord0Header, "short folder or url filename and various other information", SubRecord1
SubRecord1 = SubRecord1Header, "long folder or url filename and various other information"

_{(note that {..} means: repeat zero or more times)}
The exact binary format is given in the following tables:

MenuOrderTable
Field length	Field type	Value	Description
4	UInt32	0x00000008	Header ID, always 8
4	UInt32	0x00000002	Header ID, always 2
4	UInt32	x	Total length of the payload that follows, including the 4 bytes of this field
4	UInt32	0x00000001	Unknown, always 1
4	UInt32	NumberOfRecords	The number of MenuOrder records that follow
NumberOfRecords * n	MenuOrderRecord	See below	'NumberOfRecords' times the MenuOrderRecord structure

MenuOrderRecord
Field length	Field type	Value	Description
4	UInt32	x	Total length of this record upto and including the filler bytes. Also includes the 4 bytes of this field
4	Int32	ordernumber	Number that determines the display order of this entry, or -5 when the entry is unordered
variable	SubRecord0	See below	SubRecord that contains the type, length, name of the menuorder entry
6	byte[6]	0	filler, all zero

SubRecord0
Field length	Field type	Value	Description
2	UInt16	x	Total length of this record, including the 2 bytes of this field
2	UInt16		Flags: Bit 0 : 1 => it's a folder, 0 otherwise Bit 1 : 1 => it's an URL, 0 otherwise Bit 2 : 1 => short 8.3 filename is stored in unicode format, 0 => short name is stored using ASCII
4	UInt32	?	Unknown
4	UInt32	?	Might be the 'last visited' date, is updated everytime you visit this link. Unknown encoding.
2	UInt16	0x20, 0x10 or 0x11	Flags: Bit 0 : 1 => the long filename in SubRecord1 is followed by a string that points to a resource string Bit 4 : 1 => it's a folder, 0 otherwise Bit 5 : 1 => it's an URL, 0 otherwise
variable (even number)	zero-terminated ascii or unicode string, depending on the Flags		Zero-terminated 8.3 style folder name or url name. This field is padded with zeros to contain an even number of bytes.
variable	SubRecord1	See below	SubRecord that contains the long unicode name of this menuorder entry

SubRecord1
Field length	Field type	Value	Description
2	UInt16	x	Total length of this record, including the 2 bytes of this field
36 on vista 18 on XP/2003	byte[]	?	Unknown purpose
variable	zero-terminated unicode string	variable	long name of the folder or URL
variable	zero-terminated unicode string	variable	optional field: examine bit 0 of the second 'flags' field in SubRecord0 to see if this field is present or not. This string points to a resource string located in a DLL, and is formatted like this: @shell32.dll,-12693
2	UInt16	variable	location (byte-index) of SubRecord1 within SubRecord0

How to use

When reading the above information, you are probably only interested in the following bits of information:

• is it a folder or URL
• the ordering number
• the long filename

Caveat

Please note that this information is the result of reverse engineering.. I have only verified these structures on XP and Vista. You should also be aware of the structure differences between Vista and XP: on Vista, the 'unknown' byte array in SubRecord1 has a different size than on XP.

The code

I have condensed the information above into a class called FavoritesNode that allows you to load and examine the internet favorites of the current user. FavoritesNode implements IList<FavoritesNode>, so it's actually a tree of sorts. Every node in this tree is either a 'folder' or 'url'.

How to use

The following example shows how you would load the favorites and print the top-level bookmarks:

    ...
    using CodeProject.IEFavDecon;
    ...
    FavoritesNode node = FavoritesNode.LoadRoot();

    foreach(FavoritesNode child in node)
    {
        if(child.NodeType == FavoritesNodeType.Url)
        {
            Console.WriteLine("Name: {0}, Url: {0}", child.DisplayName, child.Url);
        }
    }

XBEL: XML Bookmark Exchange Language

The project in the attached zip does something slightly more useful: it loads your favorites and writes them to an XBEL formatted XML file (which is the bookmark format that Microsoft should have been using by now). Once you have your bookmarks in XML format its easy to transform them into webpages, import or export them, merge, sort, etcetera.
As an example, the main program file contains a commented-out section that shows how you can apply a stylesheet to transform the generated XML bookmarks into a nice indented HTML file.

Other resources

The following links contain bits and pieces on this subject that were very helpful during my research:

• http://www.arstdesign.com/articles/iefavorites.html
• http://www.cyanwerks.com/file-format-url.html
• http://sourceforge.net/projects/xbel/
• http://pyxml.sourceforge.net/topics/xbel/

History

• Dec 19, 2007 : Initial release
• Dec 22, 2007 : Added sample code, updated text & resource links
• Dec 24, 2007 : Added vs2005 version of the solution

You must Sign In to use this message board.

FAQ    Layout Expand Posts & RepliesThread ViewNo JavascriptNo JS + Preview   Per page 102550

Msgs 1 to 7 of 7 (Total in Forum: 7) (Refresh) FirstPrevNext

Create a URL Shortcut [modified] vb.philip 3:46 7 Dec '08   Great example, I wasn't aware of the XBEL format. I wrote my own VB.NET utility for moving Favorites around different PCs by using the Windows Script Host. The Script Host has routines for creating shortcut url files to disk. Instead of creating a list, like the XBEL file I create the actual favorite file. I typically run my program from a flash drive to capture the favorites for moving to another PC. I like your approach of using the XBEL file because it's managed code where using the Windows Script Host is not in .NET. I need to expand on your solution by providing functionality to create the favorites on a PC from the XBEL file and also to capture the Favorites image icon if there is one. -phil modified on Sunday, December 7, 2008 9:25 AM Sign In·View Thread·PermaLink How to create a custom favorite entry? debussyster 0:31 18 Jul '08   I want to create a favorite by special name and url,and set it the first item in the top menu.how to do that?thanks Sign In·View Thread·PermaLink Date format... slickvguy 17:32 22 May '08   I've started looking into this matter,and while I've only spent a few hours investigating, I've learned quite a bit. Your tables are not always correct. After an import, I see many records in the registry that have the long filename BEFORE the short, and at the location where the short name would be. If you alter anything in that favorite folder, it will re-write the data, and then it'll be like the table in your article. Seems simple now, but after trying to figure out what the date format was in the registry...it's the FAT32 format! Doh! 2 bytes for year/month/day and 2 bytes for the time. Search the net for the exact format - it's well-known. The date format in the Netscape bookmark file is of course Unix UTC format. Unsurprisingly, Microsoft does not use the 3 date fields in the Netscape bookmark file format as they were intended. You'd think "Last_Visit" would be pretty clear. Wrong. Last visit gets put in the "Date Accessed" field, which has nothing to do with visiting the favorite's URL! It has to do with the .url itself. Ridiculous. Date Modified (Windows) is the field that shows you when you "last visited" a particular site. There are many inconsistencies in how various routines in IE and XP use the Favorites registry key and the .url files. Still trying to make (some) sense of it. Thanks for your efforts. Sign In·View Thread·PermaLink But IE is a P.O.S. so what difference does it make? clintonG 10:37 24 Dec '07   The Favorites should be stored in the file system as XML and IE8 should make it possible to rapidly load and unload XML fragments as collections to make it possible for those of us who have collections consisting of tens of thousands of Favorites to actually use and manage Favorites efficiently. EVERYBODY knows this including the greasy stinking filthy pieces of dog sh!t at Microsoft who knew this all the way back at IE5 when many of us developers begged them to change the way IE functioned in this regard but the filthy pieces of sh!t refused and the filthy pieces of sh!t have continued to go out of their way to make life miserable for everybody. Sign In·View Thread·PermaLink Looks nice... 2005 possible? George Bush 9:24 24 Dec '07   I've been looking for ways to do this; unfortunately, I don't have VS2008. Any chance of a 2005 version? Thanks... Sign In·View Thread·PermaLink Re: Looks nice... 2005 possible? Jean-Paul Mikkers 9:31 24 Dec '07   Sure! I don't think I used any 2008 or .NET 3.5 specific constructs, so it should compile without changes on VS2005. I'll give it a shot and upload it as a separate zip. Thanks for the suggestion. Sign In·View Thread·PermaLink Thanks, nice work ! BillWoodruff 18:17 22 Dec '07   Hi Jean-Paul, Very well done article. I personally "hate" the two ways you can re-order the Favorite folders in IE7 : either using drag-and-drop in the clunky dialog that pops up when you choose 'Organize Favorites' from the 'Favorites' menu, or directly in the 'Favorites drop-down menu through drag-and-drop. A glaring omission, to me, in MS's two ways of arranging Favorites is that you can only apply sort-by-name to a selected folder, not the whole collection of Favorites. It would be nice to see you take your ability to parse the Favorites folders and data and then construct a TreeView interface where you could visually drag and drop without the tedious scrolling driven by "bumping" your item-to-drag against the top or bottom border of the dialog or drop-down, then save the results back And in your spare time you could throw in some nice ways of sorting them, also thanks ! Bill "The greater the social and cultural distances between people, the more magical the light that can spring from their contact." Milan Kundera in Testaments Trahis Sign In·View Thread·PermaLink

Last Visit: 10:12 8 Nov '09     Last Update: 10:12 8 Nov '09 1