ExpTreeLib Version 3 - Explorer-like Navigation and Operation for your Forms

• Download demo - 81.7 KB
• Download source - 806.8 KB
• Download Help File - 687.8 KB

Introduction

ExpTreeLib is a Class Library which provides much of the functionality of Windows Explorer. It is typically used to implement File/Directory utilities and Document Management applications.

The library's central class is CShItem which is an API based superset of the .NET FileSystemInfo class. It also contains a User Control, ExpTree, which is an Explorer-like TreeView of the folders of the Windows Shell. ExpTree supports system context menus, drag from and drop on, and normal Shell-like navigation. The library provides dynamic notification of all changes to the contents of the Shell namespace (the FileSystem plus), classes to support drag & drop to/from various controls, and several utility functions that are useful in their own right. Included in the download package are example forms that illustrate the use of the library to build a full Windows Explorer-like application. The demo forms may be easily modified and/or extracted from to meet the needs of your application. Also included is a demo form that uses the library without an ExpTree or ListView and works with files displayed in a DataGridView.

This article only gives a summary of ExpTreeLib and the demo project. To keep the article to a reasonable length and to provide a better reference source, I provide a Help File (ExpTreeLib.chm) as a separate download. In addition to the usual Help content, the Help File contains guidance on how to use the library and demo forms in your own applications and details on how the library and demo forms work. The first step in using the library should be downloading and browsing through the Help File! For a better understanding of the library, you may also browse through the original CodeProject article. You may also download the demo package to see the library in action without actually downloading the source or Help.

Simply displaying the structure and content of folders is not enough for a useful application. The real question is what the application can do with the displayed items. Out of the box, the main demo forms (frmThread, frmTemplate, and frmThreadCS (a C# version of frmThread)) provide the key functionality of Windows Explorer, but in a form that is easily modified to suit the needs of the application. ExpTree and the main demo forms support the following capabilities:

• Automatic update of the TreeView or ListView when any change is made by any process to the displayed folders or their content.
• Dropping a file/folder onto ExpTree and/or the ListView. This feature supports normal Windows Control-Key modifiers and right button drag menu to instruct the receiving control to copy or move the file(s)/folders(s) to the target folder. The drag source may be itself or any application (including Windows Explorer) that provides at least FileDrop (CF_HDROP) DataFormatted information. One useful class of such applications is Windows email clients (Outlook, Outlook Express, Thunderbird, Windows Live Mail, etc.)
• Dragging file(s)/folder(s) from the ExpTree or the ListView. Normal Windows Control-key and right button drag operations are supported. The drop target may be itself or any window (including Windows Explorer) that accepts at least FileDrop (CF_HDROP) DataFormatted information.
• Right-click on a file/folder to display the same ContextMenu that Windows Explorer would display on right-click. File/folder rename support included.
• Edit of the first column of the ListView to rename a file/folder -- if and only if the first column is the file/folder name.
• Column-click sorting (with Sort Glyph) of the ListView columns when displayed in Details view.
• Selected item(s) delete in response to the Delete key or Context Menu.
• Double-click on a file to "Open" it.
• Substantial optimization of both the library and demo forms relative to earlier versions.

Package Summary

Downloadable Zip packages for ExpTreeLib

• Source code -- The full source as a Visual Studio 2008 Solution with four Projects including a C# complete project implementing frmThread in C#. The VS2008 Solution should upgrade without errors to a VS2010 Solution. This download includes:
  • ExpTreeLib - the library.
  • ExpTree_Demo -- Three demo/example forms demonstrating how to use ExpTreeLib.
  • Documentation - The documentation set for the library and examples organized as a website. This material is also included in the Help File download as part of the .chm file.
  • ExpDemoCSharp - frmThread implemented in C# to illustrate how a C# developer can use ExpTreeLib, including the ExpTree control. Set as Startup Project to run this project.
• Demo -- Just the executable and its' required .dll file for a quick look.
• Help File -- A .chm file including substantial additional documentation of the demo/example forms and how to use them to build your own applications.

Background

Version 1.0 of this library was initially published on CodeProject in 2004. It provided a static view of the Shell namespace in a TreeView. Through multiple revisions it advanced to a still static but refreshable view and came to also support a (mostly correct) form of drag & drop, finally arriving at Version 2.11 as the original CodeProject article now describes and provides as downloads. The last update to that version was posted as Version 2.12 in 2012. The drag & drop facility available in that version is also published in a now obsolete article. That article may be of interest to those who prefer to use Version 2.12 instead of Version 3.0 which is covered here.

In order to correctly implement Drag & Drop, it was necessary to substantially rewrite the CShItem class and to add several supporting classes. The rewrite of CShItem included a mechanism that provides notification of any changes to the folders of interest to the application. This allows ExpTree and any other controls to be notified of changes and to update the GUI. This change transformed ExpTreeLib from a static view of the Shell namespace to a Dynamic view. I made other enhancements, most notably System Context Menus, to the library, ExpTree, and to a demo form. This work's result was Version 2.14, also known in the forum as the "Unpublished Version".

Version 2.14 has been distributed to a number of people who contacted me through the forum of the original article. Both 2.11 and version 2.14 have proven popular and are still discussed in the forums of the original article and in direct email to me. The most recent update to Version 2.14 was sent to the distribution list on 12/26/2010.

Version 3.01

Version 3.01 is primarily a bug fix release. It also includes an additional Project with an instrumented version of frmThread.vb and two other Forms that may be of mild interest to a developer who wishes to dig into ExpTreeLib. Release Notes for 3.01 are in the Documentation Project included in the Source download.

Version 3.00

Version 3.00 of this package adds optimization of both the ExpTreeLib library and the demo forms. Much of this optimization is not needed when the user's application is running on a Windows XP system. However, it is very important when the application is running on later (Vista/Win7) systems and accessing large server based folders. Version 3.00 also adds a Help File (.chm), new and corrected comments, and a collection of additional documentation in the form of HTML pages, including additional information about how to modify or extract from the demo forms for use in your own projects. The documentation of the demo project is also included in the Help File available as a separate download.

Summary of changes relative to Version 2.12

More detail is available in the Version 2.14 and Version 3.00 Release Notes found in the Help File. To summarize, Version 3.00 improves on Version 2.12 by providing:

• Dynamic Change Notification of all changes to the Shell namespace that are of interest to the application that are made by all processes on the system.
• A much more robust handling of Drops onto ExpTree and properly coded ListViews or other Controls. See the demo forms and the documentation for how to properly code ListViews and other Controls.
• Drags from ExpTree and properly coded ListViews.
• Windows System Context Menus in ExpTree and properly coded ListViews. See demo forms for how to properly code ListView System Context Menus.
• All changes needed to compensate for differences between various versions of Windows - Win7, Vista, XP.
• Significant improvements in GUI responsiveness for applications running on Windows7 and Vista systems and accessing Remote Folders. XP was always fast enough.
• New demo forms which incorporate changes to improve responsiveness and to be easier to modify and/or extract from for use in applications.
• A C# version of the most complex demo form to illustrate how to use ExpTreeLib from C#.
• Very significant improvements to the Documentation of both ExpTreeLib and the demo forms. This includes provision of a Help File and corresponding improvements in the XML comments within the code.
• Use of Win7/Vista Themes for ExpTree.

Using ExpTreeLib and the demo forms

There are a number of approaches to take to integrate ExpTreeLib into your own application.

• Start from scratch using the documentation and worked examples to use some or all of ExpTreeLib's functionality. Note that it is not necessary to use any Windows Forms components. A Console app might benefit from ExpTreeLib.
• or Start with a copy of one of the demo project's forms, adding your application's Controls and code. See the topic "Building an App" in the Help File.
• or Copy the ExpTree and/or the ListView Controls and code from one of the demo forms to your own form(s). See the topic "Deriving from a Demo Form" in the Help File.

Building a Form to use ExpTreeLib

The first article gives an overview on how to use the ExpTree Control in your application. Also see the topic "Deriving from a Demo Form" in the Help File. To summarize using frmThread as a model:

1. Add a reference to the ExpTreeLib DLL to your project.
2. Add the ExpTree control to the Visual Studio Toolbox.
3. Add an ExpTree control to your form.
4. Add the appropriate Imports (or using in C#) statements to your form. See the demo forms for usage.
5. Add an Event Handler to handle the ExpTree1.ExpTreeNodeSelected Event to receive notification of Node Selections:

       Private Sub AfterNodeSelect(ByVal pathName As String, ByVal CSI As CShItem) _
              Handles ExpTree1.ExpTreeNodeSelected
   

6. If using SystemImageListManager to provide file and folder icons for a ListView, add the code found in either demo form in the #Region "Form Load/VisibleChanged lv1 HandleCreated" region.
7. If using a ListView to display Files/Folders in a manner similar to the demo forms, and if you wish to support drag & drop from/to that ListView, add the appropriate Declarations as found in either of the demo forms to declare and initialize instances of CDragWrapper and ClvDropWrapper. You must also set the ListView.Tag to be the CShItem passed in as CSI in the AfterNodeSelect handler whose declaration is shown above. This is needed to properly handle drops onto a ListView that currently has no Items.
8. To support Dynamic Updating of your ListView:
   • Add the code found in the Dynamic Update Handler Region of either demo form to your code.
   • Add an AddHandler statement to your Form's Load event:

     AddHandler CShItemUpdate, AddressOf UpdateInvoke

   • Declare a FormClosing Event Handler to Remove that Handler:

     Private Sub frmThread_FormClosing(ByVal sender As Object, _
             ByVal e As System.Windows.Forms.FormClosingEventArgs) Handles Me.FormClosing
         RemoveHandler CShItemUpdate, AddressOf UpdateInvoke
     End Sub

   • For completeness sake, also remove the handler in the Dispose routine of your form.
9. See the demo forms, the Help file, and the other included documentation for additional code elements needed to support your application's use of CShItems.

The demo forms

There are several demo forms in the ExpTree_Demo Project. To see a particular form in action, modify the Module modMain in that project by uncommenting the Application.Run(someform) statement and ensuring that all others are commented out. To run frmThreadCS, set its' project to be the Startup Project in Visual Studio. These forms will do normal file manipulation as is posible in Windows Explorer. However, they primarily exist to provide worked examples of how to use ExpTree, CShItem, and the related classes.

frmTemplate

This form is a fully working start point for any form which requires an ExpTree and ListView with enough room left for application specific controls. frmTemplate includes all optimizations that do not require use of a BackgroundWorker. Because of that omission, it is much simpler to understand and to work with. For applications that will be working with small to medium Local Folders, frmTemplate will give fully acceptable performance. It is not recommended for applications that will be working with Remote Folders.

frmTemplate illustrates:

• Use of the ExpTreeNodeSelected Event Handler.
• Use of CDragWrapper and ClvDropWrapper to support Drags From and Drops onto the ListView.
• Handling of dynamic update Events from CShItemUpdate Events.
• Full Context Menus in the ListView.
• Use of LVColSorter for Sorting on ColumnHeader Click.
  • Use of MakeLviItem as a custom ListViewItem builder which builds ListViewItems which include the properties useful for LVColSorter.
  • Use of ListViewSortGlyph.SetSortIcon to set the Sort Glyph on the Sort Column.
  • SortLVItems for how to perform a Refresh of the ListView in response to a Refresh command from the Context Menu.
• ListViewItem editing (first SubItem only).
• Proper handling of the Delete Key.
• Shows how to handle a DoubleClick on a ListViewItem.

frmThread

frmThread is a fully working start point for any form which requires an ExpTree and ListView and is likely to access large and/or remote Folders, requiring a multi-threaded approach to improve GUI responsiveness. As with frmTemplate, it has enough room left for application specific controls.

frmThread includes:

• All features included in frmTemplate.
• Use of a BackgroundWorker Thread to populate a ListView with information that may take excessive time to gather.

frmThread - C# version

Included in the Source download is a separate C# Project whose only form is frmThread implemented in C#. Aside from the implementation language, it is identical to the VB.Net version. In the full Solution, set the Project "CS" to be the Startup Project to run this form.

frmDragToControl

This form illustrates how to use ControlDropWrapper to handle Drag To/Drop On events for a DataBound DataGridView. It is a relatively bare bones form and should only be used to aid understanding of use of ControlDropWrapper.

Available documentation

The definitive explanation of how ExpTreeLib and the demo forms work is the Source code. A full explanation of the details would take at least as many lines of text as there are lines of code - probably much more.

The definitive reference to the classes is the Help file. The Help file documents all important public classes and their methods, events, and properties. Appropriate comments are also displayed by Visual Studio's Intellisense mechanism. The Help file also includes substantial supplemental documentation.

The supplemental documentation is also available as a Project in the Source download. That Project - named "Documentation" contains all the supplemental documentation in the form of a "web site" like collection of HTML pages which can be viewed in your favorite Browser. Simply open the file Documentation\Exp_Index.htm and follow the links. The first article in this series contains valuable information, especially when supplemented by the Release Notes for Version 2.14 and Version 3.00 which are found in the Help file and the Documentation Project. You may also ask a question in the Forum for this article.

How it Works - Key Concepts

Rather than make this article too long, here is a summary of some key points. The concepts to take away are in Bold. The Help file has much more detail.

The CShItem Class

The CShItem class is the primary class of ExpTreeLib. Each instance of CShItem wraps a collection of information about a ShellNamespace Item. Shell Namespace Items include both FileSystem Items and non-FileSystem Items. Windows Explorer (WinExp) is the visible representation of the Shell namespace. An instance of CShItem is the equivalent of one of the Items (file or folder) as displayed by WinExp. CShItem uses the Windows API to obtain and to obtain information about those Items. Every major method or property of ExpTreeLib involves or requires a CShItem. In the demo forms, every TreeNode and ListViewItem has a CShItem in its .Tag property. In frmDragToControl, every Row in the DataGridView is DataBound to a CShItem.

Internally, the CShItem class maintains a shared tree structure that contains CShItem instances that are Items of Interest to the application. Given an existing CShItem called "CSI" that represents a folder in the Shell namespace, the application can obtain CShItems representing the contents of that folder through the CSI.Directories or CSI.Files properties or by calling the CSI.GetItems or CSI.GetDirectories or CSI.GetFiles methods. Once the application has obtained a CShItem via one of these properties or methods, it is considered an Item of Interest to the application until it is explicitly released via the CSI.ClearItems method. To conserve memory, an Application should always explictly release unneeded file CShItems. See the Version 2.14 Release Notes for details.

Each Item of Interest is represented by exactly one CShItem. That CShItem is stored in the internal tree in exactly the same position that the corresponding Shell NameSpace Item is located in the Shell namespace. When a change is made to the Shell namespace, CShItem is notified by a Windows Message of that change. If the change affects one or more Item of Interest in the internal Tree, an Event is Raised to notify the application of that change. The application can then make corresponding changes to the GUI and do whatever else the application needs to do to reflect that change. See the demo forms for examples of how the application may handle Notification Events.

There is no Publicly Accessible CShItem Constructor (Sub New). CShItems are obtained using the .Directories or .Files Properties. In rare cases the application may need to obtain a CShItem corresponding to a file system path. In those cases, use the CShItem.GetCShItem(Path) method or one of its overloads. This is normally only done to change the RootItem property of ExpTree so as to root the TreeView in a folder that is specific to the Application.

Drag and Drop

Drags from and Drops onto ExpTree or any properly configured ListView behaves exactly like dragging from or dropping onto Windows Explorer items. A properly configured ListView will contain ListViewItems which all have a CShItem in their .Tag Properties. Drops onto any other type of Control (eg a DataGridView) is possible if the Control is associated with a single folder. In the case of other application specific Controls, Drops are accomplished by associating the Control with a New instance of a ClvDropWrapper, ControlDropWrapper, or CtvDropWrapper class. Drags from an application specific ListView or TreeView are accomplished by associating the ListView or TreeView with a new instance of the CDragWrapper class.

Just like Windows Explorer, successful Drag and Drop operations will result in the Dragged Files and/or Folders being Copied or Moved to another folder. Right Button Drops will display the same menu of choices as displayed by Windows Explorer for the same operation. Since any successful Drop will result in a change to the underlying Shell namespace, CShItem will be notified of the change and will Raise one or more CShItemUpdate events thus notifying the application of the change.

Solving Responsiveness Issues

Writing a CodeProject article about Version 2.14 of ExpTreeLib had been on my To Do list for years. However, I had never gotten around to it. Recently, users of both 2.11 and 2.14 reported performance/responsiveness issues. The common thread was that everything was fine until they started using Windows 7 clients to access large Folders on a Server. The same applications had no problems accessing those same Folders when running XP on the client. I investigated. The investigation and the results of that investigation are covered in depth in the supplemental Documentation and are summarized below. The results led to a number of large and small optimizations of both ExpTreeLib and the demo forms. Note - the tested operations against copies of the Test Folders on the Local machine take between 15 and 1200 milliseconds using either unoptimized or optimized code. Operations on normally sized Local Folders are effectively instantaneous. Optimization produces noticeable improvements only when operating against Remote Folders.

My test environment consisted of a Win7 client accessing two different folders on a slow, small, WHS system running Server 2003. One folder contained 3,000 files of various types. The other folder contained 2,000 empty sub-Folders.

Optimization came in three phases, each with its own culprits and solutions:

1. Optimization of code in the demo form itself. This major changes were:
   • Elimination of use of GetAttr function. This included changing the definition of a CShItem property, and adding a new one.
   • Simply using AddRange instead of multiple Adds to fill the ListView.
   These changes made an improvement, but, were not enough to create an acceptable user experience on the Test Folders.
2. Certain CShItem properties (Length and the Creation, LastWrite, LastAccess dates) are obtained using FileInfo/DirectoryInfo classes. I have long been aware that these classes are not optimal, especially when accessing Remote Files/Folders. I added a class to define the Windows API FindFirstFile/FindNextFile (FFF/FNF) methods. Unfortunately, the benefits of using FFF/FNF only occur when all Items are retrieved in one pass using the SafeFindHandle returned by the initial call to FindFirstFile.
   Although FFF/FNF is a large improvement over FileInfo/DirectoryInfo, using it from the GUI Thread still caused serious delays in GUI responsiveness on the Test Folders. The design and usage of CShItem makes it complex to incorporate the use of FFF/FNF in a separate Thread into CShItem. I instead created frmThread to use FFF/FNF from a BackgroundWorker to gather the required information for the contents of large Folders.
3. The Windows handling of the HasSubFolders attribute changed sometime after XP sp2. The earlier version would always return True for a remote folder. The current definition queries the remote folder to obtain a accurate value. Acquiring HasSubFolders went from an operation which would take no detectable time to one that was very costly. CShItem would routinely get the value of that attribute for every folder CShItem created. This one change to the API accounts for the majority of the performance differences between ExpTreeLib on XP versus ExpTreeLib on later systems.
   I changed CShItem to only retrieve HasSubFolders when the application explicitly asked for it and also changed CShItem to revert to the XP usage of always returning True for Remote Folders. Note that ExpTree does explicitly use HasSubFolders to determine if a TreeNode should be Expandable. The effect of the optimization is that TreeNodes representing Remote Folders will always be Expandable, even if the folder is empty. This is corrected on an attempt to Expand the TreeNode.
4. Made changes to the ExpTree Control to eliminate redundant and time consuming code. Eliminated the use of the .NET method TreeView.TreeViewNodeSorter which behaves very inefficiently when adding a node to the TreeView.

The results of the Optimizations are given below. F1 is a remote folder containing 3,000 files of various types. F2 is a remote folder containing 2,000 sub-Folders. All times are in seconds:

Action	Original GUI Freeze*	Optimized GUI Freeze*	Optimized Total Time**
Display Contents of F1 in ListView	67.907	1.014	6.536
Direct Access to sub-folder of F2	78.405	1.435	1.435
Expand folder F2 node in Exptree	25.146	1.060	1.060

   * Time before the GUI becomes responsive to user action.
** Time to fully complete the information gathering.

Your mileage may vary! I am interested in hearing about any significant differences and the circumstances that may have contributed to those differences. Compare to the time it takes to do the same operation via Windows Explorer. Note: referencing a remote folder on a unavailable machine will always encounter network timeout delays.

Credits

• Calum McLellan made significant contributions that improved this control. Calum's article Explorer ComboBox and ListView in VB.NET extends this library with both ComboBox and ListView classes. The ContextMenu portion of my library and Demos are derived from Calum's work. Calum uses an early version of ExpTreeLib in his project. It lacks all optimization and some Vista/Win7 modifications.
• My original version of ExpTreeLib contained a class for accessing System image lists. Some important fragments of that class survive in SystemImageListManager. My original was simply a translation from C# to VB.NET of some of Steve McMahon's System image list class which may be found here. Steve's class has substantial additional capabilities for drawing icons and attaching them to other types of controls.
• Steven Roebert's article describes a C# package that was, in part, inspired by my original article. His work formed the basis of my fixes to the original Drag & Drop and change notification. His code has not been updated or supported since 2006 and seems, from the Forum, to have significant problems with Vista/Win7 and 64-bit OSes.
• Eric Woodruff's Sandcastle HelpFileBuilder is a terrific tool for building Help Files. 
• I have also been helped by contributors to the Forums of my previous articles and by other open source code and discussions found in CodeProject and elsewhere on the Internet.

History

• 11/14/2012 -- Version 3.01 Bug fixes, and minor enhancements.
• 07/16/2012 -- Version 3.00. Initial submission to CodeProject of this, the third, article.
• 04/20/2012 -- Version 2.12. Update of Downloads and first article - includes all optimization fixes that can be implemented in version 2.11. Also includes the rollup of all 2.11 bug fixes.
• 12/26/2010 -- Last update to 2.14 sent to mailing list.
• 12/12/2006 -- (approximate date) Version 2.14, (the "unpublished version") sent to those who requested it.
• 03/12/2006 -- Version 2.11. Updated to both articles to support VS2005.
• 09/16/2005 -- Version 2.1. Update to source and demo to make equal to same files in second article.
• 09/16/2005 -- Publication of second article in this series.
• 08/23/2005 -- Version 2 release - Update to part 1 of this article series, source, and demo. Adds a form of Drag & Drop.
• 10/11/2004 -- Initial version of first article and ExpTreeLib - Version 1.0