Reading contents of Web Folders in C#

• Download source files - 25.6 Kb

Problem

Once I needed to make some automation processing of Microsoft Word documents located in a SharePoint Portal Server. But I faced an interesting problem – how to find all the documents in SharePoint programmatically?

Investigation – Flops and Boomers

Trial of the FileSearch interface

First, as long as I already used Word automation interfaces, I tried to use the file searching capabilities implemented in the Microsoft Office API, namely the FileSearch interface exposed by Office’s Application object. I assumed that whilst Microsoft Word is capable of opening documents directly from SharePoint, of course, it would be able to search through Word documents on the server…

Naïve assumption - of course not. Things are never so simple that you may relax and have the fun of the “famous” KISS principle. It was able to work properly only on local disk, or, at most, at a network file share.

Trial of the SharePoint Web Services

OK, I know that the SharePoint Server exposes its content in the form of web services. I tried that stuff too, but those entities that I had to work with seemed to me much more complicated than was required for such a simple task - a lot of lists and items of different types. Also, I was not sure about the compatibility between different versions of the SharePoint Server (like 2001, 2003, and the next 2006), and furthermore, I wanted a solution that might work with not only the SharePoint Portal Server, but with any Web Folder that you may browse in Windows Explorer (e.g., running under pure Windows Server 2003).

Trial of WebDAV and Web Extender Client (WEC) protocols

As you may know, it is possible to browse a web folder in Windows Explorer only if the IIS server supports either the FrontPage Web Extender Client (WEC) or the Web Distributed Authoring and Versioning (WebDAV) protocol extensions.

There are different ways of making WebDAV/WEC requests – through composing XML requests, or through writing a defined set of HTTP headers into the HTTP request. These would require deep scrutinizing of the standards.

I found out that MS Office products use the FPWEC.DLL COM library usually located at %PROGRAMFILES%\Common Files\Microsoft Shared\web server extensions\60\BIN. It contains a set of wrappers for these protocols, but I found no documentation on them, and their usage did not seem to be straightforward, mainly in the asynchronous manner.

Unfortunately, I didn’t find any free really-working C# library that would allow mw to easily use WebDAV.

One of the ways is to use the OLE DB Provider for Internet Publishing from old ADO. However, on MSDN, there is this statement that “the .NET Framework Data Provider for OLE DB does not support OLE DB version 2.5 interfaces. OLE DB Providers that require support for OLE DB 2.5 interfaces will not function properly with the .NET Framework Data Provider for OLE DB. This includes the Microsoft OLE DB Provider for Exchange and the Microsoft OLE DB Provider for Internet Publishing”. I didn’t like to use COM interop over ADO 2.5 interfaces from my C# code, and I skipped this solution. However, you may try it – it may be quite feasible in some cases.

The Alternative Solution – Web Folder Short Cuts

Suddenly, I remembered that sometimes when you try browsing a web folder, a shortcut to the web folder is added to the My Network Places folder available in the Windows Explorer.

As you may know, these shortcuts are physically placed into the c:\Documents and Settings\{USER NAME}\NetHood hidden folder. A web folder shortcut is a physical directory with a read-only file attribute (the directory name is equal to the name of the shortcut) that contains two specific files:

• Desktop.ini (file attributes are Hidden, System) – a text file that contains the ProgID of a shell extension that will handle how to represent the content of the shortcut to the user.
• target.lnk – a binary file that contains a URL to the web folder the shortcut represents.

So I tried to read the content of the web folder shortcut just as Windows Explorer is expected to do; I used the “Shell Objects for Scripting” (VBScript sample):

dim oShell
dim oFolder
dim sDir
Dim s

set oShell = CreateObject("Shell.Application")
set oFolder = oShell.NameSpace("c:\Documents and" & _ 
              " Settings\Administrator" & _ 
              "\NetHood\The_ShortCut_Name")

for i=0 to oFolder.Items.Count-1
    s = s + oFolder.Items.Item(i).Path+ _
            chr(10)+chr(13)
next

MsgBox s

… and it has worked out. Inside, the appropriate shell extension (pre-installed with Windows) provides the necessary Folder and FolderItem objects, doing all the hard work of interacting with the web server through WebDAV/WEC for us – we will just utilize their results. Note, that passing a pure HTTP URL to the Shell.Application object won't work.

Recently, I came across a script code that allows creating web folder shortcuts (Code Ccomments) - many thanks to the author. I converted the code into C#:

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using System.Collections.Specialized;
using System.Text.RegularExpressions;

namespace Utils
{
    public static class PathRoutines
    {

        //'44 seems to be the length where we have 
        //to change a byte from 00 to a 01.
        private const int URL_CUTOFF = 44;

        //This is where we construct the target.lnk
        //file byte by byte. Most of the lines 
        //are shown in 16 byte chunks,
        //mostly because that is the way I saw it 
        //in the Debug utility I was using to inspect shortcut files.
        private static readonly byte[] LINK_CONTENT_PREFIX = new byte[]
        {//Line 1, 16 bytes
          0x4C, 0x00, 0x00, 0x00, 0x01, 0x14, 0x02, 
          0x00, 0x00, 0x00, 0x00, 0x00, 0xC0, 
          0x00, 0x00, 0x00,
         //Line 2, 16 bytes
          0x00, 0x00, 0x00, 0x46, 0x81, 0x00, 0x00, 
          0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
          0x00, 0x00, 0x00, 
        //Line 3, 16 bytes
          0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
          0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
          0x00, 0x00, 0x00,
        //Line 4., 16 bytes. 13th byte is significant.
          0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
          0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 
          0x00, 0x00, 0x00,
        //Line 5. 13th byte is significant.
          0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
          0x00, 0x00, 0x00, 0x00, 0x00,
        //When I was analyzing the next byte of shortcuts 
        //I created, I found that it is set to various values,
        //and I have no idea what they are referring to. 
        //In desperation I tried substituting some values.
        //00 caused a crash of Explorer. FF seeems to work fine for all.
          0xFF};

        private static readonly byte[] LINK_CONTENT_MID = new byte[]
        {
            0x14, 0x00,
            //Line 6, 16 bytes
            0x1F, 0x50, 0xE0, 0x4F, 0xD0, 0x20, 0xEA, 0x3A, 
            0x69, 0x10, 0xA2, 0xD8, 0x08, 0x00, 0x2B, 0x30,
            //Line 7, 16 bytes
            0x30, 0x9D, 0x14, 0x00, 0x2E, 0x00, 0x00, 0xDF, 
            0xEA, 0xBD, 0x65, 0xC2, 0xD0, 0x11, 0xBC, 0xED,
            //Line 8, 16 bytes
            0x00, 0xA0, 0xC9, 0x0A, 0xB5, 0x0F, 0xA4
        };

        private static readonly 
                byte[] LINK_CONTENT_MID2 = new byte[]
        {
            0x4C, 0x50, 0x00, 0x01, 0x42, 0x57, 0x00, 0x00,
            //Line 9, 16 bytes
            0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
            0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00,
            //Line 10, 2 bytes
            0x00, 0x00
        };

        private static readonly 
                byte[] LINK_CONTENT_POSTFIX = new byte[]
        {
            //Last line, 13 bytes
            0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 
            0x00, 0x00, 0x00, 0x00, 0x00, 0x00
        };        

        /// <summary>
        /// Returns path to a directory with name 
        /// 'shortcutName' that is mapped
        /// to the specified web siteUrl.
        /// If the directory already exists - does nothing
        /// </summary>
        /// <param name="siteUrl">The target url</param>
        /// <param name="shortcutContainerPath">Folder where
        ///           the shortcut folder mapped to
        /// the specified web siteUrl will be created</param>
        /// <param name="shortcutName">Name of the 
        ///          shortcut folder</param>
        public static string CreateWebFolderLink(string siteUrl, 
               string shortcutContainerPath, string shortcutName)
        {
            if (siteUrl.Length > 255)
                throw new ArgumentOutOfRangeException("Length" + 
                      " of Url [" + siteUrl + "] is more" + 
                      " than 255 symbols");

            string localFolderPath = 
              Path.Combine(shortcutContainerPath, shortcutName);

            DirectoryInfo localDir = null;
            if (Directory.Exists(localFolderPath))
                return localFolderPath;

            localDir = Directory.CreateDirectory(localFolderPath);
            localDir.Attributes = FileAttributes.ReadOnly;

            string desktop_ini_filePath = 
              Path.Combine(localFolderPath, "Desktop.ini");
            StreamWriter writer = new 
              StreamWriter(desktop_ini_filePath, false);
            writer.WriteLine("[.ShellClassInfo]");
            writer.WriteLine("CLSID2={0AFACED1" + 
                   "-E828-11D1-9187-B532F1E9575D}");
            writer.WriteLine("Flags=2");
            writer.WriteLine("ConfirmFileOp=0");
            writer.Close();

            string lnk_filePath = 
              Path.Combine(localFolderPath, "target.lnk");
            FileStream fs = new FileStream(lnk_filePath, 
                                       FileMode.Create);
            BinaryWriter stream = new BinaryWriter(fs, 
                           Encoding.BigEndianUnicode);
            stream.Write(LINK_CONTENT_PREFIX, 0, 
                         LINK_CONTENT_PREFIX.Length);

            //'This byte is 00 if the URL is 44 characters 
            //or less, 01 if greater.
            if (siteUrl.Length > URL_CUTOFF)
            {
                stream.Write((byte)0x01);
            }
            else
            {
                stream.Write((byte)0x00);
            }

            stream.Write(LINK_CONTENT_MID, 0, 
                         LINK_CONTENT_MID.Length);

            //'This byte is 00 if the URL is 44 characters 
            //or less, 01 if greater.
            if (siteUrl.Length > URL_CUTOFF)
            {
                stream.Write((byte)0x01);
            }
            else
            {
                stream.Write((byte)0x00);
            }

            stream.Write(LINK_CONTENT_MID2, 0, 
                         LINK_CONTENT_MID2.Length);

            //The next byte represents the length of the site name.
            stream.Write((byte)shortcutName.Length);

            char[] webFolderNameChars = 
                         shortcutName.ToCharArray();
            stream.Write(webFolderNameChars, 0, 
                         webFolderNameChars.Length);

            //Middle line, separates the Folder Name 
            //from the URL. 3 bytes.
            stream.Write((byte)0x00);
            stream.Write((byte)0x00);
            stream.Write((byte)0x00);

            //The next byte represents the length of the site URL.
            stream.Write((byte)siteUrl.Length);

            char[] sitePathChars = siteUrl.ToCharArray();
            stream.Write(sitePathChars, 0, sitePathChars.Length);

            stream.Write(LINK_CONTENT_POSTFIX, 0, 
                         LINK_CONTENT_POSTFIX.Length);

            stream.Close();

            return localFolderPath;
        }
    }
}

By the way, you may use the CreateWebFolderLink method to create the necessary web folder shortcuts to be visible in the “My Network Places” folder; for that, just pass the “c:\Documents and Settings\{USER NAME}\NetHood” string as the shortcutContainerPath parameter.

So now, when one wants to get a list of files in a web folder, the algorithm will be as follows:

• We create a temporary shortcut to the web folder.
• We use Shell32 to read its contents.

Create a temporary shortcut to the web folder:

/// <summary>
/// Returns path to a shortcut folder that is mapped
/// to the specified web siteUrl. Name of the folder is derived
/// automatically.
/// If the shortcut folder already exists - does nothing
/// </summary>
/// <param name="siteUrl"></param>
/// <param name="shortcutContainerPath"></param>
/// <returns></returns>
public static string CreateWebFolderLink(Uri siteUrl)
{
    //derive shortcut name
    string sitePath = siteUrl.ToString();
    string shortcutName = 
           sitePath.GetHashCode().ToString();

    return CreateWebFolderLink(sitePath, 
           Path.GetTempPath(), shortcutName);
}

Use Shell32 to read contents:

In order that the following code is compiled, you need to add a COM reference to the Shell32 library (its caption is “Microsoft Shell Controls and Automation”) to your project.

/// <summary>
/// Uses Shell32
/// </summary>
/// <param name="siteUrl"></param>
/// <param name="filePathPattern">For example,
///        to make a wildcard search ".doc" use the following
/// reg expression: Regex regex = new Regex(@".*\.doc",
///         RegexOptions.Compiled | RegexOptions.IgnoreCase);
/// to exclude word template documents assigned
/// to a SharePoint workspace, use the following 
/// reg expression: new Regex(@"(?<!.*/Forms/[^/]+)\.doc$", 
///          RegexOptions.Compiled | RegexOptions.IgnoreCase)</param>
/// <param name="searchSubFolders"></param>
/// <returns></returns>
public static StringCollection 
       SearchFilesInWebFolder(Uri siteUrl, 
       Regex filePathPattern, bool searchSubFolders)
{
    string mapFolderPath = CreateWebFolderLink(siteUrl);
    
    StringCollection ret = new StringCollection();

    Shell32.ShellClass shell = new Shell32.ShellClass();

    Shell32.Folder mapFolder = shell.NameSpace(mapFolderPath);
    SearchInFolder(mapFolder, filePathPattern, searchSubFolders, ret);

    return ret;
}

private static void SearchInFolder(Shell32.Folder folder, 
        Regex filePathPattern, bool searchSubFolders, 
        StringCollection resultList)
{
    foreach (Shell32.FolderItem item in folder.Items())
    {
        if (item.IsLink)
            continue;

        if (item.IsFolder && searchSubFolders)
        {
            SearchInFolder((Shell32.Folder) item.GetFolder, 
                filePathPattern, searchSubFolders, resultList);
            continue;
        }

        if (filePathPattern.IsMatch(item.Path))
        {
            resultList.Add(item.Path);
        }  
    }
}

Conclusion

The suggested solution seems to be quite simple and feasible for Windows or console applications. However, for server applications, use it with caution – you may likely face some problems with security (I am a bit doubtful whether an “ASP.NET” user would be able to use the Shell32 COM Interop) and multithreading. For such circumstances, I think the best approach would still be to use Web Services or WebDAV/WEC protocols directly.