Click here to Skip to main content
Click here to Skip to main content

Source Code Super Search

, 10 Mar 2009 CPOL
Rate this:
Please Sign up or sign in to vote.
A simple solution for searching source code directories

This information pertains to the v 0.9 release. The latest release v 1.0 is discussed in the 03-2009 Updates Section.

Introduction

This tool is a simple search utility I developed to search all my source code, regardless of language. The tool integrates with Visual Studio 2008, so you can kickoff a search while writing code. The utility will search any directory for files that match the extensions specified, and it will match search strings (supports Regular Expressions) inside the matched files. Results are presented in a clean treeview format, and allow you to take various actions with the results.

Background

I spend around 80% of my day in front of a computer screen writing code for customers, and a lot of that time is spent digging through old code that I've written in the past, trying to remember how I solved a problem previously so I can implement the solution in a current project. I wish I could put a precise time value on this exercise, but the closest value I can come up with is a lot. I determined early on that I needed a GREP type application that I could use to search a large directory hierarchy containing thousands of source files. There are a lot of Windows Grep programs out there, and I tried a few of them (the ones that were free). Ultimately, it proved too tedious and cumbersome to use many of them for my needs, and there were some features I really wanted specific to Visual Studio IDE programmers, such as the ability to open a *.sln solution file for a matched code file, that just isn't possible in a switchblade (one size fits all) type Grep-ping application.

Visual Studio's Find In Files Option

Of course, Visual Studio already provides a robust search feature (CTRL+SHIFT+F) which allows you to search for patterns in files based on extensions and user defined search strings. The functionality of this tool mimics the functionality of the Visual Studio feature, but adds some additional options like the ability to (try to locate and) open the *.sln file associated to the matched code file, and the ability to open the folder containing the matched file. This tool can search and match patterns against file names as well as match patterns in files.

Thanks Drew Stainton for this tip: You can further customize the behavior of the Find in Files Visual Studio function by editing Visual Studio's Registry settings: Customize your Find in Files Results experience!

Usage Outside of Visual Studio

This solution plugs into Visual Studio as an added feature, so you can launch it while working within Visual Studio, but it's not dependent on Visual Studio in any way. It can be run on its own, and can be used for searching any code (or any file readable in Notepad, for that matter). I've used it quite a bit to search in VBScript, batch, and Perl scripts as well as for searching file server log files.

Project Goals

The ultimate goal of the project is to significantly reduce the effort required to look through code files. The solution needs to search the file system as fast as possible and present a simple output that the user can interact with.

In summary:

  • Find file names that match the specified extensions and search pattern
  • Find values inside the files that match the specified search pattern and the specified extension
  • Search needs to be as fast as possible
  • Solution needs to persist the user defined data between sessions

Matches need to allow these behaviors:

  • Open file
  • Open file in Notepad
  • Open containing folder
  • Open code file's owning solution file (*.sln)

To Index or Not To Index

This is a question I have toiled over for quite some time. If I index all the files in a location, then the search results would be instant. This would make users very happy, but it would add a lot of overhead to the solution. Ultimately, I decided, for this utility, every search would be real time, and all actions would be conducted at the time of the search instead of indexing prior to execution.

Search Methodology

There are a lot of ways we can perform a search. In the past, benchmarks seemed to indicate that using...

string[] myFiles = Directory.GetFiles
			(USER_PATH, FILEMASK, SearchOption.AllDirectories);

... was considerably slower than using the command line function:

c:\>dir /S /B USER_PATH\*.FILEMASK > ApplicationAppDataDirectory\cache.txt

However, I'm not so certain of this anymore. I have run a series of completely unscientific benchmarks on these two scenarios.

Scenario A: Command.com dir Function

  1. Run the dir command with Process.Start
  2. Use a StreamReader to match a RegEx pattern in each line of the output file
  3. Build a SearchResult object, List<T>
  4. Populate a TreeView control with the results
Path FileMasks Pattern SType Found Searched Time
c:\ .log [Ll]aunching In Files 58 1525 34 sec
VS2005\Projects .cs .vb DirectoryEntry In Files 70 3081 13 sec
VS2005\Projects .cs DirectoryInfo In Files 57 3037 7
\\baileyfs01\Music .mp3 Megadeth File Name 225 14,903 8

Scenario B : Directory.GetFiles

  1. Generate an output file with StreamWriter based on the Directory.GetFiles results
  2. Use a StreamReader to match a RegEx pattern in each line of the output file
  3. Build a SearchResult object List<T>
  4. Populate a TreeView control with the results
Path FileMasks Pattern SType Found Searched Time
c:\ .log [Ll]aunching In Files 58 1525 57 sec
VS2005\Projects .cs .vb DirectoryEntry In Files 70 3081 14 sec
VS2005\Projects .cs DirectoryInfo In Files 57 3037 7
\\baileyfs01\Music .mp3 Megadeth File Name 225 14,903 13

These benchmarks were run on a Vista machine with 3GB of RAM and a Q6600 Quad-Core Processor. As you can see from the statistics above, the results slightly favor running the DIR command over Directory.GetFiles().

However, in all fairness, it doesn't really make sense to have Directory.GetFiles write the results to a file since realistically, the results are already in an Array.

As it stands right now, I have depreciated the DIR search functionality while I'm testing all the ways to use GetFiles(). The version included in this article's deliverables isn't using DIR, though the classes are present to turn it back on for further benchmarks. Update: I have noticed from daily use that the first time the application is run each day takes a really, really long time to execute the first search but it has not been enough of a irritant to make me go in another direction.

The CommandLine engine is housed in a class called CommandLine.cs, and minus all the abstraction members (fields, properties, constructors, etc.), it looks like this:

private void DoShellExecute()
{
    ProcessStartInfo psi =
      new ProcessStartInfo(m_ApplicationPath, m_ApplicationParameters);
    psi.UseShellExecute = true;
    psi.RedirectStandardOutput = false;
    psi.CreateNoWindow = true;
    psi.WindowStyle = ProcessWindowStyle.Normal;

    if (ExecuteType == ExecutionType.SHELL_EXECUTE_HIDDEN)
        psi.WindowStyle = ProcessWindowStyle.Hidden;

    Process p_exec = new Process();

    try
    {
        p_exec = Process.Start(psi);

        if (m_TimeOutThreshold != null || m_TimeOutThreshold == 0)
        {
            p_exec.WaitForExit((int)m_TimeOutThreshold);
        }
        else
        {
            p_exec.WaitForExit();
        }
    }
    catch (SystemException e)
    {
        m_Error = e;
    }
    finally
    {
        p_exec.Close();
    }
}

On the other hand, the GetFiles() route doesn't need its own type, so we use it directly in the IOWorker class.

/// <summary>
/// Creates the index of the user defined root path
/// </summary>
public void CreateIdx()
{
    foreach (string f in m_uso.filters)
    {
        try
        {
            m_idx.AddRange(Directory.GetFiles(
                m_uso.path, "*" + f,
                SearchOption.AllDirectories));
        }
        catch (UnauthorizedAccessException e)
        {
            System.Diagnostics.Debug.WriteLine(e);
        }
    }
    //m_idx.Reverse();
    m_SearchedCount = m_idx.Count;
    SearchResults(m_uso.pattern);
}

Adding to Visual Studio 2005/2008

Included with this article is the source code as well as the binary application files. If you wish to add this application to the Tools menu in Visual Studio 2008, extract all the files in the VS2008 add-in zip file into the Addins folder in your Visual Studio user path (c:\Users\You\Visual Studio 2008\Addins).

As noted earlier, there is no dependency on Visual Studio, so you can just run the *.exe to run the program on its own.

~~ Update 03-2009 ~~

Regular Expressions

Recently I had a requirement where I needed to remove a code block from every page in a legacy web site. The web site was entirely composed of HTM, HTML, and PHP files and in total there were about 3000 of them! The code block looked like this:

<script language="javascript">
  var breakThis = "http://customerdomain/cfaq/browse.php"; //break frames
  if (window != parent) parent.location.href = breakThis;
  //-->
</script>

The previous build of the tool executed searches against each line in each file. In order to match strings that span multiple lines, I added an option to treat the entire FileStream as a single string. The new option Search all lines together, allows you to search entire files at once to match patterns through entire files. In addition, there is now an Ignore Case flag so searches are no longer case sensitive.

Replace In Files

Per the requirement above, I had to find the script block in all 3000+ web files and remove them. A new Replace feature is now available which allows you to replace a string matched in the RegEx search with a new string (or in my case, empty string).

Undo Replace In Files

When you initiate a Replace action, the original files (the ones that were matched in the search action) are backed up and given a *.orig extension. If you find that you shouldn't have allowed an automated Regex.Replace action against 3000 files, you can undo the replace action. The undo feature effectively deletes the modified files and restores the *.orig files to their previous state as the original files.

Reporting/Export

A new report feature is now available which allows you to export the results of each search.

Conclusion

To this point, I'm extremely pleased with the results of this project. I find myself using this search tool constantly throughout the day, and it makes my day to day tasks much easier. Having used it for several days now, I'm looking into indexing to a server, and the possibilities that may open up for finding code blocks.

History

  • 10/08/2008 - Article submitted
  • 10/08/2008 - Corrected incorrect statements regarding existing VS functionality
  • 03/09/2009 - Released v 1.0

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

thund3rstruck
Software Developer
United States United States
I'm a typical 30 year old generation X guy that likes video games, NFL football, and comic style art. I have an insatiable passion for programming and doing what ever it takes to become a better programmer.

Comments and Discussions

 
GeneralExcellent! but.... PinmemberDoncp19-Mar-09 13:40 
GeneralRe: Excellent! but.... Pinmemberthund3rstruck19-Mar-09 14:04 
Generalexcellent Pinmemberrodelizius11-Jan-09 18:36 
Generalgreat dev util2 Pinmemberrodelizius11-Jan-09 18:34 
GeneralRe: great dev util2 Pinmemberthund3rstruck11-Jan-09 19:15 
GeneralRe: great dev util2 [modified] Pinmemberthund3rstruck9-Mar-09 18:40 
General[Message Deleted] Pinmemberrodelizius11-Jan-09 18:01 
GeneralSimply fantastic, what a great idea. Pinmembermikehatlak16-Oct-08 6:37 
GeneralWhere was this a year ago? [modified] PinmemberJason Barry16-Oct-08 2:24 
GeneralRe: Where was this a year ago? Pinmemberthund3rstruck9-Mar-09 18:38 
Generalnice PinmemberUnruled Boy9-Oct-08 17:03 
GeneralVS Find In Files PinprotectorMarc Clifton8-Oct-08 9:45 
GeneralRe: VS Find In Files Pinmemberthund3rstruck8-Oct-08 10:21 
GeneralRe: VS Find In Files PinmemberDrew Stainton8-Oct-08 13:41 
GeneralRe: VS Find In Files Pinmemberthund3rstruck8-Oct-08 15:19 
GeneralRe: VS Find In Files PinmemberDrew Stainton8-Oct-08 16:25 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web02 | 2.8.141022.2 | Last Updated 10 Mar 2009
Article Copyright 2008 by thund3rstruck
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid