Click here to Skip to main content
13,151,514 members (40,741 online)
Click here to Skip to main content
Add your own
alternative version


120 bookmarked
Posted 28 Feb 2006

Seekafile Server 1.0 - Flexible open-source search server

, 8 Mar 2006
Rate this:
Please Sign up or sign in to vote.
A Windows Service that indexes DOC, PDF, XLS, PPT, RTF, HTML, TXT, XML, and other file formats. Desktop and ASP.NET search samples included.

Sample ASP.NET client

Sample Windows Forms client


It has never been easier to create applications with search capabilities - open-source DotLucene [] allows building powerful and super-fast full-text search applications. Moreover, it's easy to use. Let's demonstrate it by exploring Seekafile Server [] - a flexible indexing server with capabilities similar to that of Windows Indexing Service [].

This article is a follow-up of Desktop Search Application: Part 1. In that article, I have discussed indexing and searching Office document using DotLucene []. This time we will build a more serious application that can be either used directly or as a studying material for practical usage of DotLucene.

In this article, you will learn:

  • How to perform indexing in the background.
  • How to update documents in DotLucene index.
  • How to create queries for DotLucene programmatically.
  • How to use IFilter to parse Office documents, Adobe PDF and other file types correctly (it includes the updated parsing code from Desktop Search Application: Part 1).


Seekafile Server [] is a Windows service that indexes documents in the specified directories and watches them for changes.

  • Background indexing
    • The indexer runs as a Windows service.
    • You specify the directories to be watched for changes in the configuration file.
    • Indexer works on the background (it doesn't slow down other operations).
    • It recognizes any change within a second.
  • Powered by DotLucene []
    • Super-fast searching.
    • The index is stored in DotLucene/Lucene 1.3+ compatible format.
    • The index can be accessed directly from other applications (you can search even when the indexing is in progress).
    • Access the index from any custom application (ASP.NET, Windows Forms application, Java application).
  • Built-in support for common file formats:
    • Microsoft PowerPoint (PPT)
    • Microsoft Word (DOC)
    • Microsoft Excel (XLS)
    • Text files (TXT)
    • Rich Text Format (RTF)
  • Supports custom plug-ins written in C# or VB.NET.
  • Supports IFilter for searching other extensions:
    • Adobe Acrobat (PDF)
    • Microsoft Visio (VSD)
    • XML
    • and other...
  • Runs on Windows 2000/XP/2003

How it works


The is the overview of the architecture:

The architecture is index-centric. It uses the index to communicate with the client search applications. The index is flexible enough to allow this:

  • It is possible to search the index while the Seekafile Server is modifying it.
  • There can be multiple clients accessing the index simultaneously.
  • The changes are visible immediately to all the clients.
  • The only information clients need to know is the index location and the available DotLucene document fields [].
  • The index is compatible with the Java version - you can access it from a Java client as well.

Watching changes

This is an overview of the indexing process:

  1. When the service is started it checks whether the index was already created at the specified location; if not it creates a new one:
    if (!IndexReader.IndexExists(cfg.IndexPath))
        Log.Echo("Creating a new index");
        IndexWriter writer = new IndexWriter(cfg.IndexPath, 
                              new StandardAnalyzer(), true);
  2. It goes through all the indexed directories and adds all the files to the IndexerQueue (to ensure that everything is indexed properly):
    foreach (string folder in cfg.Items)
  3. It starts the FileSystemWatcher to watch all file changes in the indexed directories:
    private void startWatcher(string directory)
        watcher = new FileSystemWatcher();
        watcher.Path = directory;
        watcher.NotifyFilter = NotifyFilters.LastWrite | 
                                 NotifyFilters.FileName | 
        watcher.IncludeSubdirectories = true;
        watcher.Filter = "";
        watcher.Changed += new FileSystemEventHandler(OnChanged);
        watcher.Created += new FileSystemEventHandler(OnChanged);
        watcher.Deleted += new FileSystemEventHandler(OnChanged);
        watcher.Renamed += new RenamedEventHandler(OnRenamed);
        // start watching
        watcher.EnableRaisingEvents = true;
  4. If there is a change event, it adds the file to the IndexerQueue:
    private void OnChanged(object source, FileSystemEventArgs e)
        // skip directory changes if it's not a name change
        if (Directory.Exists(e.FullPath) && 
                e.ChangeType == WatcherChangeTypes.Changed)
    private void OnRenamed(object source, RenamedEventArgs e)


The IndexerQueue works this way:

  1. It works in a separate thread. There is only a single thread processing a single queue at any moment:
    public static void Start()
        if (instanceDirectory == null)
            throw new ApplicationException("You must " + 
               "initialize the queue first by calling Init().");
        lock (runningLock)
            if (!isRunning)
                indexerThread = new Thread(new ThreadStart(Run));
                indexerThread.Name = "Indexer";
  2. It processes the items from the queue. It waits if there is nothing in the queue:
    while (!shouldStop)
        if (nextPath != null)
            // process nextPath
            // ...
            // remove it from the list
            lock (items.SyncRoot) 
        // nothing to do, let the processor do something else
        // try to take a next item
        nextPath = next();
  3. If the path is a directory, it goes through it and adds all its content to the queue:
    private static void parseDirectory(DirectoryInfo di)
        foreach (FileInfo f in di.GetFiles())
            Add(f.FullName, false);
        foreach (DirectoryInfo d in di.GetDirectories())
  4. If the path does not exist, it deletes it from the index (deleteDocuments) including all subfiles if there are any (deleteDirectory):
    private static void deleteDocuments(string fullName)
        IndexReader r = IndexReader.Open(instanceDirectory);
        int deletedCount = r.Delete(new Term("fullname", fullName));
    private static void deleteDirectory(string fullName)
        IndexReader r = IndexReader.Open(instanceDirectory);
        int deletedCount = r.Delete(new Term("parent", fullName));
  5. If the path is already in the index, it checks whether there is any change in file length, creation time, or last write time. To check whether the document is in the index, we create a query programmatically using BooleanQuery and TermQuery classes:
    private static bool isInIndex(FileInfo fi)
        IndexSearcher searcher = new IndexSearcher(instanceDirectory);
        BooleanQuery bq = new BooleanQuery();
        bq.Add(new TermQuery(new Term("fullname", 
                             fi.FullName)), true, false);
        bq.Add(new TermQuery(new Term("length", 
                             fi.Length.ToString())), true, false);
        bq.Add(new TermQuery(new Term("created", 
               DateField.DateToString(fi.CreationTime))), true, false);
        bq.Add(new TermQuery(new Term("modified", 
              DateField.DateToString(fi.LastWriteTime))), true, false);
        Hits hits = searcher.Search(bq);
        int count = hits.Length();
        return count == 1;
  6. If there are changes it updates the document in the index. Updating requires deleting the old document and adding a new one:
    // updates are expensive - proceed only if the 
    // file is not up-to-date
    if (isInIndex(fi))
    // delete all existing document with this name
    // add the document again
  7. When adding a document, we record the following metadata:
    • name: file name, e.g. document.doc,
    • fullname: path, e.g. c:\storage\marketing\document.doc,
    • parent: all parent directories, inserted as multiple fields, e.g. c:\; c:\storage; c:\storage\marketing,
    • created: creation time,
    • modified: last write time,
    • length: file length in bytes,
    • extension: file extensions, e.g. .doc.
    Document doc = new Document();
    doc.Add(new Field("name", fi.Name, true, true, true));
    doc.Add(new Field("fullname", fi.FullName, true, 
                                            true, false));
    DirectoryInfo di = fi.Directory;
    while (di != null)
        doc.Add(new Field("parent", di.FullName, true, 
                                            true, false));
        di = di.Parent;
    doc.Add(Field.Keyword("length", fi.Length.ToString()));
    doc.Add(Field.Keyword("extension", fi.Extension));

Parsing the files

DotLucene is able to index only plain text. Therefore, we need to extract the plain text from the rich file formats like Microsoft Word DOC, RTF, or Adobe PDF. The parsing can be done using a .NET plug-in found in the plugins subdirectory of the Seekafile Server or by IFilter interface (which is available in all Windows 2000/XP/2003 installations).

Read more about IFilter:


Generally, there are two ways of extending the parsing system:

Read more about custom plug-ins:

There is also a sample plug-in included in Seekafile Server download [].

Sample ASP.NET client search application

This ASP.NET application accesses the index directly to search it. It searches the file content only (file and directory names are ignored). It shows a relevant snippet from the document.

Read more about building an ASP.NET client search application [].

Download [] this sample as a part of the Seekafile Server from

Sample Windows Forms client search application

This Windows Forms application accesses the index directly to search it. It searches the file content only (file and directory names are ignored).

Read more about building a Windows Forms client search application [].

Download [] this sample as a part of the Seekafile Server from

Features planned for next versions

  • Exclude filters.
  • Multiple indexes per service.
  • Windows Forms client search application.
  • Simple GUI management.
  • Convenient installer.
  • Indexing status and notification support.
  • Multi-user desktop search.



This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


About the Author

Dan Letecky
Czech Republic Czech Republic

You may also be interested in...

Comments and Discussions is expired! Pin
kv40009-Apr-09 17:07
memberkv40009-Apr-09 17:07 
QuestionCan I create multiple indexes on different folders? Pin
CPShameem30-Dec-08 8:40
memberCPShameem30-Dec-08 8:40 
GeneralIs it possible to search in the SharePoint Document using this Pin
LoveConsultant3-Dec-08 19:51
memberLoveConsultant3-Dec-08 19:51 
Generalseek a file server Pin
venky45628-Oct-08 20:31
membervenky45628-Oct-08 20:31 
QuestionSeekafile Server 1.5 beta 3 Pin
chinna7921-Oct-08 5:44
memberchinna7921-Oct-08 5:44 
QuestionHow to index my own local or web site Pin
xmindx30-Sep-08 5:22
memberxmindx30-Sep-08 5:22 
AnswerRe: How to index my own local or web site Pin
Dan Letecky30-Sep-08 22:52
memberDan Letecky30-Sep-08 22:52 
GeneralRe: How to index my own local or web site Pin
xmindx2-Oct-08 12:00
memberxmindx2-Oct-08 12:00 
GeneralI don't understand how the project works Pin
alisson_abreu22-Sep-08 4:14
memberalisson_abreu22-Sep-08 4:14 
GeneralRe: I don't understand how the project works Pin
Dan Letecky22-Sep-08 8:02
memberDan Letecky22-Sep-08 8:02 
GeneralBooleanQuery Pin
AntonioFc31-May-08 5:00
memberAntonioFc31-May-08 5:00 
Questionweb sample seraches c: temp only why ? Pin
xmindx16-May-08 1:52
memberxmindx16-May-08 1:52 
QuestionSearch Scanned Image PDF Pin
kapilmalhotra25-Nov-07 19:58
memberkapilmalhotra25-Nov-07 19:58 
AnswerRe: Search Scanned Image PDF Pin
ubgunner9-Sep-09 13:14
memberubgunner9-Sep-09 13:14 
GeneralDisplay title, subject of file word, pdf.. in properties-summary file Pin
tuanlib6-Jul-07 17:41
membertuanlib6-Jul-07 17:41 
Questionwaste space about the meta information?! Pin
gobr1-Jun-07 20:16
membergobr1-Jun-07 20:16 
QuestionIndexing CitekNet filters Pin
Sairaj Sunil14-Dec-06 16:33
memberSairaj Sunil14-Dec-06 16:33 
QuestionDisplay web path instead of directory path Pin
BurdekinSC17-Oct-06 20:36
memberBurdekinSC17-Oct-06 20:36 
AnswerRe: Display web path instead of directory path Pin
kevdelkevdel5-Jan-07 13:11
memberkevdelkevdel5-Jan-07 13:11 
QuestionIs it possible to index a network shared folder Pin
Sairaj Sunil23-Sep-06 20:46
memberSairaj Sunil23-Sep-06 20:46 
AnswerRe: Is it possible to index a network shared folder Pin
Dan Letecky24-Sep-06 6:23
memberDan Letecky24-Sep-06 6:23 
QuestionRe: Is it possible to index a network shared folder [modified] Pin
Sairaj Sunil24-Sep-06 19:27
memberSairaj Sunil24-Sep-06 19:27 
AnswerRe: Is it possible to index a network shared folder Pin
Dan Letecky2-Oct-06 20:43
memberDan Letecky2-Oct-06 20:43 
QuestionRe: Is it possible to index a network shared folder [modified] Pin
Sairaj Sunil2-Oct-06 20:50
memberSairaj Sunil2-Oct-06 20:50 
AnswerRe: Is it possible to index a network shared folder Pin
Jason( J.Zhang)16-Jul-07 0:03
memberJason( J.Zhang)16-Jul-07 0:03 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web03 | 2.8.170924.2 | Last Updated 8 Mar 2006
Article Copyright 2006 by Dan Letecky
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid