Click here to Skip to main content
13,197,548 members (45,057 online)
Click here to Skip to main content
Add your own
alternative version


82 bookmarked
Posted 8 Jan 2009

Implementing a TextReader to extract various files contents using IFilter

, 9 Feb 2011
Rate this:
Please Sign up or sign in to vote.
A solution that can extract various file contents using an IFilter implementation. Special thanks to Eyal Post and his article 'Using IFilter in C# '.


The IFilter interface is an important component for the Microsoft Indexing Service. The intent of this project is to provide a solution which is high performing and also has a low memory footprint. This will be accomplished by using a TextReader and manipulating a wrapper of IFilter to interop with managed code.

The article is written to target users who have had development experience with COM in a managed environment, and is aimed at creating a solution that can extract various file contents using an IFilter implementation.

The solution in the source code package contains a test project, which includes a Unit Test and a load test. These were created using a higher edition of Visual Studio. If you cannot open the test project, you can always remove the IFilterTest project.


The IFilter component is an in-process COM server that extracts the text and values for a specific file type. The appropriate IFilter component for a file type is called by the Filtering component.

A customized IFilter component can be developed for almost any selected file type. The standard IFilter components supplied with Indexing Service include the following:

File Name



Filters Multipurpose Internet Mail Extension (MIME) files.


Filters HTML 3.0 or earlier files.


Filters Microsoft Office files: Microsoft Word, Microsoft Excel, and Microsoft PowerPoint®.


Filters plain text files (default filter) and binary files (null filter).

The IFilter interface scans documents for text and properties (also called attributes). It extracts chunks of text from these documents, filtering out embedded formatting and retaining information about the position of the text. It also extracts chunks of values, which are properties of an entire document or of well-defined parts of a document. IFilter provides the foundation for building higher-level applications such as document indexers and application-independent viewers.


Main Classes Diagram

IFilter Interface and Mixed Class

The IFilter code:

[ComImport, Guid(Constants.IFilterGUID), 
    SuppressUnmanagedCodeSecurity, ComVisible(true), AutomationProxy(false)]
public interface IFilter
    /// <summary>
    /// The IFilter::Init method initializes a filtering session.
    /// </summary>
                MethodCodeType = MethodCodeType.Runtime)]
    IFilterReturnCodes Init(
        //[in] Flag settings from the IFILTER_INIT enumeration for
        // controlling text standardization, property output, embedding
        // scope, and IFilter access patterns. 
        [MarshalAs(UnmanagedType.U4)]IFILTER_INIT grfFlags,
        // [in] The size of the attributes array. When nonzero, cAttributes
        //  takes 
        // precedence over attributes specified in grfFlags. If no
        // attribute flags 
        // are specified and cAttributes is zero, the default is given by
        // the 
        // PSGUID_STORAGE storage property set, which contains the date and
        //  time 
        // of the last write to the file, size, and so on; and by the
        //  PID_STG_CONTENTS 
        // 'contents' property, which maps to the main contents of the
        // file. 
        // For more information about properties and property sets, see
        // Property Sets. 
        uint cAttributes,
        //[in] Array of pointers to FULLPROPSPEC structures for the
        // requested properties. 
        // When cAttributes is nonzero, only the properties in aAttributes
        // are returned. 
        FULLPROPSPEC[] aAttributes,
        // [out] Information about additional properties available to the
        //  caller; from the IFILTER_FLAGS enumeration. 
        out IFILTER_FLAGS pdwFlags);
    /// <summary>
    /// The IFilter::GetChunk method positions the filter at the beginning
    /// of the next chunk, 
    /// or at the first chunk if this is the first call to the GetChunk
    /// method, and returns a description of the current chunk. 
    /// </summary>
     MethodCodeType = MethodCodeType.Runtime)]
    IFilterReturnCodes GetChunk(out STAT_CHUNK pStat);
    /// <summary>
    /// The IFilter::GetText method retrieves text (text-type properties)
    /// from the current chunk, 
    /// which must have a CHUNKSTATE enumeration value of CHUNK_TEXT.
    /// </summary>
     MethodCodeType = MethodCodeType.Runtime)]
    IFilterReturnCodes GetText(
        // [in/out] On entry, the size of awcBuffer array in wide/Unicode
        // characters. On exit, the number of Unicode characters written to
        // awcBuffer. 
        // Note that this value is not the number of bytes in the buffer. 
        ref uint pcwcBuffer,
        // Text retrieved from the current chunk. Do not terminate the
        // buffer with a character.
        [Out]IntPtr awcBuffer);
    /// <summary>
    /// The IFilter::GetValue method retrieves a value (public
    /// value-type property) from a chunk, 
    /// which must have a CHUNKSTATE enumeration value of CHUNK_VALUE.
    /// </summary>
     MethodCodeType = MethodCodeType.Runtime)]
    IFilterReturnCodes GetValue(
        // Allocate the PROPVARIANT structure with CoTaskMemAlloc. Some
        // PROPVARIANT 
        // structures contain pointers, which can be freed by calling the
        // PropVariantClear function. 
        // It is up to the caller of the GetValue method to call the
        //   PropVariantClear method.
        // ref IntPtr ppPropValue
        // [MarshalAs(UnmanagedType.Struct)]
        out PROPVARIANT PropVal);
    /// <summary>
    /// The IFilter::BindRegion method retrieves an interface representing
    /// the specified portion of the object. 
    /// Currently reserved for future use.
    /// </summary>
     MethodCodeType = MethodCodeType.Runtime)]
    IFilterReturnCodes BindRegion(ref FILTERREGION origPos,ref Guid riid,
        ref object ppunk);

The mixed class:

public class MixedIFilterClass : IFilterClass, IDisposable
    public override string TmpFilePath
    public override Object InternalObj
    //private MixedIFilterClass()
    //    InternalPtr = Marshal.GetComInterfaceForObject(this, typeof(IFilter));
    protected virtual void Dispose(bool disposing)
        if(null != InternalObj)
            InternalObj = null;
        if (null != TmpFilePath)
                TmpFilePath = null;
            catch { }
        if (disposing)
    public void Dispose()

How it Works

There are two steps needed to show how the process works They are:

  1. Get the current chunk
  2. Call GetText() on the chunk

Step 1: Get the current chunk

If you reach the last chunk, terminate the reading process.

var returnCode = _filter.GetChunk(out chunk);

Step 2: Call GetText() on the chunk

Depending on the state gotten from the GetChunk method, call the GetText method on the text chunk. When reading the end of current chunk flag, repeat step 1.

while (true)
    if (remaining <= _topSize)
    bool useBuffer = !forceDirectlyWrite && remaining < BufferSize;
    var size = BufferSize;
    if (useBuffer)
        size -= _topSize;
        if (remaining < BufferSize)
            size = (uint)remaining;
    if (size < ResBufSize)
        size = ResBufSize;
    var handle = GCHandle.Alloc(useBuffer ? _buffer : array,
    var ptr = Marshal.UnsafeAddrOfPinnedArrayElement(
        useBuffer ? _buffer : array, useBuffer ? (int)_topSize : offset);
    IFilterReturnCodes returnCode;
        returnCode = _filter.GetText(ref size, ptr);
    if(returnCode != IFilterReturnCodes.FILTER_E_NO_TEXT)
        if (useBuffer)
            _topSize += size;
            offset += (int)size;
            remaining -= (int)size;
        if(_topSize > BufferSize)
            _resTopSize = _topSize - BufferSize;
            _topSize = BufferSize;
    if (returnCode == IFilterReturnCodes.FILTER_S_LAST_TEXT || 
        returnCode == IFilterReturnCodes.FILTER_E_NO_MORE_TEXT ||
        (returnCode == IFilterReturnCodes.FILTER_E_NO_TEXT && size != 0) ||
        (null == FileName && IgnoreError && returnCode == 
        _endOfCurrChunk = true;
        if (remaining <= _topSize)
    if(returnCode != IFilterReturnCodes.S_OK)
        throw new Exception(
            "a error occur when getting text by current filter",
            new Exception(returnCode.ToString()));

Using the Code

The following code uses just a filename:

var fileName = "";
using (var reader = new FilterReader(fileName))
    // write your code here;

This code will specify a file and an extension:

using (var reader = new FilterReader(fileName, ".docx"))
    // write your code here;
using (var reader = new FilterReader(fileName, 0x1000))
    // write your code here;

The code below shows how to pass a byte array into a FilterReader:

byte[] bytes = null;
using (var reader = new FilterReader(bytes, ".docx"))
    // write your code here;
using (var reader = new FilterReader(bytes, ".docx", 0x1000))
    // write your code here;
using (var reader = new FilterReader(bytes))
    // write your code here;



  • 2/9/2010 - Updated download files.
  • 3/03/2009 - v2.0: Replaced Copyright and added OLE FMTIDs, Windows Search Schema, and OLE property interfaces for further needs.
  • 2/05/2009 - v2.0: Reconstructed some phrases (thanks Sean Kenney for reviewing this article).
  • 1/07/2009 - v2.0: Added comments for Adobe's PDF filter and small changes.
  • 12/09/2008 - v2.0 [Stable release].
  • 11/24/2008 - v1.0 [Initial release].


This article, along with any associated source code and files, is licensed under The Eclipse Public License 1.0


About the Author

Technical Lead HP
China China
No Biography provided

You may also be interested in...


Comments and Discussions

QuestionException when trying to use this to read the provided sample PDF Pin
Brian C Hart3-Sep-17 10:42
memberBrian C Hart3-Sep-17 10:42 
BugException from HRESULT : 0x8004170C Pin
Member 1096235812-Oct-14 5:11
memberMember 1096235812-Oct-14 5:11 
GeneralRe: Exception from HRESULT : 0x8004170C Pin
gerfasan2323-Apr-15 6:51
membergerfasan2323-Apr-15 6:51 
SuggestionSupport for PDF file indexing Pin
Daniel Kornev4-May-14 14:03
memberDaniel Kornev4-May-14 14:03 
GeneralRe: Support for PDF file indexing Pin
vladimir-v7721-Sep-14 23:39
membervladimir-v7721-Sep-14 23:39 
QuestionURGENT: Unable to extract text from docx, xlsx files Pin
Shiva Wahi16-Oct-13 20:23
memberShiva Wahi16-Oct-13 20:23 
AnswerRe: URGENT: Unable to extract text from docx, xlsx files Pin
vladimir-v773-Oct-14 2:19
membervladimir-v773-Oct-14 2:19 
QuestionUnable to retrieve content from .pdf, .docx, .xlsx... Pin
RicardoRomão1-Jul-13 7:44
memberRicardoRomão1-Jul-13 7:44 
AnswerRe: Unable to retrieve content from .pdf, .docx, .xlsx... Pin
santoshthankachan28-Aug-13 19:54
membersantoshthankachan28-Aug-13 19:54 
QuestionLoading from a Stream? Pin
Matt Johnson14-Jan-13 3:25
memberMatt Johnson14-Jan-13 3:25 
BugPeek() still doesn't work Pin
Member 874096323-May-12 4:13
memberMember 874096323-May-12 4:13 
BugFrequent use of FilterReader() breaks it Pin
Snæbjørn23-Apr-12 1:59
memberSnæbjørn23-Apr-12 1:59 
QuestionUnable to extract content from rtf file (Rich Text Format) Pin
ajayk.gupta20-Feb-12 22:21
memberajayk.gupta20-Feb-12 22:21 
AnswerRe: Unable to extract content from rtf file (Rich Text Format) Pin
Snæbjørn23-Apr-12 4:16
memberSnæbjørn23-Apr-12 4:16 
AnswerRe: Unable to extract content from rtf file (Rich Text Format) Pin
vladimir-v7712-Dec-14 4:41
membervladimir-v7712-Dec-14 4:41 
GeneralMy vote of 5 Pin
manoj kumar choubey9-Feb-12 22:15
membermanoj kumar choubey9-Feb-12 22:15 
GeneralMy vote of 5 Pin
Jonathan Manley29-Jun-11 9:19
memberJonathan Manley29-Jun-11 9:19 
QuestionReading document properties by using the IFilter Pin
Ramkrishnan Seerangasamy1-Jun-11 0:07
memberRamkrishnan Seerangasamy1-Jun-11 0:07 
GeneralMy vote of 5 Pin
alessandro7216-Feb-11 23:59
memberalessandro7216-Feb-11 23:59 
This solution is clear, complete and structured according to Microsoft GuideLine.
It's an excellent example to learn IFilter object
Generala bug while text extracting from "docx" Pin
Emre Özgür İnce11-Feb-11 2:40
memberEmre Özgür İnce11-Feb-11 2:40 
GeneralRe: a bug while text extracting from "docx" Pin
alex_zero11-Feb-11 18:58
memberalex_zero11-Feb-11 18:58 
GeneralRe: a bug while text extracting from "docx" Pin
Emre Özgür İnce13-Feb-11 20:36
memberEmre Özgür İnce13-Feb-11 20:36 
GeneralRe: a bug while text extracting from "docx" Pin
alex_zero14-Feb-11 17:23
memberalex_zero14-Feb-11 17:23 
GeneralRe: a bug while text extracting from "docx" Pin
Emre Özgür İnce14-Feb-11 23:29
memberEmre Özgür İnce14-Feb-11 23:29 
QuestionPeek Fails on some file types - Any Ideas? Pin
Erra erra eric14-Dec-10 11:30
memberErra erra eric14-Dec-10 11:30 
AnswerRe: Peek Fails on some file types - Any Ideas? Pin
alex_zero6-Feb-11 22:50
memberalex_zero6-Feb-11 22:50 
GeneralRe: Peek Fails on some file types - Any Ideas? Pin
Erra erra eric8-Mar-11 10:52
memberErra erra eric8-Mar-11 10:52 
GeneralCreating an instance of the COM component from the IClassFactory failed Pin
belikemike12129-Jul-10 12:55
memberbelikemike12129-Jul-10 12:55 
GeneralRe: Creating an instance of the COM component from the IClassFactory failed Pin
alex_zero15-Aug-10 4:25
memberalex_zero15-Aug-10 4:25 
GeneralRe: Creating an instance of the COM component from the IClassFactory failed Pin
alex_zero6-Feb-11 22:53
memberalex_zero6-Feb-11 22:53 
QuestionCan I specify which IFilter to use? Pin
DThrasher1-Mar-10 7:19
memberDThrasher1-Mar-10 7:19 
GeneralDoes not work for DOCX files on 64-bit Windows Pin
ykara14-Jan-10 18:07
memberykara14-Jan-10 18:07 
GeneralRe: Does not work for DOCX files on 64-bit Windows Pin
Tom Miller8531-Jan-11 7:14
memberTom Miller8531-Jan-11 7:14 
GeneralRe: Does not work for DOCX files on 64-bit Windows Pin
alex_zero6-Feb-11 23:08
memberalex_zero6-Feb-11 23:08 
GeneralRe: Does not work for DOCX files on 64-bit Windows Pin
Martin_Dann19-May-11 2:31
memberMartin_Dann19-May-11 2:31 
QuestionWhy not use the LoadIFilter function to get IFilter? Pin
rohit_lodha15-Jul-09 21:23
memberrohit_lodha15-Jul-09 21:23 
Generalvery cool stuff but i have some problems with html\xml files Pin
ickeoderwat1-Jul-09 8:56
memberickeoderwat1-Jul-09 8:56 
GeneralDoes not load filter for .docx file under Vista Pin
hendi610-Apr-09 11:34
memberhendi610-Apr-09 11:34 
AnswerRe: Does not load filter for .docx file under Vista Pin
hendi67-May-09 6:36
memberhendi67-May-09 6:36 
GeneralComparision to Eyal Post article [modified] Pin
Petr Felzmann4-Mar-09 21:46
memberPetr Felzmann4-Mar-09 21:46 
GeneralRe: Comparision to Eyal Post article Pin
alex_zero5-Mar-09 15:18
memberalex_zero5-Mar-09 15:18 
QuestionIs there version for visual studio 2005? Pin
fsben3-Feb-09 5:25
memberfsben3-Feb-09 5:25 
AnswerRe: Is there version for visual studio 2005? Pin
alex_zero5-Mar-09 15:32
memberalex_zero5-Mar-09 15:32 
QuestionWhat about properties of *.doc Pin
tprakash12-Jan-09 23:22
membertprakash12-Jan-09 23:22 
AnswerRe: What about properties of *.doc [modified] Pin
alex_zero5-Mar-09 15:57
memberalex_zero5-Mar-09 15:57 
GeneralGood job Pin
wooting12-Jan-09 21:49
memberwooting12-Jan-09 21:49 
GeneralRe: Good job Pin
alex_zero5-Mar-09 15:33
memberalex_zero5-Mar-09 15:33 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.171020.1 | Last Updated 9 Feb 2011
Article Copyright 2009 by alex_zero
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid