Also interesting question for me. This is surely not the optimal solution (in case I'm understanding it correctly), maybe something like a workaround or help with research.
I've read your link and especially the IFilter interface and ISearchCatalogManager are good points to give a try to test it.
I first tried it the manually way, so I've added a custom file type in Windows Registry \HKEY_CLASSES_ROOT\ , I've taken .tkz for example. You can see some sample settings in .txt file which contains Properties like this:
\HKEY_CLASSES_ROOT\.txt contains string (Standard)/(Default) with value txtfile,Content Type (text/plain) and perceived Type(text)
\HKEY_CLASSES_ROOT\.txt\OpenWithList -- to open .txt with named links (in my case Dreamweaver is linked there as string)
\HKEY_CLASSES_ROOT\.txt\OpenWithProgIds -- also "alternative" applications to open with (will be displayed in context menue when selecting a txt-file to open "Open with..."). In my case there are values from Visual Studio and Open Office.
\HEKY_CLASSES_ROOT\.txt\PersistandHandler --> Important because it holds the CLSID that represents a COM class and also a file name extension.
\HEKY_CLASSES_ROOT\.txt\ShellNew -- not so important for our case.
okay, so I've created a custom extension, then I've quickly used Windows search and searched for my extension *.tkz. After searching for it I had a look into
Windows indexing options (run control.exe srchadmin.dll) and my custom file was successfully indexed in Advanced->File Types.
It's a bit dirty, but it would be an easy task to insert a new extension in registry with strings and values and doing a short search with windows after this. I've don't tried it yet to do this programmatically.
A second point I got interested to test is trying to use the Index-Management-Interfaces with method AddCatalog(). Maybe it would be doing our above step to search after our custom file above needless.
Hope it drives forward your researching in any way.
With Best Regards