Click here to Skip to main content
14,937,644 members
Articles / Programming Languages / C#
Posted 4 Nov 2002


79 bookmarked

RTools.Util - C# Library

Rate me:
Please Sign up or sign in to vote.
4.71/5 (16 votes)
26 Oct 20042 min read
A C# Library for tokenizing, argument parsing, and file finding.


RTools.Util is a small C# library containing some utility classes. The primary classes are:

  • StreamTokenizer: A stream tokenizer like Java's which can be useful for parsing text. This takes text (from a file, string, etc) in a stream and produces a list of Token objects. The Token objects are subclassed to provide type information. Some Token subclasses are WordToken, CharToken, WhitespaceToken, QuoteToken, CommentToken, FloatToken, etc.
  • Opts: A command-line option parsing class similar to the perl/unix getopts.
  • Finder: A small "find" utility class for producing lists of files under a particular directory. I created this to use in zipping up this package.
  • HighResClock: A wrapper for Kernel32.dll's high resolution clock API. This allows you time information with resolution greater than DateTime.Now's, which is machine dependent but typically 10 or 15 ms. The effective resolution of HighResClock on a P4 1.4GHz is about 10us.
  • SoftwarePackage: This class provides some utility methods to do things list installed software packages, etc. This is Windows-only.

Using the code

I've included an Ndoc-generated .chm file in the source download. It contains a lot more information. The information and samples here are a subset of what's in the .chm.

Here are a couple example uses of StreamTokenizer which illustrate the basics:

// one Token at a time
StreamTokenizer tokenizer = new StreamTokenizer();
tokenizer.TextReader = File.OpenText(fileName);
tokenizer.GrabWhitespace = true;
tokenizer.Verbosity = VerbosityLevel.Debug; // just for debugging
Token token;
while (NextToken(out token))
     if (token == '{') // do something

This use of StreamTokenizer shows all-at-once tokenization, which trades off performance (tokenize everything even if you aren't going to use all the tokens, and memory) for ease of programming:

// all at once Tokenize
StreamTokenizer tokenizer = new StreamTokenizer();
tokenizer.Settings.ParseNumbers = true;
ArrayList tokens = new ArrayList();
if (!tokenizer.TokenizeString("some string", tokens)) 
    // error handling
foreach (Token t in tokens) Console.WriteLine("t = {0}", t);

Here's an example use of Opts:

Opts opts = new Opts();
opts.ParseSpec = "file=s, type:s, v, d");
 - or -
opts.UsageSpec = "-file fileName -type [typeName] [-v|-d]";
if (!opts.Parse(args, out errorMessage)) { // display error message and usage }
if (opts.Options.ContainsKey("v")) // -v was specified
if (!opts.Options.ContainsKey("file")) { error... // need -file specified }
Console.WriteLine("-file specified is {0}", opts.Options["file"]);

Finder has a Main() which I use for a simple command-line find. Here's an example use of Finder's other methods:

// setup args
string dirName = ".";
ArrayList regexps = new ArrayList();
SortedList list = new SortedList();

// do the find
Finder.FindFiles(dirName, regexps, ref list, false);

// display the results
foreach(DictionaryEntry entry in list)
    string s = (string)entry.Key;

Points of Interest

There's some chance that the version on is more up to date.

I had high hopes of keeping StreamTokenizer's tokenization state machine very clean, but the number (floating point) parse forced me into some complexity. I'd be happy to hear better strategies.

It might be better to use a parser-generator (I see that Mono has one, and there's another article on CodeProject that used it) for parsing. The main advantage of StreamTokenizer , I believe, is that there's a smaller learning curve to use it.

I haven't included an AssemblyInfo.cs file that doesn't reference my private key (.snk) file. I am not including my private key file, and therefore the project I've included won't build as is. You have to remove the reference to my .snk file from AssemblyInfo.cs to get it to build. Sorry, I don't like the idea of having multiple .csproj and AssemblyInfo.cs files, and haven't found a more elegant solution.


  • Nov 5 2002 - First CodeProject submission.
  • Nov 16 2002 - First CodeProject download.
  • Jan 27 2003 - Update.
  • Oct 22 2004 - Update library to latest version (see changelog.txt in the download). Update this web page accordingly.


This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


About the Author

Ryan Seghers
United States United States
No Biography provided

Comments and Discussions

GeneralHighResClock.Now may need adjustment Pin
wlgrinfeld@yahoo.com13-Sep-06 3:31
Memberwlgrinfeld@yahoo.com13-Sep-06 3:31 
GeneralRe: HighResClock.Now may need adjustment Pin
Ryan Seghers27-Sep-06 12:12
MemberRyan Seghers27-Sep-06 12:12 
Questionfile open/close problem? Pin
K-Bob30-Aug-06 3:49
MemberK-Bob30-Aug-06 3:49 
AnswerRe: file open/close problem? Pin
Ryan Seghers27-Sep-06 12:02
MemberRyan Seghers27-Sep-06 12:02 
GeneralUsing Other packages in C# Pin
R.Premkumar1-Feb-06 18:45
MemberR.Premkumar1-Feb-06 18:45 
GeneralAnother option if you're doing command line parsing Pin
Member 11297421-Dec-04 23:34
MemberMember 11297421-Dec-04 23:34 
Generalthere have no words in "RTools.Utils.chm" file Pin
powerfulchen21-Dec-03 18:03
Memberpowerfulchen21-Dec-03 18:03 
GeneralRe: there have no words in "RTools.Utils.chm" file Pin
Frankie Leung28-Oct-04 18:12
MemberFrankie Leung28-Oct-04 18:12 
Generalthanks for great job! Pin
objet2117-Mar-03 15:17
Memberobjet2117-Mar-03 15:17 
GeneralJust what I needed Pin
dog_spawn10-Mar-03 13:18
Memberdog_spawn10-Mar-03 13:18 
GeneralThanks again Pin
dog_spawn22-Aug-03 7:22
Memberdog_spawn22-Aug-03 7:22 
GeneralFailure to compile... Pin
RSYOUNG16-Feb-03 3:57
MemberRSYOUNG16-Feb-03 3:57 
GeneralRe: Failure to compile... Pin
Ryan Seghers16-Feb-03 16:42
MemberRyan Seghers16-Feb-03 16:42 
GeneralRe: Failure to compile... Pin
RSYOUNG17-Feb-03 2:45
MemberRSYOUNG17-Feb-03 2:45 
GeneralGreat job! Pin
Luis Alonso Ramos27-Jan-03 18:15
MemberLuis Alonso Ramos27-Jan-03 18:15 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.