Sometimes you need to use
strtok in MFC projects, but it has
two big limitations:
- Can't use it to tokenize more than one string at the time.
- It's not easy to use safely with
To overcome these limitations I wrote
It's quite easy to use as the following code shows:
CTokenizer tok(_T("A-B+C*D-E"), _T("-+"));
TRACE2("Token: '%s', Tail: '%s'\n", (LPCTSTR)cs, (LPCTSTR)tok.Tail());
The preceding code produces the following output:
Token: 'A', Tail: 'B+C*D-E'
Token: 'B', Tail: 'C*D-E'
Token: 'C*D', Tail: 'E'
Token: 'E', Tail: ''
As you can see, it's no rocket science! It's a very simple and handy class
- Create an object of type
CTokenizer, passing as arguments the
string to tokenize and the delimiters
Next() until it returns
- If you need to change the delimiters, call
- If you need the un-tokenized part of the string, call
- 10/22/2001 Using
std::bitset instead of
Reduces memory requirements and increases performance.
- 10/22/2001 Fixed bug related to ASCII characters above 127. Reported by John Simpsons