<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<title>The Lexer Interface</title>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<link href="theme/style.css" rel="stylesheet" type="text/css">
</head>
<body>
<table width="100%" border="0" cellspacing="2" background="theme/bkd2.gif">
<tr>
<td width="21"> <h1></h1></td>
<td width="885"> <font face="Verdana, Arial, Helvetica, sans-serif"><b><font size="6">The
Lexer Interface</font></b></font></td>
<td width="96"><a href="http://spirit.sf.net"><img src="theme/wave.gif" width="93" height="68" align="right" border="0"></a></td>
</tr>
</table>
<br>
<table border="0">
<tr>
<td width="10"></td>
<td width="30"><a href="index.html"><img src="theme/u_arr.gif" border="0"></a></td>
<td width="30"><a href="class_reference_tracepolicy.html"><img src="theme/l_arr.gif" width="20" height="19" border="0"></a></td>
<td width="20"><a href="class_reference_tokentype.html"><img src="theme/r_arr.gif" border="0"></a></td>
</tr>
</table>
<blockquote>
<p><a href="class_reference_lexer.html#introduction">Introduction</a><br>
<a href="class_reference_lexer.html#header_synopsis">Header 'wave/cpplexer/cpp_lexer_interface.hpp'
synopsis</a><br>
<a href="class_reference_lexer.html#template_parameters">Template parameters</a><br>
<a href="class_reference_lexer.html#member_functions">Member functions</a></p>
</blockquote>
<h2><b><a name="introduction"></a>Introduction</b></h2>
<p>The <tt>wave::cpplexer::lex_interface</tt> template is designed to be used
as a base class for any C++ lexer, which should be plugged in into the Wave
preprocessor engine. It is an abstract base class, which allows to abstract
the lexer itself from the other parts of the library. Additionally it contains
a static functions, through which a new instance of the lexer object should
be instantiated. For this to function correctly you need to supply an implementation
for the <tt>new_lexer_gen</tt> template, which actually is used to generate
a new instance of your lexer object. This lexer object is expected to have a
constructor taking three parameters: the begin and end iterators of the underlying
input stream to analyse and the file_position of the point in the input stream,
which corresponds to the first of the given iterators.</p>
<h2><a name="header_synopsis"></a>Header <a href="http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/spirit/spirit/wave/wave/cpplexer/cpp_lex_interface.hpp?rev=HEAD&content-type=text/vnd.viewcvs-markup">wave/cpplexer/cpp_lexer_interface.hpp</a>
synopsis</h2>
<pre><span class="keyword">namespace</span> wave {
<span class="keyword">namespace</span> cpplexer {
<span class="keyword">template</span> <<span class="keyword">typename</span> IteratorT, <span class="keyword">typename</span> PositionT>
<span class="keyword">struct</span> new_lexer_gen
{
<span class="keyword">static</span> lex_input_interface<lex_token<PositionT> > *
<a href="class_reference_lexer.html#new_lexer_gen_new_lexer">new_lexer</a>(IteratorT <span class="keyword">const</span> &first, IteratorT <span class="keyword">const</span> &last,
PositionT <span class="keyword">const</span> &pos);<br> };
<span class="keyword">template</span> <<span class="keyword">typename</span> TokenT>
<span class="keyword">struct</span> lex_input_interface
{
<span class="keyword">typedef typename</span> TokenT::position_t position_t;
<span class="keyword">virtual</span> TokenT <a href="class_reference_lexer.html#get">get</a>()<span class="keyword"> = 0</span>;
<span class="keyword">virtual void</span> <a href="class_reference_lexer.html#set_position">set_position</a>(position_t const &pos)<span class="keyword"> = 0</span>;
<span class="keyword">virtual</span> ~<a href="class_reference_lexer.html#destructor">lex_input_interface</a>() {}
<span class="comment">// The new_lexer function allows the opaque generation of </span>
<span class="comment">// a new lexer object. It is coupled to the token type </span>
<span class="comment">// to allow to distinguish different lexer/token
// configurations at compile time.</span>
<span class="keyword">template</span> <<span class="keyword">typename</span> IteratorT>
<span class="keyword">static</span> lex_input_interface *
new_lexer(IteratorT <span class="keyword">const</span> &first, IteratorT <span class="keyword">const</span> &last,
position_t const &pos)
{
<span class="keyword">return</span> new_lexer_gen<IteratorT, position_t>::
new_lexer (first, last, pos);
}
}
};
} <span class="comment">// namespace cpplexer</span>
} <span class="comment">// namespace wave</span>
</pre>
<h2><a name="template_parameters"></a>Template parameters</h2>
<p>The <tt>lex_input_interface</tt> itself needs the token type to be supplied
as the only template parameter. It specifies the token type to be returned by
the actual C++ lexer. The predefined C++ lexers inside the <tt>Wave</tt> library
use the lex_token template (described <a href="class_reference_tokentype.html">here</a>)
for this purpose.</p>
<p>The <tt>new_lexer_gen</tt> structure needs to be implemented besides the actual
lexer object. It takes two template parameters: the iterator type of the underlying
input stream to use and the type of the <tt>file_position</tt> struct to be
used by the C++ lexer. If the second parameter is omitted, it should default
to the <tt>PositionT</tt> template parameter used by the token type.</p>
<h2><a name="member_functions"></a>Member functions</h2>
<p><a name="destructor"></a><b>destructor</b></p>
<pre> <span class="keyword">virtual</span> ~lex_input_interface();</pre>
<blockquote>
<p>The destructor is defined as <tt>virtual</tt> to garant the correct destruction
of the derived lexers. This is very important to avoid memory leaks.</p>
</blockquote>
<p><b><a name="get"></a>get</b></p>
<pre><span class="keyword"> virtual</span> TokenT get();</pre>
<blockquote>
<p>This function is called, whenever the next token from the input stream is
needed. It should return this token, if it is available otherwise the <tt>T_EOF</tt>
token (see <a href="token_ids.html">The Token Identifiers</a>), signaling
the end of the underlying input stream.</p>
</blockquote>
<p><b><a name="set_position"></a>set_position</b></p>
<pre><span class="keyword"> virtual void</span> set_position(
position_t const &pos)<span class="keyword">;</span></pre>
<blockquote>
<p>This function is called, whenever the current position of the input stream
has to be changed. Note, that this function should <b>not</b> move the iterators
of the input stream, it should instead store the new position and take it
as the reference point for all subsequent generated tokens. For instance this
function is called , whenever a valid <tt>#line</tt> directive is recognized
in the input stream. </p>
</blockquote>
<p><a name="new_lexer_gen_new_lexer"></a><b>new_lexer_gen::new_lexer</b></p>
<pre><span class="keyword"> static</span> lex_input_interface<TokenT> *
new_lexer(IteratorT <span class="keyword">const</span> &first, IteratorT <span class="keyword">const</span> &last,
PositionT <span class="keyword">const</span> &pos);</pre>
<blockquote>
<p> </p>
</blockquote>
<table border="0">
<tr>
<td width="10"></td>
<td width="30"><a href="index.html"><img src="theme/u_arr.gif" border="0"></a></td>
<td width="30"><a href="class_reference_tracepolicy.html"><img src="theme/l_arr.gif" width="20" height="19" border="0"></a></td>
<td width="20"><a href="class_reference_tokentype.html"><img src="theme/r_arr.gif" border="0"></a></td>
</tr>
</table>
<hr size="1">
<p class="copyright">Copyright © 2003 Hartmut Kaiser<br>
<br>
<font size="2">Permission to copy, use, modify, sell and distribute this document
is granted provided this copyright notice appears in all copies. This document
is provided "as is" without express or implied warranty, and with
no claim as to its suitability for any purpose. </font> </p>
<span class="updated">Last updated:
<!-- #BeginDate format:fcAm1m -->Monday, April 7, 2003 14:29<!-- #EndDate -->
</span></body>
</html>
Submit an article or tip