<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
<title>Lexer</title>
<link rel="stylesheet" href="../../../../../../../../doc/src/boostbook.css" type="text/css">
<meta name="generator" content="DocBook XSL Stylesheets V1.75.0">
<link rel="home" href="../../../../index.html" title="Spirit 2.4">
<link rel="up" href="../lexer_concepts.html" title="Lexer Concepts">
<link rel="prev" href="../lexer_concepts.html" title="Lexer Concepts">
<link rel="next" href="primitivelexer.html" title="PrimitiveLexer">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<table cellpadding="2" width="100%"><tr>
<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../../boost.png"></td>
<td align="center"><a href="../../../../../../../../index.html">Home</a></td>
<td align="center"><a href="../../../../../../../../libs/libraries.htm">Libraries</a></td>
<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
<td align="center"><a href="../../../../../../../../more/index.htm">More</a></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="../lexer_concepts.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../lexer_concepts.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="primitivelexer.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
<div class="section">
<div class="titlepage"><div><div><h5 class="title">
<a name="spirit.lex.reference.lexer_concepts.lexer"></a><a class="link" href="lexer.html" title="Lexer">Lexer</a>
</h5></div></div></div>
<a name="spirit.lex.reference.lexer_concepts.lexer.description"></a><h6>
<a name="id1187053"></a>
<a class="link" href="lexer.html#spirit.lex.reference.lexer_concepts.lexer.description">Description</a>
</h6>
<p>
The <span class="emphasis"><em>Lexer</em></span> is the most fundamental concept. A Lexer
has a member function, <code class="computeroutput"><span class="identifier">collect</span></code>,
that accepts a token definition container <code class="computeroutput"><span class="identifier">Def</span></code>,
and a the name of the lexer state the token definitions of the lexer
component need to be added to (a string). It doesn't return anything
(return type is <code class="computeroutput"><span class="keyword">void</span></code>). Each
Lexer can represent a specific pattern or algorithm, or it can be a more
complex lexer component formed as a composition of other Lexer's. Additionally,
a Lexer exposes a member <code class="computeroutput"><span class="identifier">add_actions</span></code>,
that accepts the token definition container <code class="computeroutput"><span class="identifier">Def</span></code>,
while returning nothing (again, the returned type is <code class="computeroutput"><span class="keyword">void</span></code>).
</p>
<div class="variablelist">
<p class="title"><b>Notation</b></p>
<dl>
<dt><span class="term"><code class="computeroutput"><span class="identifier">l</span></code></span></dt>
<dd><p>
A <code class="computeroutput"><span class="identifier">Lexer</span></code>.
</p></dd>
<dt><span class="term"><code class="computeroutput"><span class="identifier">L</span></code></span></dt>
<dd><p>
A <code class="computeroutput"><span class="identifier">Lexer</span></code> type.
</p></dd>
<dt><span class="term"><code class="computeroutput"><span class="identifier">Def</span></code></span></dt>
<dd><p>
A token definition conatiner type.
</p></dd>
<dt><span class="term"><code class="computeroutput"><span class="identifier">State</span></code></span></dt>
<dd><p>
A type used to represent lexer state names.
</p></dd>
</dl>
</div>
<a name="spirit.lex.reference.lexer_concepts.lexer.valid_expressions"></a><h6>
<a name="id1187219"></a>
<a class="link" href="lexer.html#spirit.lex.reference.lexer_concepts.lexer.valid_expressions">Valid
Expressions</a>
</h6>
<p>
In the expressions below, the behavior of the lexer component, <code class="computeroutput"><span class="identifier">l</span></code>, is left unspecified in the base
<code class="computeroutput"><span class="identifier">Lexer</span></code> concept. These
are specified in subsequent, more refined concepts and by the actual
models thereof.
</p>
<p>
For any Lexer the following expressions must be valid:
</p>
<div class="informaltable"><table class="table">
<colgroup>
<col>
<col>
<col>
</colgroup>
<thead><tr>
<th>
<p>
Expression
</p>
</th>
<th>
<p>
Semantics
</p>
</th>
<th>
<p>
Return type
</p>
</th>
</tr></thead>
<tbody>
<tr>
<td>
<p>
<code class="computeroutput"><span class="identifier">l</span><span class="special">.</span><span class="identifier">collect</span><span class="special">(</span><span class="identifier">def</span><span class="special">,</span>
<span class="identifier">state</span><span class="special">)</span></code>
</p>
</td>
<td>
<p>
Add all token definitions provided by this Lexer instance to
the lexer state <code class="computeroutput"><span class="identifier">state</span></code>
of the token definition container <code class="computeroutput"><span class="identifier">def</span></code>.
</p>
</td>
<td>
<p>
<code class="computeroutput"><span class="keyword">void</span></code>
</p>
</td>
</tr>
<tr>
<td>
<p>
<code class="computeroutput"><span class="identifier">l</span><span class="special">.</span><span class="identifier">add_actions</span><span class="special">(</span><span class="identifier">def</span><span class="special">)</span></code>
</p>
</td>
<td>
<p>
Add all semantic actions provided by this Lexer instance to
the token definition container <code class="computeroutput"><span class="identifier">def</span></code>.
</p>
</td>
<td>
<p>
<code class="computeroutput"><span class="keyword">void</span></code>
</p>
</td>
</tr>
</tbody>
</table></div>
<a name="spirit.lex.reference.lexer_concepts.lexer.type_expressions"></a><h6>
<a name="id1187447"></a>
<a class="link" href="lexer.html#spirit.lex.reference.lexer_concepts.lexer.type_expressions">Type
Expressions</a>
</h6>
<div class="informaltable"><table class="table">
<colgroup>
<col>
<col>
</colgroup>
<thead><tr>
<th>
<p>
Expression
</p>
</th>
<th>
<p>
Description
</p>
</th>
</tr></thead>
<tbody><tr>
<td>
<p>
<code class="computeroutput"><span class="identifier">traits</span><span class="special">::</span><span class="identifier">is_lexer</span><span class="special"><</span><span class="identifier">L</span><span class="special">>::</span><span class="identifier">type</span></code>
</p>
</td>
<td>
<p>
Metafunction that evaluates to <code class="computeroutput"><span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span></code>
if a certain type, <code class="computeroutput"><span class="identifier">L</span></code>
is a Lexer, <code class="computeroutput"><span class="identifier">mpl</span><span class="special">::</span><span class="identifier">false_</span></code>
otherwise (See <a href="../../../../../../../../libs/mpl/doc/refmanual/integral-constant.html" target="_top">MPL
Boolean Constant</a>).
</p>
</td>
</tr></tbody>
</table></div>
<a name="spirit.lex.reference.lexer_concepts.lexer.postcondition"></a><h6>
<a name="id1187587"></a>
<a class="link" href="lexer.html#spirit.lex.reference.lexer_concepts.lexer.postcondition">Postcondition</a>
</h6>
<p>
Upon return from <code class="computeroutput"><span class="identifier">l</span><span class="special">.</span><span class="identifier">collect</span></code> the following post conditions
should hold:
</p>
<div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem">
On return, <code class="computeroutput"><span class="identifier">def</span></code> holds
all token definitions defined in the Lexer, <code class="computeroutput"><span class="identifier">l</span></code>.
This includes all Lexer's contained inside <code class="computeroutput"><span class="identifier">l</span></code>.
</li></ul></div>
<p>
Upon return from <code class="computeroutput"><span class="identifier">l</span><span class="special">.</span><span class="identifier">add_actions</span></code> the following post conditions
should hold:
</p>
<div class="itemizedlist"><ul class="itemizedlist" type="disc"><li class="listitem">
On return, <code class="computeroutput"><span class="identifier">def</span></code> holds
all semantic actions correctly asociated with the corresponding token
definitions as defined in the Lexer, <code class="computeroutput"><span class="identifier">l</span></code>.
This includes all semantic actiosn defined by the Lexer's contained
inside <code class="computeroutput"><span class="identifier">l</span></code>.
</li></ul></div>
<a name="spirit.lex.reference.lexer_concepts.lexer.models"></a><h6>
<a name="id1187702"></a>
<a class="link" href="lexer.html#spirit.lex.reference.lexer_concepts.lexer.models">Models</a>
</h6>
<p>
All lexer components in <span class="emphasis"><em>Spirit.Lex</em></span> are models of
the <span class="emphasis"><em>Lexer</em></span> concept.
</p>
</div>
<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
<td align="left"></td>
<td align="right"><div class="copyright-footer">Copyright © 2001-2010 Joel de Guzman, Hartmut Kaiser<p>
Distributed under the Boost Software License, Version 1.0. (See accompanying
file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
</p>
</div></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="../lexer_concepts.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../lexer_concepts.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="primitivelexer.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
</body>
</html>
