<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
<title>Spirit.Lex Tutorials Overview</title>
<link rel="stylesheet" href="../../../../../../../doc/src/boostbook.css" type="text/css">
<meta name="generator" content="DocBook XSL Stylesheets V1.75.0">
<link rel="home" href="../../../index.html" title="Spirit 2.4">
<link rel="up" href="../tutorials.html" title="Spirit.Lex Tutorials">
<link rel="prev" href="../tutorials.html" title="Spirit.Lex Tutorials">
<link rel="next" href="lexer_quickstart1.html" title="Quickstart 1 - A word counter using Spirit.Lex">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<table cellpadding="2" width="100%"><tr>
<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../boost.png"></td>
<td align="center"><a href="../../../../../../../index.html">Home</a></td>
<td align="center"><a href="../../../../../../../libs/libraries.htm">Libraries</a></td>
<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
<td align="center"><a href="../../../../../../../more/index.htm">More</a></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="../tutorials.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="lexer_quickstart1.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
<div class="section">
<div class="titlepage"><div><div><h4 class="title">
<a name="spirit.lex.tutorials.lexer_tutorials"></a><a class="link" href="lexer_tutorials.html" title="Spirit.Lex Tutorials Overview"> <span class="emphasis"><em>Spirit.Lex</em></span>
Tutorials Overview</a>
</h4></div></div></div>
<p>
The <span class="emphasis"><em>Spirit.Lex</em></span> library implements several components
on top of possibly different lexer generator libraries. It exposes a pair
of iterators, which, when dereferenced, return a stream of tokens generated
from the underlying character stream. The generated tokens are based on
the token definitions supplied by the user.
</p>
<p>
Currently, <span class="emphasis"><em>Spirit.Lex</em></span> is built on top of Ben Hansons
excellent <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a>
library (which is a proposed Boost library). <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a>
provides the necessary functionality to build state machines based on a
set of supplied regular expressions. But <span class="emphasis"><em>Spirit.Lex</em></span>
is not restricted to be used with <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a>.
We expect it to be usable in conjunction with any other lexical scanner
generator library, all what needs to be implemented is a set of wrapper
objects exposing a well defined interface as described in this documentation.
</p>
<div class="note"><table border="0" summary="Note">
<tr>
<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../images/note.png"></td>
<th align="left">Note</th>
</tr>
<tr><td align="left" valign="top"><p>
For the sake of clarity all examples in this documentation assume <span class="emphasis"><em>Spirit.Lex</em></span>
to be used on top of <a href="http://www.benhanson.net/lexertl.html" target="_top">Lexertl</a>.
</p></td></tr>
</table></div>
<p>
Building a lexer using <span class="emphasis"><em>Spirit.Lex</em></span> is highly configurable,
where most of this configuration is done at compile time. Almost all of
the configurable parameters have generally useful default values, allowing
project startup to be a easy and straightforward task. Here is a (non-complete)
list of features you can tweak to adjust the generated lexer instance to
the actual needs:
</p>
<div class="itemizedlist"><ul class="itemizedlist" type="disc">
<li class="listitem">
Select and customize the token type to be generated by the lexer instance.
</li>
<li class="listitem">
Select and customize the token value types the generated token instances
will be able to hold.
</li>
<li class="listitem">
Select the iterator type of the underlying input stream, which will
be used as the source for the character stream to tokenize.
</li>
<li class="listitem">
Customize the iterator type returned by the lexer to enable debug support,
special handling of certain input sequences, etc.
</li>
<li class="listitem">
Select the <span class="emphasis"><em>dynamic</em></span> or the <span class="emphasis"><em>static</em></span>
runtime model for the lexical analyzer.
</li>
</ul></div>
<p>
Special care has been taken during the development of the library that
optimal code will be generated regardless of the configuration options
selected.
</p>
<p>
The series of tutorial examples of this section will guide you through
some common use cases helping to understand the big picture. The first
two quick start examples (<a class="link" href="lexer_quickstart1.html" title="Quickstart 1 - A word counter using Spirit.Lex">Lex
Quickstart 1 - A word counter using <span class="emphasis"><em>Spirit.Lex</em></span></a>
and <a class="link" href="lexer_quickstart2.html" title="Quickstart 2 - A better word counter using Spirit.Lex">Lex Quickstart
2 - A better word counter using <span class="emphasis"><em>Spirit.Lex</em></span></a>)
introduce the <span class="emphasis"><em>Spirit.Lex</em></span> library while building two
standalone applications, not being connected to or depending on any other
part of <a href="http://boost-spirit.com" target="_top">Spirit</a>. The section
<a class="link" href="lexer_quickstart3.html" title="Quickstart 3 - Counting Words Using a Parser">Lex Quickstart 3
- Counting Words Using a Parser</a> demonstrates how to use a lexer
in conjunction with a parser (where obviously the parser is built using
<span class="emphasis"><em>Spirit.Qi</em></span>).
</p>
</div>
<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
<td align="left"></td>
<td align="right"><div class="copyright-footer">Copyright © 2001-2010 Joel de Guzman, Hartmut Kaiser<p>
Distributed under the Boost Software License, Version 1.0. (See accompanying
file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
</p>
</div></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="../tutorials.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../tutorials.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="lexer_quickstart1.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
</body>
</html>
