<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
<title>Parsers in Depth</title>
<link rel="stylesheet" href="../../../../../../../doc/src/boostbook.css" type="text/css">
<meta name="generator" content="DocBook XSL Stylesheets V1.75.0">
<link rel="home" href="../../../index.html" title="Spirit 2.4">
<link rel="up" href="../indepth.html" title="In Depth">
<link rel="prev" href="../indepth.html" title="In Depth">
<link rel="next" href="../customize.html" title="Customization of Spirit's Attribute Handling">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<table cellpadding="2" width="100%"><tr>
<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../boost.png"></td>
<td align="center"><a href="../../../../../../../index.html">Home</a></td>
<td align="center"><a href="../../../../../../../libs/libraries.htm">Libraries</a></td>
<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
<td align="center"><a href="../../../../../../../more/index.htm">More</a></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="../indepth.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../indepth.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="../customize.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
<div class="section">
<div class="titlepage"><div><div><h4 class="title">
<a name="spirit.advanced.indepth.parsers_indepth"></a><a class="link" href="parsers_indepth.html" title="Parsers in Depth"> Parsers in
Depth</a>
</h4></div></div></div>
<p>
This section is not for the faint of heart. In here, are distilled the
inner workings of <span class="emphasis"><em>Spirit.Qi</em></span> parsers, using real code
from the <a href="http://boost-spirit.com" target="_top">Spirit</a> library as
examples. On the other hand, here is no reason to fear reading on, though.
We tried to explain things step by step while highlighting the important
insights.
</p>
<p>
The <code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/parser.html" title="Parser"><code class="computeroutput"><span class="identifier">Parser</span></code></a></code> class is the base
class for all parsers.
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Derived</span><span class="special">></span>
<span class="keyword">struct</span> <span class="identifier">parser</span>
<span class="special">{</span>
<span class="keyword">struct</span> <span class="identifier">parser_id</span><span class="special">;</span>
<span class="keyword">typedef</span> <span class="identifier">Derived</span> <span class="identifier">derived_type</span><span class="special">;</span>
<span class="keyword">typedef</span> <span class="identifier">qi</span><span class="special">::</span><span class="identifier">domain</span> <span class="identifier">domain</span><span class="special">;</span>
<span class="comment">// Requirement: p.parse(f, l, context, skip, attr) -> bool
</span> <span class="comment">//
</span> <span class="comment">// p: a parser
</span> <span class="comment">// f, l: first/last iterator pair
</span> <span class="comment">// context: enclosing rule context (can be unused_type)
</span> <span class="comment">// skip: skipper (can be unused_type)
</span> <span class="comment">// attr: attribute (can be unused_type)
</span>
<span class="comment">// Requirement: p.what(context) -> info
</span> <span class="comment">//
</span> <span class="comment">// p: a parser
</span> <span class="comment">// context: enclosing rule context (can be unused_type)
</span>
<span class="comment">// Requirement: P::template attribute<Ctx, Iter>::type
</span> <span class="comment">//
</span> <span class="comment">// P: a parser type
</span> <span class="comment">// Ctx: A context type (can be unused_type)
</span> <span class="comment">// Iter: An iterator type (can be unused_type)
</span>
<span class="identifier">Derived</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">derived</span><span class="special">()</span> <span class="keyword">const</span>
<span class="special">{</span>
<span class="keyword">return</span> <span class="special">*</span><span class="keyword">static_cast</span><span class="special"><</span><span class="identifier">Derived</span> <span class="keyword">const</span><span class="special">*>(</span><span class="keyword">this</span><span class="special">);</span>
<span class="special">}</span>
<span class="special">};</span>
</pre>
<p>
</p>
<p>
The <code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/parser.html" title="Parser"><code class="computeroutput"><span class="identifier">Parser</span></code></a></code> class does not really
know how to parse anything but instead relies on the template parameter
<code class="computeroutput"><span class="identifier">Derived</span></code> to do the actual
parsing. This technique is known as the "Curiously Recurring Template
Pattern" in template meta-programming circles. This inheritance strategy
gives us the power of polymorphism without the virtual function overhead.
In essence this is a way to implement compile time polymorphism.
</p>
<p>
The Derived parsers, <code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/primitiveparser.html" title="PrimitiveParser"><code class="computeroutput"><span class="identifier">PrimitiveParser</span></code></a></code>, <code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/unaryparser.html" title="UnaryParser"><code class="computeroutput"><span class="identifier">UnaryParser</span></code></a></code>, <code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/binaryparser.html" title="BinaryParser"><code class="computeroutput"><span class="identifier">BinaryParser</span></code></a></code> and <code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/naryparser.html" title="NaryParser"><code class="computeroutput"><span class="identifier">NaryParser</span></code></a></code> provide the
necessary facilities for parser detection, introspection, transformation
and visitation.
</p>
<p>
Derived parsers must support the following:
</p>
<div class="variablelist">
<p class="title"><b>bool parse(f, l, context, skip, attr)</b></p>
<dl>
<dt><span class="term"><code class="computeroutput"><span class="identifier">f</span></code>, <code class="computeroutput"><span class="identifier">l</span></code></span></dt>
<dd><p>
first/last iterator pair
</p></dd>
<dt><span class="term"><code class="computeroutput"><span class="identifier">context</span></code></span></dt>
<dd><p>
enclosing rule context (can be unused_type)
</p></dd>
<dt><span class="term"><code class="computeroutput"><span class="identifier">skip</span></code></span></dt>
<dd><p>
skipper (can be unused_type)
</p></dd>
<dt><span class="term"><code class="computeroutput"><span class="identifier">attr</span></code></span></dt>
<dd><p>
attribute (can be unused_type)
</p></dd>
</dl>
</div>
<p>
The <span class="emphasis"><em>parse</em></span> is the main parser entry point. <span class="emphasis"><em>skipper</em></span>
can be an <code class="computeroutput"><span class="identifier">unused_type</span></code>.
It's a type used every where in <a href="http://boost-spirit.com" target="_top">Spirit</a>
to signify "don't-care". There is an overload for <span class="emphasis"><em>skip</em></span>
for <code class="computeroutput"><span class="identifier">unused_type</span></code> that is
simply a no-op. That way, we do not have to write multiple parse functions
for phrase and character level parsing.
</p>
<p>
Here are the basic rules for parsing:
</p>
<div class="itemizedlist"><ul class="itemizedlist" type="disc">
<li class="listitem">
The parser returns <code class="computeroutput"><span class="keyword">true</span></code>
if successful, <code class="computeroutput"><span class="keyword">false</span></code> otherwise.
</li>
<li class="listitem">
If successful, <code class="computeroutput"><span class="identifier">first</span></code>
is incremented N number of times, where N is the number of characters
parsed. N can be zero --an empty (epsilon) match.
</li>
<li class="listitem">
If successful, the parsed attribute is assigned to <span class="emphasis"><em>attr</em></span>
</li>
<li class="listitem">
If unsuccessful, <code class="computeroutput"><span class="identifier">first</span></code>
is reset to its position before entering the parser function. <span class="emphasis"><em>attr</em></span>
is untouched.
</li>
</ul></div>
<div class="variablelist">
<p class="title"><b>void what(context)</b></p>
<dl>
<dt><span class="term"><code class="computeroutput"><span class="identifier">context</span></code></span></dt>
<dd><p>
enclosing rule context (can be <code class="computeroutput"><span class="identifier">unused_type</span></code>)
</p></dd>
</dl>
</div>
<p>
The <span class="emphasis"><em>what</em></span> function should be obvious. It provides some
information about <span class="quote">“<span class="quote">what</span>”</span> the parser is. It is used as a debugging
aid, for example.
</p>
<div class="variablelist">
<p class="title"><b>P::template attribute<context>::type</b></p>
<dl>
<dt><span class="term"><code class="computeroutput"><span class="identifier">P</span></code></span></dt>
<dd><p>
a parser type
</p></dd>
<dt><span class="term"><code class="computeroutput"><span class="identifier">context</span></code></span></dt>
<dd><p>
A context type (can be unused_type)
</p></dd>
</dl>
</div>
<p>
The <span class="emphasis"><em>attribute</em></span> metafunction returns the expected attribute
type of the parser. In some cases, this is context dependent.
</p>
<p>
In this section, we will dissect two parser types:
</p>
<div class="variablelist">
<p class="title"><b>Parsers</b></p>
<dl>
<dt><span class="term"><code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/primitiveparser.html" title="PrimitiveParser"><code class="computeroutput"><span class="identifier">PrimitiveParser</span></code></a></code></span></dt>
<dd><p>
A parser for primitive data (e.g. integer parsing).
</p></dd>
<dt><span class="term"><code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/unaryparser.html" title="UnaryParser"><code class="computeroutput"><span class="identifier">UnaryParser</span></code></a></code></span></dt>
<dd><p>
A parser that has single subject (e.g. kleene star).
</p></dd>
</dl>
</div>
<a name="spirit.advanced.indepth.parsers_indepth.primitive_parsers"></a><h6>
<a name="id1196708"></a>
<a class="link" href="parsers_indepth.html#spirit.advanced.indepth.parsers_indepth.primitive_parsers">Primitive
Parsers</a>
</h6>
<p>
For our disection study, we will use a <a href="http://boost-spirit.com" target="_top">Spirit</a>
primitive, the <code class="computeroutput"><span class="identifier">int_parser</span></code>
in the boost::spirit::qi namespace.
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><</span>
<span class="keyword">typename</span> <span class="identifier">T</span>
<span class="special">,</span> <span class="keyword">unsigned</span> <span class="identifier">Radix</span> <span class="special">=</span> <span class="number">10</span>
<span class="special">,</span> <span class="keyword">unsigned</span> <span class="identifier">MinDigits</span> <span class="special">=</span> <span class="number">1</span>
<span class="special">,</span> <span class="keyword">int</span> <span class="identifier">MaxDigits</span> <span class="special">=</span> <span class="special">-</span><span class="number">1</span><span class="special">></span>
<span class="keyword">struct</span> <span class="identifier">int_parser_impl</span>
<span class="special">:</span> <span class="identifier">primitive_parser</span><span class="special"><</span><span class="identifier">int_parser_impl</span><span class="special"><</span><span class="identifier">T</span><span class="special">,</span> <span class="identifier">Radix</span><span class="special">,</span> <span class="identifier">MinDigits</span><span class="special">,</span> <span class="identifier">MaxDigits</span><span class="special">></span> <span class="special">></span>
<span class="special">{</span>
<span class="comment">// check template parameter 'Radix' for validity
</span> <span class="identifier">BOOST_SPIRIT_ASSERT_MSG</span><span class="special">(</span>
<span class="identifier">Radix</span> <span class="special">==</span> <span class="number">2</span> <span class="special">||</span> <span class="identifier">Radix</span> <span class="special">==</span> <span class="number">8</span> <span class="special">||</span> <span class="identifier">Radix</span> <span class="special">==</span> <span class="number">10</span> <span class="special">||</span> <span class="identifier">Radix</span> <span class="special">==</span> <span class="number">16</span><span class="special">,</span>
<span class="identifier">not_supported_radix</span><span class="special">,</span> <span class="special">());</span>
<span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Context</span><span class="special">,</span> <span class="keyword">typename</span> <span class="identifier">Iterator</span><span class="special">></span>
<span class="keyword">struct</span> <span class="identifier">attribute</span>
<span class="special">{</span>
<span class="keyword">typedef</span> <span class="identifier">T</span> <span class="identifier">type</span><span class="special">;</span>
<span class="special">};</span>
<span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Iterator</span><span class="special">,</span> <span class="keyword">typename</span> <span class="identifier">Context</span>
<span class="special">,</span> <span class="keyword">typename</span> <span class="identifier">Skipper</span><span class="special">,</span> <span class="keyword">typename</span> <span class="identifier">Attribute</span><span class="special">></span>
<span class="keyword">bool</span> <span class="identifier">parse</span><span class="special">(</span><span class="identifier">Iterator</span><span class="special">&</span> <span class="identifier">first</span><span class="special">,</span> <span class="identifier">Iterator</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">last</span>
<span class="special">,</span> <span class="identifier">Context</span><span class="special">&</span> <span class="comment">/*context*/</span><span class="special">,</span> <span class="identifier">Skipper</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">skipper</span>
<span class="special">,</span> <span class="identifier">Attribute</span><span class="special">&</span> <span class="identifier">attr</span><span class="special">)</span> <span class="keyword">const</span>
<span class="special">{</span>
<span class="identifier">qi</span><span class="special">::</span><span class="identifier">skip_over</span><span class="special">(</span><span class="identifier">first</span><span class="special">,</span> <span class="identifier">last</span><span class="special">,</span> <span class="identifier">skipper</span><span class="special">);</span>
<span class="keyword">return</span> <span class="identifier">extract_int</span><span class="special"><</span><span class="identifier">T</span><span class="special">,</span> <span class="identifier">Radix</span><span class="special">,</span> <span class="identifier">MinDigits</span><span class="special">,</span> <span class="identifier">MaxDigits</span><span class="special">></span>
<span class="special">::</span><span class="identifier">call</span><span class="special">(</span><span class="identifier">first</span><span class="special">,</span> <span class="identifier">last</span><span class="special">,</span> <span class="identifier">attr</span><span class="special">);</span>
<span class="special">}</span>
<span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Context</span><span class="special">></span>
<span class="identifier">info</span> <span class="identifier">what</span><span class="special">(</span><span class="identifier">Context</span><span class="special">&</span> <span class="comment">/*context*/</span><span class="special">)</span> <span class="keyword">const</span>
<span class="special">{</span>
<span class="keyword">return</span> <span class="identifier">info</span><span class="special">(</span><span class="string">"integer"</span><span class="special">);</span>
<span class="special">}</span>
<span class="special">};</span>
</pre>
<p>
</p>
<p>
The <code class="computeroutput"><span class="identifier">int_parser</span></code> is derived
from a <code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/primitiveparser.html" title="PrimitiveParser"><code class="computeroutput"><span class="identifier">PrimitiveParser</span></code></a><span class="special"><</span><span class="identifier">Derived</span><span class="special">></span></code>,
which in turn derives from <code class="computeroutput"><span class="identifier">parser</span><span class="special"><</span><span class="identifier">Derived</span><span class="special">></span></code>. Therefore, it supports the following
requirements:
</p>
<div class="itemizedlist"><ul class="itemizedlist" type="disc">
<li class="listitem">
The <code class="computeroutput"><span class="identifier">parse</span></code> member function
</li>
<li class="listitem">
The <code class="computeroutput"><span class="identifier">what</span></code> member function
</li>
<li class="listitem">
The nested <code class="computeroutput"><span class="identifier">attribute</span></code>
metafunction
</li>
</ul></div>
<p>
<span class="emphasis"><em>parse</em></span> is the main entry point. For primitive parsers,
our first thing to do is call:
</p>
<p>
</p>
<pre class="programlisting"><span class="identifier">qi</span><span class="special">::</span><span class="identifier">skip</span><span class="special">(</span><span class="identifier">first</span><span class="special">,</span> <span class="identifier">last</span><span class="special">,</span> <span class="identifier">skipper</span><span class="special">);</span>
</pre>
<p>
</p>
<p>
to do a pre-skip. After pre-skipping, the parser proceeds to do its thing.
The actual parsing code is placed in <code class="computeroutput"><span class="identifier">extract_int</span><span class="special"><</span><span class="identifier">T</span><span class="special">,</span> <span class="identifier">Radix</span><span class="special">,</span> <span class="identifier">MinDigits</span><span class="special">,</span> <span class="identifier">MaxDigits</span><span class="special">>::</span><span class="identifier">call</span><span class="special">(</span><span class="identifier">first</span><span class="special">,</span> <span class="identifier">last</span><span class="special">,</span> <span class="identifier">attr</span><span class="special">);</span></code>
</p>
<p>
This simple no-frills protocol is one of the reasons why <a href="http://boost-spirit.com" target="_top">Spirit</a>
is fast. If you know the internals of <a href="../../../../../../../libs/spirit/classic/index.html" target="_top"><span class="emphasis"><em>Spirit.Classic</em></span></a>
and perhaps even wrote some parsers with it, this simple <a href="http://boost-spirit.com" target="_top">Spirit</a>
mechanism is a joy to work with. There are no scanners and all that crap.
</p>
<p>
The <span class="emphasis"><em>what</em></span> function just tells us that it is an integer
parser. Simple.
</p>
<p>
The <span class="emphasis"><em>attribute</em></span> metafunction returns the T template
parameter. We associate the <code class="computeroutput"><span class="identifier">int_parser</span></code>
to some placeholders for <code class="computeroutput"><span class="identifier">short_</span></code>,
<code class="computeroutput"><span class="identifier">int_</span></code>, <code class="computeroutput"><span class="identifier">long_</span></code>
and <code class="computeroutput"><span class="identifier">long_long</span></code> types. But,
first, we enable these placeholders in namespace boost::spirit:
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><></span>
<span class="keyword">struct</span> <span class="identifier">use_terminal</span><span class="special"><</span><span class="identifier">qi</span><span class="special">::</span><span class="identifier">domain</span><span class="special">,</span> <span class="identifier">tag</span><span class="special">::</span><span class="identifier">short_</span><span class="special">></span> <span class="comment">// enables short_
</span> <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{};</span>
</pre>
<p>
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><></span>
<span class="keyword">struct</span> <span class="identifier">use_terminal</span><span class="special"><</span><span class="identifier">qi</span><span class="special">::</span><span class="identifier">domain</span><span class="special">,</span> <span class="identifier">tag</span><span class="special">::</span><span class="identifier">int_</span><span class="special">></span> <span class="comment">// enables int_
</span> <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{};</span>
</pre>
<p>
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><></span>
<span class="keyword">struct</span> <span class="identifier">use_terminal</span><span class="special"><</span><span class="identifier">qi</span><span class="special">::</span><span class="identifier">domain</span><span class="special">,</span> <span class="identifier">tag</span><span class="special">::</span><span class="identifier">long_</span><span class="special">></span> <span class="comment">// enables long_
</span> <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{};</span>
</pre>
<p>
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><></span>
<span class="keyword">struct</span> <span class="identifier">use_terminal</span><span class="special"><</span><span class="identifier">qi</span><span class="special">::</span><span class="identifier">domain</span><span class="special">,</span> <span class="identifier">tag</span><span class="special">::</span><span class="identifier">long_long</span><span class="special">></span> <span class="comment">// enables long_long
</span> <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{};</span>
</pre>
<p>
</p>
<p>
Notice that <code class="computeroutput"><span class="identifier">int_parser</span></code>
is placed in the namespace boost::spirit::qi while these <span class="emphasis"><em>enablers</em></span>
are in namespace boost::spirit. The reason is that these placeholders are
shared by other <a href="http://boost-spirit.com" target="_top">Spirit</a> <span class="emphasis"><em>domains</em></span>.
<span class="emphasis"><em>Spirit.Qi</em></span>, the parser is one domain. <span class="emphasis"><em>Spirit.Karma</em></span>,
the generator is another domain. Other parser technologies may be developed
and placed in yet another domain. Yet, all these can potentially share
the same placeholders for interoperability. The interpretation of these
placeholders is domain-specific.
</p>
<p>
Now that we enabled the placeholders, we have to write generators for them.
The make_xxx stuff (in boost::spirit::qi namespace):
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">T</span><span class="special">></span>
<span class="keyword">struct</span> <span class="identifier">make_int</span>
<span class="special">{</span>
<span class="keyword">typedef</span> <span class="identifier">int_parser_impl</span><span class="special"><</span><span class="identifier">T</span><span class="special">></span> <span class="identifier">result_type</span><span class="special">;</span>
<span class="identifier">result_type</span> <span class="keyword">operator</span><span class="special">()(</span><span class="identifier">unused_type</span><span class="special">,</span> <span class="identifier">unused_type</span><span class="special">)</span> <span class="keyword">const</span>
<span class="special">{</span>
<span class="keyword">return</span> <span class="identifier">result_type</span><span class="special">();</span>
<span class="special">}</span>
<span class="special">};</span>
</pre>
<p>
</p>
<p>
This one above is our main generator. It's a simple function object with
2 (unused) arguments. These arguments are
</p>
<div class="orderedlist"><ol class="orderedlist" type="1">
<li class="listitem">
The actual terminal value obtained by proto. In this case, either a
short<span class="underline">, int</span>, long_ or long_long.
We don't care about this.
</li>
<li class="listitem">
Modifiers. We also don't care about this. This allows directives such
as <code class="computeroutput"><span class="identifier">no_case</span><span class="special">[</span><span class="identifier">p</span><span class="special">]</span></code>
to pass information to inner parser nodes. We'll see how that works
later.
</li>
</ol></div>
<p>
Now:
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Modifiers</span><span class="special">></span>
<span class="keyword">struct</span> <span class="identifier">make_primitive</span><span class="special"><</span><span class="identifier">tag</span><span class="special">::</span><span class="identifier">short_</span><span class="special">,</span> <span class="identifier">Modifiers</span><span class="special">></span> <span class="special">:</span> <span class="identifier">make_int</span><span class="special"><</span><span class="keyword">short</span><span class="special">></span> <span class="special">{};</span>
</pre>
<p>
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Modifiers</span><span class="special">></span>
<span class="keyword">struct</span> <span class="identifier">make_primitive</span><span class="special"><</span><span class="identifier">tag</span><span class="special">::</span><span class="identifier">int_</span><span class="special">,</span> <span class="identifier">Modifiers</span><span class="special">></span> <span class="special">:</span> <span class="identifier">make_int</span><span class="special"><</span><span class="keyword">int</span><span class="special">></span> <span class="special">{};</span>
</pre>
<p>
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Modifiers</span><span class="special">></span>
<span class="keyword">struct</span> <span class="identifier">make_primitive</span><span class="special"><</span><span class="identifier">tag</span><span class="special">::</span><span class="identifier">long_</span><span class="special">,</span> <span class="identifier">Modifiers</span><span class="special">></span> <span class="special">:</span> <span class="identifier">make_int</span><span class="special"><</span><span class="keyword">long</span><span class="special">></span> <span class="special">{};</span>
</pre>
<p>
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Modifiers</span><span class="special">></span>
<span class="keyword">struct</span> <span class="identifier">make_primitive</span><span class="special"><</span><span class="identifier">tag</span><span class="special">::</span><span class="identifier">long_long</span><span class="special">,</span> <span class="identifier">Modifiers</span><span class="special">></span>
<span class="special">:</span> <span class="identifier">make_int</span><span class="special"><</span><span class="identifier">boost</span><span class="special">::</span><span class="identifier">long_long_type</span><span class="special">></span> <span class="special">{};</span>
</pre>
<p>
</p>
<p>
These, specialize <code class="computeroutput"><span class="identifier">qi</span><span class="special">:</span><span class="identifier">make_primitive</span></code> for specific tags. They
all inherit from <code class="computeroutput"><span class="identifier">make_int</span></code>
which does the actual work.
</p>
<a name="spirit.advanced.indepth.parsers_indepth.composite_parsers"></a><h6>
<a name="id1198813"></a>
<a class="link" href="parsers_indepth.html#spirit.advanced.indepth.parsers_indepth.composite_parsers">Composite
Parsers</a>
</h6>
<p>
Let me present the kleene star (also in namespace spirit::qi):
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Subject</span><span class="special">></span>
<span class="keyword">struct</span> <span class="identifier">kleene</span> <span class="special">:</span> <span class="identifier">unary_parser</span><span class="special"><</span><span class="identifier">kleene</span><span class="special"><</span><span class="identifier">Subject</span><span class="special">></span> <span class="special">></span>
<span class="special">{</span>
<span class="keyword">typedef</span> <span class="identifier">Subject</span> <span class="identifier">subject_type</span><span class="special">;</span>
<span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Context</span><span class="special">,</span> <span class="keyword">typename</span> <span class="identifier">Iterator</span><span class="special">></span>
<span class="keyword">struct</span> <span class="identifier">attribute</span>
<span class="special">{</span>
<span class="comment">// Build a std::vector from the subject's attribute. Note
</span> <span class="comment">// that build_std_vector may return unused_type if the
</span> <span class="comment">// subject's attribute is an unused_type.
</span> <span class="keyword">typedef</span> <span class="keyword">typename</span>
<span class="identifier">traits</span><span class="special">::</span><span class="identifier">build_std_vector</span><span class="special"><</span>
<span class="keyword">typename</span> <span class="identifier">traits</span><span class="special">::</span>
<span class="identifier">attribute_of</span><span class="special"><</span><span class="identifier">Subject</span><span class="special">,</span> <span class="identifier">Context</span><span class="special">,</span> <span class="identifier">Iterator</span><span class="special">>::</span><span class="identifier">type</span>
<span class="special">>::</span><span class="identifier">type</span>
<span class="identifier">type</span><span class="special">;</span>
<span class="special">};</span>
<span class="identifier">kleene</span><span class="special">(</span><span class="identifier">Subject</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">subject</span><span class="special">)</span>
<span class="special">:</span> <span class="identifier">subject</span><span class="special">(</span><span class="identifier">subject</span><span class="special">)</span> <span class="special">{}</span>
<span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Iterator</span><span class="special">,</span> <span class="keyword">typename</span> <span class="identifier">Context</span>
<span class="special">,</span> <span class="keyword">typename</span> <span class="identifier">Skipper</span><span class="special">,</span> <span class="keyword">typename</span> <span class="identifier">Attribute</span><span class="special">></span>
<span class="keyword">bool</span> <span class="identifier">parse</span><span class="special">(</span><span class="identifier">Iterator</span><span class="special">&</span> <span class="identifier">first</span><span class="special">,</span> <span class="identifier">Iterator</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">last</span>
<span class="special">,</span> <span class="identifier">Context</span><span class="special">&</span> <span class="identifier">context</span><span class="special">,</span> <span class="identifier">Skipper</span> <span class="keyword">const</span><span class="special">&</span> <span class="identifier">skipper</span>
<span class="special">,</span> <span class="identifier">Attribute</span><span class="special">&</span> <span class="identifier">attr</span><span class="special">)</span> <span class="keyword">const</span>
<span class="special">{</span>
<span class="comment">// create a local value if Attribute is not unused_type
</span> <span class="keyword">typedef</span> <span class="keyword">typename</span> <span class="identifier">traits</span><span class="special">::</span><span class="identifier">container_value</span><span class="special"><</span><span class="identifier">Attribute</span><span class="special">>::</span><span class="identifier">type</span>
<span class="identifier">value_type</span><span class="special">;</span>
<span class="identifier">value_type</span> <span class="identifier">val</span> <span class="special">=</span> <span class="identifier">value_type</span><span class="special">();</span>
<span class="comment">// Repeat while subject parses ok
</span> <span class="identifier">Iterator</span> <span class="identifier">save</span> <span class="special">=</span> <span class="identifier">first</span><span class="special">;</span>
<span class="keyword">while</span> <span class="special">(</span><span class="identifier">subject</span><span class="special">.</span><span class="identifier">parse</span><span class="special">(</span><span class="identifier">save</span><span class="special">,</span> <span class="identifier">last</span><span class="special">,</span> <span class="identifier">context</span><span class="special">,</span> <span class="identifier">skipper</span><span class="special">,</span> <span class="identifier">val</span><span class="special">)</span> <span class="special">&&</span>
<span class="identifier">traits</span><span class="special">::</span><span class="identifier">push_back</span><span class="special">(</span><span class="identifier">attr</span><span class="special">,</span> <span class="identifier">val</span><span class="special">))</span> <span class="comment">// push the parsed value into our attribute
</span> <span class="special">{</span>
<span class="identifier">first</span> <span class="special">=</span> <span class="identifier">save</span><span class="special">;</span>
<span class="identifier">traits</span><span class="special">::</span><span class="identifier">clear</span><span class="special">(</span><span class="identifier">val</span><span class="special">);</span>
<span class="special">}</span>
<span class="keyword">return</span> <span class="keyword">true</span><span class="special">;</span>
<span class="special">}</span>
<span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Context</span><span class="special">></span>
<span class="identifier">info</span> <span class="identifier">what</span><span class="special">(</span><span class="identifier">Context</span><span class="special">&</span> <span class="identifier">context</span><span class="special">)</span> <span class="keyword">const</span>
<span class="special">{</span>
<span class="keyword">return</span> <span class="identifier">info</span><span class="special">(</span><span class="string">"kleene"</span><span class="special">,</span> <span class="identifier">subject</span><span class="special">.</span><span class="identifier">what</span><span class="special">(</span><span class="identifier">context</span><span class="special">));</span>
<span class="special">}</span>
<span class="identifier">Subject</span> <span class="identifier">subject</span><span class="special">;</span>
<span class="special">};</span>
</pre>
<p>
</p>
<p>
Looks similar in form to its primitive cousin, the <code class="computeroutput"><span class="identifier">int_parser</span></code>.
And, again, it has the same basic ingredients required by <code class="computeroutput"><span class="identifier">Derived</span></code>.
</p>
<div class="itemizedlist"><ul class="itemizedlist" type="disc">
<li class="listitem">
The nested attribute metafunction
</li>
<li class="listitem">
The parse member function
</li>
<li class="listitem">
The what member function
</li>
</ul></div>
<p>
kleene is a composite parser. It is a parser that composes another parser,
its <span class="quote">“<span class="quote">subject</span>”</span>. It is a <code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/unaryparser.html" title="UnaryParser"><code class="computeroutput"><span class="identifier">UnaryParser</span></code></a></code> and subclasses
from it. Like <code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/primitiveparser.html" title="PrimitiveParser"><code class="computeroutput"><span class="identifier">PrimitiveParser</span></code></a></code>, <code class="computeroutput"><a class="link" href="../../qi/reference/parser_concepts/unaryparser.html" title="UnaryParser"><code class="computeroutput"><span class="identifier">UnaryParser</span></code></a><span class="special"><</span><span class="identifier">Derived</span><span class="special">></span></code>
derives from <code class="computeroutput"><span class="identifier">parser</span><span class="special"><</span><span class="identifier">Derived</span><span class="special">></span></code>.
</p>
<p>
unary_parser<Derived>, has these expression requirements on Derived:
</p>
<div class="itemizedlist"><ul class="itemizedlist" type="disc">
<li class="listitem">
p.subject -> subject parser ( <span class="emphasis"><em>p</em></span> is a <a class="link" href="../../qi/reference/parser_concepts/unaryparser.html" title="UnaryParser"><code class="computeroutput"><span class="identifier">UnaryParser</span></code></a> parser.)
</li>
<li class="listitem">
P::subject_type -> subject parser type ( <span class="emphasis"><em>P</em></span>
is a <a class="link" href="../../qi/reference/parser_concepts/unaryparser.html" title="UnaryParser"><code class="computeroutput"><span class="identifier">UnaryParser</span></code></a> type.)
</li>
</ul></div>
<p>
<span class="emphasis"><em>parse</em></span> is the main parser entry point. Since this is
not a primitive parser, we do not need to call <code class="computeroutput"><span class="identifier">qi</span><span class="special">::</span><span class="identifier">skip</span><span class="special">(</span><span class="identifier">first</span><span class="special">,</span> <span class="identifier">last</span><span class="special">,</span> <span class="identifier">skipper</span><span class="special">)</span></code>. The <span class="emphasis"><em>subject</em></span>, if
it is a primitive, will do the pre-skip. If if it is another composite
parser, it will eventually call a primitive parser somewhere down the line
which will do the pre-skip. This makes it a lot more efficient than <a href="../../../../../../../libs/spirit/classic/index.html" target="_top"><span class="emphasis"><em>Spirit.Classic</em></span></a>.
<a href="../../../../../../../libs/spirit/classic/index.html" target="_top"><span class="emphasis"><em>Spirit.Classic</em></span></a>
puts the skipping business into the so-called "scanner" which
blindly attempts a pre-skip everytime we increment the iterator.
</p>
<p>
What is the <span class="emphasis"><em>attribute</em></span> of the kleene? In general, it
is a <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span><span class="special"><</span><span class="identifier">T</span><span class="special">></span></code>
where <code class="computeroutput"><span class="identifier">T</span></code> is the attribute
of the subject. There is a special case though. If <code class="computeroutput"><span class="identifier">T</span></code>
is an <code class="computeroutput"><span class="identifier">unused_type</span></code>, then
the attribute of kleene is also <code class="computeroutput"><span class="identifier">unused_type</span></code>.
<code class="computeroutput"><span class="identifier">traits</span><span class="special">::</span><span class="identifier">build_std_vector</span></code> takes care of that minor
detail.
</p>
<p>
So, let's parse. First, we need to provide a local attribute of for the
subject:
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">typename</span> <span class="identifier">traits</span><span class="special">::</span><span class="identifier">attribute_of</span><span class="special"><</span><span class="identifier">Subject</span><span class="special">,</span> <span class="identifier">Context</span><span class="special">>::</span><span class="identifier">type</span> <span class="identifier">val</span><span class="special">;</span>
</pre>
<p>
</p>
<p>
<code class="computeroutput"><span class="identifier">traits</span><span class="special">::</span><span class="identifier">attribute_of</span><span class="special"><</span><span class="identifier">Subject</span><span class="special">,</span> <span class="identifier">Context</span><span class="special">></span></code>
simply calls the subject's <code class="computeroutput"><span class="keyword">struct</span>
<span class="identifier">attribute</span><span class="special"><</span><span class="identifier">Context</span><span class="special">></span></code>
nested metafunction.
</p>
<p>
<span class="emphasis"><em>val</em></span> starts out default initialized. This val is the
one we'll pass to the subject's parse function.
</p>
<p>
The kleene repeats indefinitely while the subject parser is successful.
On each successful parse, we <code class="computeroutput"><span class="identifier">push_back</span></code>
the parsed attribute to the kleen's attribute, which is expected to be,
at the very least, compatible with a <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span></code>.
In other words, although we say that we want our attribute to be a <code class="computeroutput"><span class="identifier">std</span><span class="special">::</span><span class="identifier">vector</span></code>, we try to be more lenient than
that. The caller of kleene's parse may pass a different attribute type.
For as long as it is also a conforming STL container with <code class="computeroutput"><span class="identifier">push_back</span></code>, we are ok. Here is the kleene
loop:
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">while</span> <span class="special">(</span><span class="identifier">subject</span><span class="special">.</span><span class="identifier">parse</span><span class="special">(</span><span class="identifier">first</span><span class="special">,</span> <span class="identifier">last</span><span class="special">,</span> <span class="identifier">context</span><span class="special">,</span> <span class="identifier">skipper</span><span class="special">,</span> <span class="identifier">val</span><span class="special">))</span>
<span class="special">{</span>
<span class="comment">// push the parsed value into our attribute
</span> <span class="identifier">traits</span><span class="special">::</span><span class="identifier">push_back</span><span class="special">(</span><span class="identifier">attr</span><span class="special">,</span> <span class="identifier">val</span><span class="special">);</span>
<span class="identifier">traits</span><span class="special">::</span><span class="identifier">clear</span><span class="special">(</span><span class="identifier">val</span><span class="special">);</span>
<span class="special">}</span>
<span class="keyword">return</span> <span class="keyword">true</span><span class="special">;</span>
</pre>
<p>
Take note that we didn't call attr.push_back(val). Instead, we called a
Spirit provided function:
</p>
<p>
</p>
<pre class="programlisting"><span class="identifier">traits</span><span class="special">::</span><span class="identifier">push_back</span><span class="special">(</span><span class="identifier">attr</span><span class="special">,</span> <span class="identifier">val</span><span class="special">);</span>
</pre>
<p>
</p>
<p>
This is a recurring pattern. The reason why we do it this way is because
attr <span class="bold"><strong>can</strong></span> be <code class="computeroutput"><span class="identifier">unused_type</span></code>.
<code class="computeroutput"><span class="identifier">traits</span><span class="special">::</span><span class="identifier">push_back</span></code> takes care of that detail.
The overload for unused_type is a no-op. Now, you can imagine why <a href="http://boost-spirit.com" target="_top">Spirit</a> is fast! The parsers are so
simple and the generated code is as efficient as a hand rolled loop. All
these parser compositions and recursive parse invocations are extensively
inlined by a modern C++ compiler. In the end, you get a tight loop when
you use the kleene. No more excess baggage. If the attribute is unused,
then there is no code generated for that. That's how <a href="http://boost-spirit.com" target="_top">Spirit</a>
is designed.
</p>
<p>
The <span class="emphasis"><em>what</em></span> function simply wraps the output of the subject
in a "kleene<span class="quote">“<span class="quote">... "</span>”</span>".
</p>
<p>
Ok, now, like the <code class="computeroutput"><span class="identifier">int_parser</span></code>,
we have to hook our parser to the <span class="underline">qi</span>
engine. Here's how we do it:
</p>
<p>
First, we enable the prefix star operator. In proto, it's called the "dereference":
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><></span>
<span class="keyword">struct</span> <span class="identifier">use_operator</span><span class="special"><</span><span class="identifier">qi</span><span class="special">::</span><span class="identifier">domain</span><span class="special">,</span> <span class="identifier">proto</span><span class="special">::</span><span class="identifier">tag</span><span class="special">::</span><span class="identifier">dereference</span><span class="special">></span> <span class="comment">// enables *p
</span> <span class="special">:</span> <span class="identifier">mpl</span><span class="special">::</span><span class="identifier">true_</span> <span class="special">{};</span>
</pre>
<p>
</p>
<p>
This is done in namespace <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span></code>
like its friend, the <code class="computeroutput"><span class="identifier">use_terminal</span></code>
specialization for our <code class="computeroutput"><span class="identifier">int_parser</span></code>.
Obviously, we use <span class="emphasis"><em>use_operator</em></span> to enable the dereference
for the qi::domain.
</p>
<p>
Then, we need to write our generator (in namespace qi):
</p>
<p>
</p>
<pre class="programlisting"><span class="keyword">template</span> <span class="special"><</span><span class="keyword">typename</span> <span class="identifier">Elements</span><span class="special">,</span> <span class="keyword">typename</span> <span class="identifier">Modifiers</span><span class="special">></span>
<span class="keyword">struct</span> <span class="identifier">make_composite</span><span class="special"><</span><span class="identifier">proto</span><span class="special">::</span><span class="identifier">tag</span><span class="special">::</span><span class="identifier">dereference</span><span class="special">,</span> <span class="identifier">Elements</span><span class="special">,</span> <span class="identifier">Modifiers</span><span class="special">></span>
<span class="special">:</span> <span class="identifier">make_unary_composite</span><span class="special"><</span><span class="identifier">Elements</span><span class="special">,</span> <span class="identifier">kleene</span><span class="special">></span>
<span class="special">{};</span>
</pre>
<p>
</p>
<p>
This essentially says; for all expressions of the form: <code class="computeroutput"><span class="special">*</span><span class="identifier">p</span></code>, to build a kleene parser. Elements
is a <a href="../../../../../../../libs/fusion/doc/html/index.html" target="_top">Boost.Fusion</a>
sequence. For the kleene, which is a unary operator, expect only one element
in the sequence. That element is the subject of the kleene.
</p>
<p>
We still don't care about the Modifiers. We'll see how the modifiers is
all about when we get to deep directives.
</p>
</div>
<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
<td align="left"></td>
<td align="right"><div class="copyright-footer">Copyright © 2001-2010 Joel de Guzman, Hartmut Kaiser<p>
Distributed under the Boost Software License, Version 1.0. (See accompanying
file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
</p>
</div></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="../indepth.html"><img src="../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../indepth.html"><img src="../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../index.html"><img src="../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="../customize.html"><img src="../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
</body>
</html>
