<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII"> <title>Inhibiting Skipping (lexeme[])</title> <link rel="stylesheet" href="../../../../../../../../doc/src/boostbook.css" type="text/css"> <meta name="generator" content="DocBook XSL Stylesheets V1.75.0"> <link rel="home" href="../../../../index.html" title="Spirit 2.4"> <link rel="up" href="../directive.html" title="Directive"> <link rel="prev" href="../directive.html" title="Directive"> <link rel="next" href="no_skip.html" title="Inhibiting Skipping Without Pre-skip (no_skip[])"> </head> <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> <table cellpadding="2" width="100%"><tr> <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../../boost.png"></td> <td align="center"><a href="../../../../../../../../index.html">Home</a></td> <td align="center"><a href="../../../../../../../../libs/libraries.htm">Libraries</a></td> <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> <td align="center"><a href="../../../../../../../../more/index.htm">More</a></td> </tr></table> <hr> <div class="spirit-nav"> <a accesskey="p" href="../directive.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../directive.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="no_skip.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a> </div> <div class="section"> <div class="titlepage"><div><div><h5 class="title"> <a name="spirit.qi.reference.directive.lexeme"></a><a class="link" href="lexeme.html" title="Inhibiting Skipping (lexeme[])"> Inhibiting Skipping (<code class="computeroutput"><span class="identifier">lexeme</span><span class="special">[]</span></code>)</a> </h5></div></div></div> <a name="spirit.qi.reference.directive.lexeme.description"></a><h6> <a name="id926507"></a> <a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.description">Description</a> </h6> <p> The <code class="computeroutput"><span class="identifier">lexeme</span></code> directive makes its subject a primitive. In a logical point of view, lexemes (and primitives) are minimal atomic units (e.g. words, numbers, identifiers, etc). These are the things that you'd normally put in the lexer (hinting at the term "lexeme"), but in a lexer-less world, you put these in a lexeme. Seeing its subject as a primitive, the <code class="computeroutput"><span class="identifier">lexeme</span></code> directive does an initial pre-skip (as all primitives do) and turns off white space skipping. </p> <p> At the phrase level, the parser ignores white spaces, possibly including comments. Use <code class="computeroutput"><span class="identifier">lexeme</span></code> in situations where you want to work at the character level instead of the phrase level. Parsers can be made to work at the character level by enclosing the pertinent parts inside the <code class="computeroutput"><span class="identifier">lexeme</span></code> directive. For example, here's a rule that parses integers: </p> <pre class="programlisting"><span class="identifier">integer</span> <span class="special">=</span> <span class="identifier">lexeme</span><span class="special">[</span> <span class="special">-(</span><span class="identifier">lit</span><span class="special">(</span><span class="char">'+'</span><span class="special">)</span> <span class="special">|</span> <span class="char">'-'</span><span class="special">)</span> <span class="special">>></span> <span class="special">+</span><span class="identifier">digit</span> <span class="special">];</span> </pre> <p> The <code class="computeroutput"><span class="identifier">lexeme</span></code> directive instructs its subject parser to work on the character level. Without it, the <code class="computeroutput"><span class="identifier">integer</span></code> rule would have allowed erroneous embedded white spaces in inputs such as <code class="computeroutput"><span class="string">"1 2 345"</span></code> which will be parsed as <code class="computeroutput"><span class="string">"12345"</span></code>. </p> <div class="note"><table border="0" summary="Note"> <tr> <td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../images/note.png"></td> <th align="left">Note</th> </tr> <tr><td align="left" valign="top"><p> Keep in mind that <code class="computeroutput"><span class="identifier">lexeme</span><span class="special">[]</span></code> pre-skips spaces. If this is not desired, use the <a class="link" href="no_skip.html" title="Inhibiting Skipping Without Pre-skip (no_skip[])"><code class="computeroutput"><span class="identifier">no_skip</span></code></a> directive instead. </p></td></tr> </table></div> <a name="spirit.qi.reference.directive.lexeme.header"></a><h6> <a name="id926688"></a> <a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.header">Header</a> </h6> <pre class="programlisting"><span class="comment">// forwards to <boost/spirit/home/qi/directive/lexeme.hpp> </span><span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">spirit</span><span class="special">/</span><span class="identifier">include</span><span class="special">/</span><span class="identifier">qi_lexeme</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> </pre> <p> Also, see <a class="link" href="../../../structure/include.html" title="Include">Include Structure</a>. </p> <a name="spirit.qi.reference.directive.lexeme.namespace"></a><h6> <a name="id928408"></a> <a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.namespace">Namespace</a> </h6> <div class="informaltable"><table class="table"> <colgroup><col></colgroup> <thead><tr><th> <p> Name </p> </th></tr></thead> <tbody><tr><td> <p> <code class="computeroutput"><span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">lexeme</span> <span class="comment">// alias: boost::spirit::qi::lexeme</span></code> </p> </td></tr></tbody> </table></div> <a name="spirit.qi.reference.directive.lexeme.model_of"></a><h6> <a name="id928488"></a> <a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.model_of">Model of</a> </h6> <div class="blockquote"><blockquote class="blockquote"><p> <a class="link" href="../parser_concepts/unaryparser.html" title="UnaryParser"><code class="computeroutput"><span class="identifier">UnaryParser</span></code></a> </p></blockquote></div> <div class="variablelist"> <p class="title"><b>Notation</b></p> <dl> <dt><span class="term"><code class="computeroutput"><span class="identifier">a</span></code></span></dt> <dd><p> A <a class="link" href="../parser_concepts/parser.html" title="Parser"><code class="computeroutput"><span class="identifier">Parser</span></code></a>. </p></dd> </dl> </div> <a name="spirit.qi.reference.directive.lexeme.expression_semantics"></a><h6> <a name="id928556"></a> <a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.expression_semantics">Expression Semantics</a> </h6> <p> Semantics of an expression is defined only where it differs from, or is not defined in <a class="link" href="../parser_concepts/unaryparser.html" title="UnaryParser"><code class="computeroutput"><span class="identifier">UnaryParser</span></code></a>. </p> <div class="informaltable"><table class="table"> <colgroup> <col> <col> </colgroup> <thead><tr> <th> <p> Expression </p> </th> <th> <p> Semantics </p> </th> </tr></thead> <tbody><tr> <td> <p> <code class="computeroutput"><span class="identifier">lexeme</span><span class="special">[</span><span class="identifier">a</span><span class="special">]</span></code> </p> </td> <td> <p> Pre-skip and turn off white space skipping for the subject parser, <code class="computeroutput"><span class="identifier">a</span></code> (and all its children). </p> </td> </tr></tbody> </table></div> <a name="spirit.qi.reference.directive.lexeme.attributes"></a><h6> <a name="id928663"></a> <a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.attributes">Attributes</a> </h6> <p> See <a class="link" href="../../quick_reference/compound_attribute_rules.html#spirit.qi.quick_reference.compound_attribute_rules.notation">Compound Attribute Notation</a>. </p> <div class="informaltable"><table class="table"> <colgroup> <col> <col> </colgroup> <thead><tr> <th> <p> Expression </p> </th> <th> <p> Attribute </p> </th> </tr></thead> <tbody><tr> <td> <p> <code class="computeroutput"><span class="identifier">lexeme</span><span class="special">[</span><span class="identifier">a</span><span class="special">]</span></code> </p> </td> <td> <p> </p> <pre xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" class="table-programlisting"><span class="identifier">a</span><span class="special">:</span> <span class="identifier">A</span> <span class="special">--></span> <span class="identifier">lexeme</span><span class="special">[</span><span class="identifier">a</span><span class="special">]:</span> <span class="identifier">A</span> <span class="identifier">a</span><span class="special">:</span> <span class="identifier">Unused</span> <span class="special">--></span> <span class="identifier">lexeme</span><span class="special">[</span><span class="identifier">a</span><span class="special">]:</span> <span class="identifier">Unused</span></pre> <p> </p> </td> </tr></tbody> </table></div> <a name="spirit.qi.reference.directive.lexeme.complexity"></a><h6> <a name="id928837"></a> <a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.complexity">Complexity</a> </h6> <div class="blockquote"><blockquote class="blockquote"><p> The complexity is defined by the complexity of the subject parser, <code class="computeroutput"><span class="identifier">a</span></code> </p></blockquote></div> <a name="spirit.qi.reference.directive.lexeme.example"></a><h6> <a name="id928864"></a> <a class="link" href="lexeme.html#spirit.qi.reference.directive.lexeme.example">Example</a> </h6> <div class="note"><table border="0" summary="Note"> <tr> <td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../images/note.png"></td> <th align="left">Note</th> </tr> <tr><td align="left" valign="top"><p> The test harness for the example(s) below is presented in the <a class="link" href="../basics.html#spirit.qi.reference.basics.examples">Basics Examples</a> section. </p></td></tr> </table></div> <p> Some using declarations: </p> <p> </p> <pre class="programlisting"><span class="keyword">using</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">qi</span><span class="special">::</span><span class="identifier">lexeme</span><span class="special">;</span> <span class="keyword">using</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">qi</span><span class="special">::</span><span class="identifier">lit</span><span class="special">;</span> <span class="keyword">using</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">spirit</span><span class="special">::</span><span class="identifier">ascii</span><span class="special">::</span><span class="identifier">digit</span><span class="special">;</span> </pre> <p> </p> <p> Simple usage of <code class="computeroutput"><span class="identifier">lexeme</span><span class="special">[]</span></code>: </p> <p> The use of lexeme here will prevent skipping in between the digits and the sign making inputs such as <code class="computeroutput"><span class="string">"1 2 345"</span></code> erroneous. </p> <p> </p> <pre class="programlisting"><span class="identifier">test_phrase_parser</span><span class="special">(</span><span class="string">"12345"</span><span class="special">,</span> <span class="identifier">lexeme</span><span class="special">[</span> <span class="special">-(</span><span class="identifier">lit</span><span class="special">(</span><span class="char">'+'</span><span class="special">)</span> <span class="special">|</span> <span class="char">'-'</span><span class="special">)</span> <span class="special">>></span> <span class="special">+</span><span class="identifier">digit</span> <span class="special">]);</span> </pre> <p> </p> </div> <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> <td align="left"></td> <td align="right"><div class="copyright-footer">Copyright © 2001-2010 Joel de Guzman, Hartmut Kaiser<p> Distributed under the Boost Software License, Version 1.0. (See accompanying file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>) </p> </div></td> </tr></table> <hr> <div class="spirit-nav"> <a accesskey="p" href="../directive.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../directive.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="no_skip.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a> </div> </body> </html>