<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html xmlns:fn="http://www.w3.org/2005/02/xpath-functions">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<link rel="stylesheet" href="../../../../doc/otp_doc.css" type="text/css">
<title>Erlang -- leex</title>
</head>
<body bgcolor="white" text="#000000" link="#0000ff" vlink="#ff00ff" alink="#ff0000"><div id="container">
<script id="js" type="text/javascript" language="JavaScript" src="../../../../doc/js/flipmenu/flipmenu.js"></script><script id="js2" type="text/javascript" src="../../../../doc/js/erlresolvelinks.js"></script><script language="JavaScript" type="text/javascript">
<!--
function getWinHeight() {
var myHeight = 0;
if( typeof( window.innerHeight ) == 'number' ) {
//Non-IE
myHeight = window.innerHeight;
} else if( document.documentElement && ( document.documentElement.clientWidth ||
document.documentElement.clientHeight ) ) {
//IE 6+ in 'standards compliant mode'
myHeight = document.documentElement.clientHeight;
} else if( document.body && ( document.body.clientWidth || document.body.clientHeight ) ) {
//IE 4 compatible
myHeight = document.body.clientHeight;
}
return myHeight;
}
function setscrollpos() {
var objf=document.getElementById('loadscrollpos');
document.getElementById("leftnav").scrollTop = objf.offsetTop - getWinHeight()/2;
}
function addEvent(obj, evType, fn){
if (obj.addEventListener){
obj.addEventListener(evType, fn, true);
return true;
} else if (obj.attachEvent){
var r = obj.attachEvent("on"+evType, fn);
return r;
} else {
return false;
}
}
addEvent(window, 'load', setscrollpos);
//--></script><div id="leftnav"><div class="innertube">
<img alt="Erlang logo" src="../../../../doc/erlang-logo.png"><br><small><a href="index.html">Reference Manual</a><br><a href="release_notes.html">Release Notes</a><br><a href="../pdf/parsetools-2.0.7.pdf">PDF</a><br><a href="../../../../doc/index.html">Top</a></small><p><strong>Parse Tools</strong><br><strong>Reference Manual</strong><br><small>Version 2.0.7</small></p>
<br><a href="javascript:openAllFlips()">Expand All</a><br><a href="javascript:closeAllFlips()">Contract All</a><p><small><strong>Table of Contents</strong></small></p>
<ul class="flipMenu">
<li id="no" title="yecc " expanded="false">yecc<ul>
<li><a href="yecc.html">
Top of manual page
</a></li>
<li title="file-1"><a href="yecc.html#file-1">file/1</a></li>
<li title="format_error-1"><a href="yecc.html#format_error-1">format_error/1</a></li>
</ul>
</li>
<li id="loadscrollpos" title="leex " expanded="true">leex<ul>
<li><a href="leex.html">
Top of manual page
</a></li>
<li title="file-1"><a href="leex.html#file-1">file/1</a></li>
<li title="file-2"><a href="leex.html#file-2">file/2</a></li>
<li title="format_error-1"><a href="leex.html#format_error-1">format_error/1</a></li>
<li title="string-1"><a href="leex.html#string-1">string/1</a></li>
<li title="string-2"><a href="leex.html#string-2">string/2</a></li>
<li title="token-2"><a href="leex.html#token-2">token/2</a></li>
<li title="token-3"><a href="leex.html#token-3">token/3</a></li>
<li title="tokens-2"><a href="leex.html#tokens-2">tokens/2</a></li>
<li title="tokens-3"><a href="leex.html#tokens-3">tokens/3</a></li>
</ul>
</li>
</ul>
</div></div>
<div id="content">
<div class="innertube">
<!-- refpage --><center><h1>leex</h1></center>
<h3>MODULE</h3>
<div class="REFBODY">leex</div>
<h3>MODULE SUMMARY</h3>
<div class="REFBODY">Lexical analyzer generator for Erlang</div>
<h3>DESCRIPTION</h3>
<div class="REFBODY"><p>
<p>A regular expression based lexical analyzer generator for
Erlang, similar to lex or flex.</p>
<div class="note">
<div class="label">Note</div>
<div class="content"><p><p>The Leex module should be considered experimental
as it will be subject to changes in future releases.</p></p></div>
</div>
</p></div>
<h3><a name="id62247">DATA TYPES</a></h3>
<div class="REFBODY">
<div class="example"><pre>
ErrorInfo = {ErrorLine,module(),error_descriptor()}
ErrorLine = integer()
Token = tuple()</pre></div>
</div>
<h3>EXPORTS</h3>
<p><a name="file-1"><span class="bold_code">file(FileName) -> ok | error</span></a><br><a name="file-2"><span class="bold_code">file(FileName, Options) -> ok | error</span></a><br></p>
<div class="REFBODY">
<p>Types:</p>
<div class="REFTYPES">
<span class="bold_code">FileName = filename()</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">Options = Option | [Option]</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">Option = - see below -</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">FileReturn = {ok, Scannerfile}
| {ok, Scannerfile, Warnings}
| error
| {error, Warnings, Errors}</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">Scannerfile = filename()</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">Warnings = Errors = [{filename(), [ErrorInfo]}]</span><br>
</div>
</div>
<div class="REFBODY"><p>
<p>Generates a lexical analyzer from the definition in the input
file. The input file has the extension <span class="code">.xrl</span>. This is
added to the filename if it is not given. The resulting module
is the Xrl filename without the <span class="code">.xrl</span> extension.</p>
<p>The current options are:</p>
<dl>
<dt><strong><span class="code">dfa_graph</span></strong></dt>
<dd>
<p>Generates a <span class="code">.dot</span> file which contains a
description of the DFA in a format which can be viewed with
Graphviz, <span class="code">www.graphviz.com</span>.</p>
</dd>
<dt><strong><span class="code">{includefile,Includefile}</span></strong></dt>
<dd>
<p>Uses a specific or customised prologue file
instead of default
<span class="code">lib/parsetools/include/leexinc.hrl</span> which is
otherwise included.</p>
</dd>
<dt><strong><span class="code">{report_errors, bool()}</span></strong></dt>
<dd>
<p>Causes errors to be printed as they occur. Default is
<span class="code">true</span>.</p>
</dd>
<dt><strong><span class="code">{report_warnings, bool()}</span></strong></dt>
<dd>
<p>Causes warnings to be printed as they occur. Default is
<span class="code">true</span>.</p>
</dd>
<dt><strong><span class="code">warnings_as_errors</span></strong></dt>
<dd>
<p>Causes warnings to be treated as errors.</p>
</dd>
<dt><strong><span class="code">{report, bool()}</span></strong></dt>
<dd>
<p>This is a short form for both <span class="code">report_errors</span> and
<span class="code">report_warnings</span>.</p>
</dd>
<dt><strong><span class="code">{return_errors, bool()}</span></strong></dt>
<dd>
<p>If this flag is set, <span class="code">{error, Errors, Warnings}</span>
is returned when there are errors. Default is <span class="code">false</span>.</p>
</dd>
<dt><strong><span class="code">{return_warnings, bool()}</span></strong></dt>
<dd>
<p>If this flag is set, an extra field containing
<span class="code">Warnings</span> is added to the tuple returned upon
success. Default is <span class="code">false</span>.</p>
</dd>
<dt><strong><span class="code">{return, bool()}</span></strong></dt>
<dd>
<p>This is a short form for both <span class="code">return_errors</span> and
<span class="code">return_warnings</span>.</p>
</dd>
<dt><strong><span class="code">{scannerfile, Scannerfile}</span></strong></dt>
<dd>
<p><span class="code">Scannerfile</span> is the name of the file that
will contain the Erlang scanner code that is generated.
The default (<span class="code">""</span>) is to add the extension
<span class="code">.erl</span> to <span class="code">FileName</span> stripped of the
<span class="code">.xrl</span> extension.</p>
</dd>
<dt><strong><span class="code">{verbose, bool()}</span></strong></dt>
<dd>
<p>Outputs information from parsing the input file and
generating the internal tables.</p>
</dd>
</dl>
<p>Any of the Boolean options can be set to <span class="code">true</span> by
stating the name of the option. For example, <span class="code">verbose</span>
is equivalent to <span class="code">{verbose, true}</span>.</p>
<p>Leex will add the extension <span class="code">.hrl</span> to the
<span class="code">Includefile</span> name and the extension <span class="code">.erl</span> to the
<span class="code">Scannerfile</span> name, unless the extension is already
there.</p>
</p></div>
<p><a name="format_error-1"><span class="bold_code">format_error(ErrorInfo) -> Chars</span></a><br></p>
<div class="REFBODY">
<p>Types:</p>
<div class="REFTYPES">
<span class="bold_code">Chars = [char() | Chars]</span><br>
</div>
</div>
<div class="REFBODY"><p>
<p>Returns a string which describes the error
<span class="code">ErrorInfo</span> returned when there is an error in a
regular expression.</p>
</p></div>
<h3><a name="id63091">GENERATED SCANNER EXPORTS</a></h3>
<div class="REFBODY">
<p>The following functions are exported by the generated scanner.</p>
</div>
<h3>EXPORTS</h3>
<p><a name="string-1"><span class="bold_code">string(String) -> StringRet</span></a><br><a name="string-2"><span class="bold_code">string(String, StartLine) -> StringRet</span></a><br></p>
<div class="REFBODY">
<p>Types:</p>
<div class="REFTYPES">
<span class="bold_code">String = string()</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">StringRet = {ok,Tokens,EndLine} | ErrorInfo</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">Tokens = [Token]</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">EndLine = StartLine = integer()</span><br>
</div>
</div>
<div class="REFBODY"><p>
<p>Scans <span class="code">String</span> and returns all the tokens in it, or an
error.</p>
<div class="note">
<div class="label">Note</div>
<div class="content"><p><p>It is an error if not all of the characters in
<span class="code">String</span> are consumed.</p></p></div>
</div>
</p></div>
<p><a name="token-2"><span class="bold_code">token(Cont, Chars) -> {more,Cont1} | {done,TokenRet,RestChars}
</span></a><br><a name="token-3"><span class="bold_code">token(Cont, Chars, StartLine) -> {more,Cont1}
| {done,TokenRet,RestChars}
</span></a><br></p>
<div class="REFBODY">
<p>Types:</p>
<div class="REFTYPES">
<span class="bold_code">Cont = [] | Cont1</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">Cont1 = tuple()</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">Chars = RestChars = string() | eof</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">TokenRet = {ok, Token, EndLine}
| {eof, EndLine}
| ErrorInfo</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">StartLine = EndLine = integer()</span><br>
</div>
</div>
<div class="REFBODY"><p>
<p>This is a re-entrant call to try and scan one token from
<span class="code">Chars</span>. If there are enough characters in <span class="code">Chars</span>
to either scan a token or detect an error then this will be
returned with <span class="code">{done,...}</span>. Otherwise
<span class="code">{cont,Cont}</span> will be returned where <span class="code">Cont</span> is
used in the next call to <span class="code">token()</span> with more characters
to try an scan the token. This is continued until a token
has been scanned. <span class="code">Cont</span> is initially <span class="code">[]</span>.</p>
<p>It is not designed to be called directly by an application
but used through the i/o system where it can typically be
called in an application by:</p>
<div class="example"><pre>
io:request(InFile, {get_until,Prompt,Module,token,[Line]})
-> TokenRet</pre></div>
</p></div>
<p><a name="tokens-2"><span class="bold_code">tokens(Cont, Chars) -> {more,Cont1} | {done,TokensRet,RestChars}
</span></a><br><a name="tokens-3"><span class="bold_code">tokens(Cont, Chars, StartLine) ->
{more,Cont1} | {done,TokensRet,RestChars}
</span></a><br></p>
<div class="REFBODY">
<p>Types:</p>
<div class="REFTYPES">
<span class="bold_code">Cont = [] | Cont1</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">Cont1 = tuple()</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">Chars = RestChars = string() | eof</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">TokensRet = {ok, Tokens, EndLine}
| {eof, EndLine}
| ErrorInfo</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">Tokens = [Token]</span><br>
</div>
<div class="REFTYPES">
<span class="bold_code">StartLine = EndLine = integer()</span><br>
</div>
</div>
<div class="REFBODY"><p>
<p>This is a re-entrant call to try and scan tokens from
<span class="code">Chars</span>. If there are enough characters in <span class="code">Chars</span>
to either scan tokens or detect an error then this will be
returned with <span class="code">{done,...}</span>. Otherwise
<span class="code">{cont,Cont}</span> will be returned where <span class="code">Cont</span> is
used in the next call to <span class="code">tokens()</span> with more
characters to try an scan the tokens. This is continued
until all tokens have been scanned. <span class="code">Cont</span> is initially
<span class="code">[]</span>.</p>
<p>This functions differs from <span class="code">token</span> in that it will
continue to scan tokens upto and including an
<span class="code">{end_token,Token}</span> has been scanned (see next
section). It will then return all the tokens. This is
typically used for scanning grammars like Erlang where there
is an explicit end token, <span class="code">'.'</span>. If no end token is
found then the whole file will be scanned and returned. If
an error occurs then all tokens upto and including the next
end token will be skipped.</p>
<p>It is not designed to be called directly by an application
but used through the i/o system where it can typically be
called in an application by:</p>
<div class="example"><pre>
io:request(InFile, {get_until,Prompt,Module,tokens,[Line]})
-> TokensRet</pre></div>
</p></div>
<h3><a name="id63367">Input File Format</a></h3>
<div class="REFBODY">
<p>Erlang style comments starting with a <span class="code">%</span> are allowed in
scanner files. A definition file has the following format:</p>
<div class="example"><pre>
<Header>
Definitions.
<Macro Definitions>
Rules.
<Token Rules>
Erlang code.
<Erlang code></pre></div>
<p>The "Definitions.", "Rules." and "Erlang code." headings are
mandatory and must occur at the beginning of a source line. The
<Header>, <Macro Definitions> and <Erlang code>
sections may be empty but there must be at least one rule.</p>
<p>Macro definitions have the following format:</p>
<div class="example"><pre>
NAME = VALUE</pre></div>
<p>and there must be spaces around <span class="code">=</span>. Macros can be used in
the regular expressions of rules by writing <span class="code">{NAME}</span>.</p>
<div class="note">
<div class="label">Note</div>
<div class="content"><p><p>When macros are expanded in expressions the macro calls
are replaced by the macro value without any form of quoting or
enclosing in parentheses.</p></p></div>
</div>
<p>Rules have the following format:</p>
<div class="example"><pre>
<Regexp> : <Erlang code>.</pre></div>
<p>The <Regexp> must occur at the start of a line and not
include any blanks; use <span class="code">\t</span> and <span class="code">\s</span> to include TAB
and SPACE characters in the regular expression. If <Regexp>
matches then the corresponding <Erlang code> is evaluated to
generate a token. With the Erlang code the following predefined
variables are available:</p>
<dl>
<dt><strong><span class="code">TokenChars</span></strong></dt>
<dd>
<p>A list of the characters in the matched token.</p>
</dd>
<dt><strong><span class="code">TokenLen</span></strong></dt>
<dd>
<p>The number of characters in the matched token.</p>
</dd>
<dt><strong><span class="code">TokenLine</span></strong></dt>
<dd>
<p>The line number where the token occurred.</p>
</dd>
</dl>
<p>The code must return:</p>
<dl>
<dt><strong><span class="code">{token,Token}</span></strong></dt>
<dd>
<p>Return <span class="code">Token</span> to the caller.</p>
</dd>
<dt><strong><span class="code">{end_token,Token}</span></strong></dt>
<dd>
<p>Return <span class="code">Token</span> and is last token in a tokens call.</p>
</dd>
<dt><strong><span class="code">skip_token</span></strong></dt>
<dd>
<p>Skip this token completely.</p>
</dd>
<dt><strong><span class="code">{error,ErrString}</span></strong></dt>
<dd>
<p>An error in the token, <span class="code">ErrString</span> is a string
describing the error.</p>
</dd>
</dl>
<p>It is also possible to push back characters into the input
characters with the following returns:</p>
<ul>
<li><span class="code">{token,Token,PushBackList}</span></li>
<li><span class="code">{end_token,Token,PushBackList}</span></li>
<li><span class="code">{skip_token,PushBackList}</span></li>
</ul>
<p>These have the same meanings as the normal returns but the
characters in <span class="code">PushBackList</span> will be prepended to the input
characters and scanned for the next token. Note that pushing
back a newline will mean the line numbering will no longer be
correct.</p>
<div class="note">
<div class="label">Note</div>
<div class="content"><p><p>Pushing back characters gives you unexpected
possibilities to cause the scanner to loop!</p></p></div>
</div>
<p>The following example would match a simple Erlang integer or
float and return a token which could be sent to the Erlang
parser:</p>
<div class="example"><pre>
D = [0-9]
{D}+ :
{token,{integer,TokenLine,list_to_integer(TokenChars)}}.
{D}+\.{D}+((E|e)(\+|\-)?{D}+)? :
{token,{float,TokenLine,list_to_float(TokenChars)}}.</pre></div>
<p>The Erlang code in the "Erlang code." section is written into
the output file directly after the module declaration and
predefined exports declaration so it is possible to add extra
exports, define imports and other attributes which are then
visible in the whole file.</p>
</div>
<h3><a name="id63605">Regular Expressions</a></h3>
<div class="REFBODY">
<p>The regular expressions allowed here is a subset of the set
found in <span class="code">egrep</span> and in the AWK programming language, as
defined in the book, The AWK Programming Language, by A. V. Aho,
B. W. Kernighan, P. J. Weinberger. They are composed of the
following characters:</p>
<dl>
<dt><strong><span class="code">c</span></strong></dt>
<dd>
<p>Matches the non-metacharacter c.</p>
</dd>
<dt><strong><span class="code">\c</span></strong></dt>
<dd>
<p>Matches the escape sequence or literal character c.</p>
</dd>
<dt><strong><span class="code">.</span></strong></dt>
<dd>
<p>Matches any character.</p>
</dd>
<dt><strong><span class="code">^</span></strong></dt>
<dd>
<p>Matches the beginning of a string.</p>
</dd>
<dt><strong><span class="code">$</span></strong></dt>
<dd><p>Matches the end of a string.</p></dd>
<dt><strong><span class="code">[abc...]</span></strong></dt>
<dd>
<p>Character class, which matches any of the characters
<span class="code">abc...</span>. Character ranges are specified by a pair of
characters separated by a <span class="code">-</span>.</p>
</dd>
<dt><strong><span class="code">[^abc...]</span></strong></dt>
<dd>
<p>Negated character class, which matches any character
except <span class="code">abc...</span>.</p>
</dd>
<dt><strong><span class="code">r1 | r2</span></strong></dt>
<dd>
<p>Alternation. It matches either <span class="code">r1</span> or <span class="code">r2</span>.</p>
</dd>
<dt><strong><span class="code">r1r2</span></strong></dt>
<dd>
<p>Concatenation. It matches <span class="code">r1</span> and then <span class="code">r2</span>.</p>
</dd>
<dt><strong><span class="code">r+</span></strong></dt>
<dd>
<p>Matches one or more <span class="code">rs</span>.</p>
</dd>
<dt><strong><span class="code">r*</span></strong></dt>
<dd>
<p>Matches zero or more <span class="code">rs</span>.</p>
</dd>
<dt><strong><span class="code">r?</span></strong></dt>
<dd>
<p>Matches zero or one <span class="code">rs</span>.</p>
</dd>
<dt><strong><span class="code">(r)</span></strong></dt>
<dd>
<p>Grouping. It matches <span class="code">r</span>.</p>
</dd>
</dl>
<p>The escape sequences allowed are the same as for Erlang strings:</p>
<dl>
<dt><strong><span class="code">\b</span></strong></dt>
<dd><p>Backspace.</p></dd>
<dt><strong><span class="code">\f</span></strong></dt>
<dd><p>Form feed.</p></dd>
<dt><strong><span class="code">\n</span></strong></dt>
<dd><p>Newline (line feed).</p></dd>
<dt><strong><span class="code">\r</span></strong></dt>
<dd><p>Carriage return.</p></dd>
<dt><strong><span class="code">\t</span></strong></dt>
<dd><p>Tab.</p></dd>
<dt><strong><span class="code">\e</span></strong></dt>
<dd><p>Escape.</p></dd>
<dt><strong><span class="code">\v</span></strong></dt>
<dd><p>Vertical tab.</p></dd>
<dt><strong><span class="code">\s</span></strong></dt>
<dd><p>Space.</p></dd>
<dt><strong><span class="code">\d</span></strong></dt>
<dd><p>Delete.</p></dd>
<dt><strong><span class="code">\ddd</span></strong></dt>
<dd><p>The octal value <span class="code">ddd</span>.</p></dd>
<dt><strong><span class="code">\xhh</span></strong></dt>
<dd><p>The hexadecimal value <span class="code">hh</span>.</p></dd>
<dt><strong><span class="code">\x{h...}</span></strong></dt>
<dd><p>The hexadecimal value <span class="code">h...</span>.</p></dd>
<dt><strong><span class="code">\c</span></strong></dt>
<dd>
<p>Any other character literally, for example <span class="code">\\</span> for
backslash, <span class="code">\"</span> for <span class="code">"</span>.</p>
</dd>
</dl>
<p>The following examples define Erlang data types:</p>
<div class="example"><pre>
Atoms [a-z][0-9a-zA-Z_]*
Variables [A-Z_][0-9a-zA-Z_]*
Floats (\+|-)?[0-9]+\.[0-9]+((E|e)(\+|-)?[0-9]+)?</pre></div>
<div class="note">
<div class="label">Note</div>
<div class="content"><p><p>Anchoring a regular expression with <span class="code">^</span> and <span class="code">$</span>
is not implemented in the current version of Leex and just
generates a parse error.</p></p></div>
</div>
</div>
</div>
<div class="footer">
<hr>
<p>Copyright © 1997-2012 Ericsson AB. All Rights Reserved.</p>
</div>
</div>
</div></body>
</html>
distrib
>
Fedora
>
17
>
i386
>
media
>
updates
>
by-pkgid
>
675c8c8167236dfcf8d66da674f931e8
>
files
>
1161
