<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<title>ANTLR3C: ANTLR3_TOKEN_SOURCE_struct Struct Reference</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript">
$(document).ready(initResizable);
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css"/>
</head>
<body>
<!-- Generated by Doxygen 1.7.3 -->
<div id="top">
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td style="padding-left: 0.5em;">
<div id="projectname">ANTLR3C <span id="projectnumber">3.1.2</span></div>
</td>
</tr>
</tbody>
</table>
</div>
<div id="navrow1" class="tabs">
<ul class="tablist">
<li><a href="index.html"><span>Main Page</span></a></li>
<li><a href="pages.html"><span>Related Pages</span></a></li>
<li><a href="modules.html"><span>Modules</span></a></li>
<li class="current"><a href="annotated.html"><span>Data Structures</span></a></li>
<li><a href="files.html"><span>Files</span></a></li>
<li><a href="dirs.html"><span>Directories</span></a></li>
</ul>
</div>
<div id="navrow2" class="tabs2">
<ul class="tablist">
<li><a href="annotated.html"><span>Data Structures</span></a></li>
<li><a href="classes.html"><span>Data Structure Index</span></a></li>
<li><a href="functions.html"><span>Data Fields</span></a></li>
</ul>
</div>
</div>
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
initNavTree('struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html','');
</script>
<div id="doc-content">
<div class="header">
<div class="summary">
<a href="#pub-attribs">Data Fields</a> </div>
<div class="headertitle">
<h1>ANTLR3_TOKEN_SOURCE_struct Struct Reference</h1> </div>
</div>
<div class="contents">
<!-- doxytag: class="ANTLR3_TOKEN_SOURCE_struct" -->
<p>Definition of a token source, which has a pointer to a function that returns the next token (using a token factory if it is going to be efficient) and a pointer to an ANTLR3_INPUT_STREAM.
<a href="#_details">More...</a></p>
<p><code>#include <antlr3tokenstream.h></code></p>
<div class="dynheader">
Collaboration diagram for ANTLR3_TOKEN_SOURCE_struct:</div>
<div class="dyncontent">
<div class="center"><img src="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct__coll__graph.png" border="0" usemap="#_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct_coll__map" alt="Collaboration graph"/></div>
<map name="_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct_coll__map" id="_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct_coll__map">
<area shape="rect" id="node2" href="struct_a_n_t_l_r3___c_o_m_m_o_n___t_o_k_e_n__struct.html" title="The definition of an ANTLR3 common token structure, which all implementations of a token stream shoul..." alt="" coords="372,517,655,544"/><area shape="rect" id="node4" href="struct_a_n_t_l_r3___s_t_r_i_n_g__struct.html" title="Base string class tracks the allocations and provides simple string tracking functions." alt="" coords="250,117,471,144"/><area shape="rect" id="node7" href="struct_a_n_t_l_r3___s_t_r_i_n_g___f_a_c_t_o_r_y__struct.html" title="Definition of the string factory interface, which creates and tracks strings for you of various shape..." alt="" coords="442,305,743,332"/><area shape="rect" id="node14" href="struct_a_n_t_l_r3___i_n_p_u_t___s_t_r_e_a_m__struct.html" title="Master context structure for an ANTLR3 C runtime based input stream." alt="" coords="262,427,544,453"/><area shape="rect" id="node17" href="struct_a_n_t_l_r3___i_n_t___s_t_r_e_a_m__struct.html" title="ANTLR3_INT_STREAM_struct" alt="" coords="27,305,288,332"/><area shape="rect" id="node10" href="struct_a_n_t_l_r3___v_e_c_t_o_r__struct.html" title="ANTLR3_VECTOR_struct" alt="" coords="656,117,878,144"/><area shape="rect" id="node12" href="struct_a_n_t_l_r3___v_e_c_t_o_r___e_l_e_m_e_n_t__struct.html" title="ANTLR3_VECTOR_ELEMENT_struct" alt="" coords="616,5,918,32"/></map>
<center><span class="legend">[<a target="top" href="graph_legend.html">legend</a>]</span></center></div>
<table class="memberdecls">
<tr><td colspan="2"><h2><a name="pub-attribs"></a>
Data Fields</h2></td></tr>
<tr><td class="memItemLeft" align="right" valign="top"><a class="el" href="struct_a_n_t_l_r3___c_o_m_m_o_n___t_o_k_e_n__struct.html">ANTLR3_COMMON_TOKEN</a> </td><td class="memItemRight" valign="bottom"><a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#a40b1631371c9aa2872f411c691d7348e">eofToken</a></td></tr>
<tr><td class="mdescLeft"> </td><td class="mdescRight">A special pre-allocated token, which signifies End Of Tokens. <a href="#a40b1631371c9aa2872f411c691d7348e"></a><br/></td></tr>
<tr><td class="memItemLeft" align="right" valign="top"><a class="el" href="struct_a_n_t_l_r3___s_t_r_i_n_g__struct.html">pANTLR3_STRING</a> </td><td class="memItemRight" valign="bottom"><a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#ac71607f39f82e74a177e44bcada52343">fileName</a></td></tr>
<tr><td class="mdescLeft"> </td><td class="mdescRight">When the token source is constructed, it is populated with the file name from whence the tokens were produced by the lexer. <a href="#ac71607f39f82e74a177e44bcada52343"></a><br/></td></tr>
<tr><td class="memItemLeft" align="right" valign="top"><a class="el" href="struct_a_n_t_l_r3___c_o_m_m_o_n___t_o_k_e_n__struct.html">pANTLR3_COMMON_TOKEN</a>(* </td><td class="memItemRight" valign="bottom"><a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#ac1e745731ff025ccd806f6becfb4c4ea">nextToken</a> )(struct <a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html">ANTLR3_TOKEN_SOURCE_struct</a> *tokenSource)</td></tr>
<tr><td class="mdescLeft"> </td><td class="mdescRight">Pointer to a function that returns the next token in the stream. <a href="#ac1e745731ff025ccd806f6becfb4c4ea"></a><br/></td></tr>
<tr><td class="memItemLeft" align="right" valign="top"><a class="el" href="struct_a_n_t_l_r3___c_o_m_m_o_n___t_o_k_e_n__struct.html">ANTLR3_COMMON_TOKEN</a> </td><td class="memItemRight" valign="bottom"><a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#a417f3d7f340e55046200a8dc20bdf5bd">skipToken</a></td></tr>
<tr><td class="mdescLeft"> </td><td class="mdescRight">A special pre-allocated token, which is returned by <a class="el" href="antlr3lexer_8c.html#ae31c6940c452ca2d98ba1e85e2a091c9">mTokens()</a> if the lexer rule said to just skip the generated token altogether. <a href="#a417f3d7f340e55046200a8dc20bdf5bd"></a><br/></td></tr>
<tr><td class="memItemLeft" align="right" valign="top"><a class="el" href="struct_a_n_t_l_r3___s_t_r_i_n_g___f_a_c_t_o_r_y__struct.html">pANTLR3_STRING_FACTORY</a> </td><td class="memItemRight" valign="bottom"><a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#ab65d5aa5431af0423d02ee9777417bd9">strFactory</a></td></tr>
<tr><td class="mdescLeft"> </td><td class="mdescRight">Whoever is providing tokens, needs to provide a string factory too. <a href="#ab65d5aa5431af0423d02ee9777417bd9"></a><br/></td></tr>
<tr><td class="memItemLeft" align="right" valign="top">void * </td><td class="memItemRight" valign="bottom"><a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#a8ea66f6d566682d41b763ff27f6a0916">super</a></td></tr>
<tr><td class="mdescLeft"> </td><td class="mdescRight">Whatever is supplying the token source interface, needs a pointer to itself so that this pointer can be passed to it when the nextToken function is called. <a href="#a8ea66f6d566682d41b763ff27f6a0916"></a><br/></td></tr>
</table>
<hr/><a name="_details"></a><h2>Detailed Description</h2>
<div class="textblock"><p>Definition of a token source, which has a pointer to a function that returns the next token (using a token factory if it is going to be efficient) and a pointer to an ANTLR3_INPUT_STREAM. </p>
<p>This is slightly different to the Java interface because we have no way to implement multiple interfaces without defining them in the interface structure or casting (void *), which is too convoluted. </p>
</div><hr/><h2>Field Documentation</h2>
<a class="anchor" id="a40b1631371c9aa2872f411c691d7348e"></a><!-- doxytag: member="ANTLR3_TOKEN_SOURCE_struct::eofToken" ref="a40b1631371c9aa2872f411c691d7348e" args="" -->
<div class="memitem">
<div class="memproto">
<table class="memname">
<tr>
<td class="memname"><a class="el" href="struct_a_n_t_l_r3___c_o_m_m_o_n___t_o_k_e_n__struct.html">ANTLR3_COMMON_TOKEN</a> <a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#a40b1631371c9aa2872f411c691d7348e">ANTLR3_TOKEN_SOURCE_struct::eofToken</a></td>
</tr>
</table>
</div>
<div class="memdoc">
<p>A special pre-allocated token, which signifies End Of Tokens. </p>
<p>Because this must be set up with the current input index and so on, we embed the structure and return the address of it. It is marked as factoryMade, so that it is never attempted to be freed. </p>
<p>Referenced by <a class="el" href="antlr3defs_8h.html#aad5d94a0ad6c1d4788f95debb67cf642">antlr3LexerNew()</a>, <a class="el" href="antlr3lexer_8c.html#ad0765a37fb94635afbfaad254c12c496">nextTokenStr()</a>, <a class="el" href="antlr3lexer_8c.html#a3f121d133cb810a9687aee3089e33be8">setCharStream()</a>, and <a class="el" href="antlr3tokenstream_8c.html#a797f427bfdba8fc5bf09dd4aa2e2d30a">tokLT()</a>.</p>
</div>
</div>
<a class="anchor" id="ac71607f39f82e74a177e44bcada52343"></a><!-- doxytag: member="ANTLR3_TOKEN_SOURCE_struct::fileName" ref="ac71607f39f82e74a177e44bcada52343" args="" -->
<div class="memitem">
<div class="memproto">
<table class="memname">
<tr>
<td class="memname"><a class="el" href="struct_a_n_t_l_r3___s_t_r_i_n_g__struct.html">pANTLR3_STRING</a> <a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#ac71607f39f82e74a177e44bcada52343">ANTLR3_TOKEN_SOURCE_struct::fileName</a></td>
</tr>
</table>
</div>
<div class="memdoc">
<p>When the token source is constructed, it is populated with the file name from whence the tokens were produced by the lexer. </p>
<p>This pointer is a copy of the one supplied by the CharStream (and may be NULL) so should not be manipulated other than to copy or print it. </p>
<p>Referenced by <a class="el" href="antlr3lexer_8c.html#a3f121d133cb810a9687aee3089e33be8">setCharStream()</a>.</p>
</div>
</div>
<a class="anchor" id="ac1e745731ff025ccd806f6becfb4c4ea"></a><!-- doxytag: member="ANTLR3_TOKEN_SOURCE_struct::nextToken" ref="ac1e745731ff025ccd806f6becfb4c4ea" args=")(struct ANTLR3_TOKEN_SOURCE_struct *tokenSource)" -->
<div class="memitem">
<div class="memproto">
<table class="memname">
<tr>
<td class="memname"><a class="el" href="struct_a_n_t_l_r3___c_o_m_m_o_n___t_o_k_e_n__struct.html">pANTLR3_COMMON_TOKEN</a>(* <a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#ac1e745731ff025ccd806f6becfb4c4ea">ANTLR3_TOKEN_SOURCE_struct::nextToken</a>)(struct <a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html">ANTLR3_TOKEN_SOURCE_struct</a> *tokenSource)</td>
</tr>
</table>
</div>
<div class="memdoc">
<p>Pointer to a function that returns the next token in the stream. </p>
<p>Referenced by <a class="el" href="antlr3defs_8h.html#aad5d94a0ad6c1d4788f95debb67cf642">antlr3LexerNew()</a>, and <a class="el" href="antlr3tokenstream_8c.html#a6e9bf0e3483b4361c86ec64e495cc160">fillBuffer()</a>.</p>
</div>
</div>
<a class="anchor" id="a417f3d7f340e55046200a8dc20bdf5bd"></a><!-- doxytag: member="ANTLR3_TOKEN_SOURCE_struct::skipToken" ref="a417f3d7f340e55046200a8dc20bdf5bd" args="" -->
<div class="memitem">
<div class="memproto">
<table class="memname">
<tr>
<td class="memname"><a class="el" href="struct_a_n_t_l_r3___c_o_m_m_o_n___t_o_k_e_n__struct.html">ANTLR3_COMMON_TOKEN</a> <a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#a417f3d7f340e55046200a8dc20bdf5bd">ANTLR3_TOKEN_SOURCE_struct::skipToken</a></td>
</tr>
</table>
</div>
<div class="memdoc">
<p>A special pre-allocated token, which is returned by <a class="el" href="antlr3lexer_8c.html#ae31c6940c452ca2d98ba1e85e2a091c9">mTokens()</a> if the lexer rule said to just skip the generated token altogether. </p>
<p>Having this single token stops us wasting memory by have the token factory actually create something that we are going to SKIP(); anyway. </p>
<p>Referenced by <a class="el" href="antlr3defs_8h.html#aad5d94a0ad6c1d4788f95debb67cf642">antlr3LexerNew()</a>, and <a class="el" href="antlr3lexer_8c.html#ad0765a37fb94635afbfaad254c12c496">nextTokenStr()</a>.</p>
</div>
</div>
<a class="anchor" id="ab65d5aa5431af0423d02ee9777417bd9"></a><!-- doxytag: member="ANTLR3_TOKEN_SOURCE_struct::strFactory" ref="ab65d5aa5431af0423d02ee9777417bd9" args="" -->
<div class="memitem">
<div class="memproto">
<table class="memname">
<tr>
<td class="memname"><a class="el" href="struct_a_n_t_l_r3___s_t_r_i_n_g___f_a_c_t_o_r_y__struct.html">pANTLR3_STRING_FACTORY</a> <a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#ab65d5aa5431af0423d02ee9777417bd9">ANTLR3_TOKEN_SOURCE_struct::strFactory</a></td>
</tr>
</table>
</div>
<div class="memdoc">
<p>Whoever is providing tokens, needs to provide a string factory too. </p>
<p>Referenced by <a class="el" href="antlr3defs_8h.html#aad5d94a0ad6c1d4788f95debb67cf642">antlr3LexerNew()</a>, <a class="el" href="antlr3lexer_8c.html#a3f121d133cb810a9687aee3089e33be8">setCharStream()</a>, and <a class="el" href="antlr3tokenstream_8c.html#aa92e164be569dcd030d5f2248b002d72">toStringSS()</a>.</p>
</div>
</div>
<a class="anchor" id="a8ea66f6d566682d41b763ff27f6a0916"></a><!-- doxytag: member="ANTLR3_TOKEN_SOURCE_struct::super" ref="a8ea66f6d566682d41b763ff27f6a0916" args="" -->
<div class="memitem">
<div class="memproto">
<table class="memname">
<tr>
<td class="memname">void* <a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html#a8ea66f6d566682d41b763ff27f6a0916">ANTLR3_TOKEN_SOURCE_struct::super</a></td>
</tr>
</table>
</div>
<div class="memdoc">
<p>Whatever is supplying the token source interface, needs a pointer to itself so that this pointer can be passed to it when the nextToken function is called. </p>
<p>Referenced by <a class="el" href="antlr3defs_8h.html#aad5d94a0ad6c1d4788f95debb67cf642">antlr3LexerNew()</a>, <a class="el" href="antlr3lexer_8c.html#ac9a1246efbb94ec9e0069b70c3c53502">nextToken()</a>, and <a class="el" href="antlr3lexer_8c.html#ad0765a37fb94635afbfaad254c12c496">nextTokenStr()</a>.</p>
</div>
</div>
<hr/>The documentation for this struct was generated from the following file:<ul>
<li>include/<a class="el" href="antlr3tokenstream_8h.html">antlr3tokenstream.h</a></li>
</ul>
</div>
</div>
<div id="nav-path" class="navpath">
<ul>
<li class="navelem"><a class="el" href="struct_a_n_t_l_r3___t_o_k_e_n___s_o_u_r_c_e__struct.html">ANTLR3_TOKEN_SOURCE_struct</a> </li>
<li class="footer">Generated on Tue Feb 8 2011 for ANTLR3C by 
<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.7.3 </li>
</ul>
</div>
</body>
</html>
