<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> <html> <head> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <title>Split given source into PHP tokens</title> </head> <body><div class="manualnavbar" style="text-align: center;"> <div class="prev" style="text-align: left; float: left;"><a href="ref.tokenizer.html">Tokenizer Functions</a></div> <div class="next" style="text-align: right; float: right;"><a href="function.token-name.html">token_name</a></div> <div class="up"><a href="ref.tokenizer.html">Tokenizer Functions</a></div> <div class="home"><a href="index.html">PHP Manual</a></div> </div><hr /><div id="function.token-get-all" class="refentry"> <div class="refnamediv"> <h1 class="refname">token_get_all</h1> <p class="verinfo">(PHP 4 >= 4.2.0, PHP 5, PHP 7)</p><p class="refpurpose"><span class="refname">token_get_all</span> — <span class="dc-title">Split given source into PHP tokens</span></p> </div> <div class="refsect1 description" id="refsect1-function.token-get-all-description"> <h3 class="title">Description</h3> <div class="methodsynopsis dc-description"> <span class="type">array</span> <span class="methodname"><strong>token_get_all</strong></span> ( <span class="methodparam"><span class="type">string</span> <code class="parameter">$source</code></span> [, <span class="methodparam"><span class="type">int</span> <code class="parameter">$flags</code><span class="initializer"> = 0</span></span> ] )</div> <p class="para rdfs-comment"> <span class="function"><strong>token_get_all()</strong></span> parses the given <code class="parameter">source</code> string into PHP language tokens using the Zend engine's lexical scanner. </p> <p class="para"> For a list of parser tokens, see <a href="tokens.html" class="xref">List of Parser Tokens</a>, or use <span class="function"><a href="function.token-name.html" class="function">token_name()</a></span> to translate a token value into its string representation. </p> </div> <div class="refsect1 parameters" id="refsect1-function.token-get-all-parameters"> <h3 class="title">Parameters</h3> <p class="para"> <dl> <dt> <code class="parameter">source</code></dt> <dd> <p class="para"> The PHP source to parse. </p> </dd> <dt> <code class="parameter">flags</code></dt> <dd> <p class="para"> Valid flags: <ul class="itemizedlist"> <li class="listitem"> <span class="simpara"> <strong><code>TOKEN_PARSE</code></strong> - Recognises the ability to use reserved words in specific contexts. </span> </li> </ul> </p> </dd> </dl> </p> </div> <div class="refsect1 returnvalues" id="refsect1-function.token-get-all-returnvalues"> <h3 class="title">Return Values</h3> <p class="para"> An array of token identifiers. Each individual token identifier is either a single character (i.e.: <em>;</em>, <em>.</em>, <em>></em>, <em>!</em>, etc...), or a three element array containing the token index in element 0, the string content of the original token in element 1 and the line number in element 2. </p> </div> <div class="refsect1 changelog" id="refsect1-function.token-get-all-changelog"> <h3 class="title">Changelog</h3> <p class="para"> <table class="doctable informaltable"> <thead> <tr> <th>Version</th> <th>Description</th> </tr> </thead> <tbody class="tbody"> <tr> <td>7.0.0</td> <td> Added the optional <code class="parameter">flags</code> parameter along with the <strong><code>TOKEN_PARSE</code></strong> flag. </td> </tr> <tr> <td>5.2.2</td> <td> Line numbers are returned in element 2 </td> </tr> </tbody> </table> </p> </div> <div class="refsect1 examples" id="refsect1-function.token-get-all-examples"> <h3 class="title">Examples</h3> <p class="para"> <div class="example" id="example-5007"> <p><strong>Example #1 <span class="function"><strong>token_get_all()</strong></span> example</strong></p> <div class="example-contents"> <div class="phpcode"><code><span style="color: #000000"> <span style="color: #0000BB"><?php<br />$tokens </span><span style="color: #007700">= </span><span style="color: #0000BB">token_get_all</span><span style="color: #007700">(</span><span style="color: #DD0000">'<?php echo; ?>'</span><span style="color: #007700">);<br /><br />foreach (</span><span style="color: #0000BB">$tokens </span><span style="color: #007700">as </span><span style="color: #0000BB">$token</span><span style="color: #007700">) {<br /> if (</span><span style="color: #0000BB">is_array</span><span style="color: #007700">(</span><span style="color: #0000BB">$token</span><span style="color: #007700">)) {<br /> echo </span><span style="color: #DD0000">"Line </span><span style="color: #007700">{</span><span style="color: #0000BB">$token</span><span style="color: #007700">[</span><span style="color: #0000BB">2</span><span style="color: #007700">]}</span><span style="color: #DD0000">: "</span><span style="color: #007700">, </span><span style="color: #0000BB">token_name</span><span style="color: #007700">(</span><span style="color: #0000BB">$token</span><span style="color: #007700">[</span><span style="color: #0000BB">0</span><span style="color: #007700">]), </span><span style="color: #DD0000">" ('</span><span style="color: #007700">{</span><span style="color: #0000BB">$token</span><span style="color: #007700">[</span><span style="color: #0000BB">1</span><span style="color: #007700">]}</span><span style="color: #DD0000">')"</span><span style="color: #007700">, </span><span style="color: #0000BB">PHP_EOL</span><span style="color: #007700">;<br /> }<br />}<br /></span><span style="color: #0000BB">?></span> </span> </code></div> </div> <div class="example-contents"><p>The above example will output something similar to:</p></div> <div class="example-contents screen"> <div class="cdata"><pre> Line 1: T_OPEN_TAG ('<?php ') Line 1: T_ECHO ('echo') Line 1: T_WHITESPACE (' ') Line 1: T_CLOSE_TAG ('?>') </pre></div> </div> </div> </p> <p class="para"> <div class="example" id="example-5008"> <p><strong>Example #2 <span class="function"><strong>token_get_all()</strong></span> incorrect usage example</strong></p> <div class="example-contents"> <div class="phpcode"><code><span style="color: #000000"> <span style="color: #0000BB"><?php<br />$tokens </span><span style="color: #007700">= </span><span style="color: #0000BB">token_get_all</span><span style="color: #007700">(</span><span style="color: #DD0000">'/* comment */'</span><span style="color: #007700">);<br /><br />foreach (</span><span style="color: #0000BB">$tokens </span><span style="color: #007700">as </span><span style="color: #0000BB">$token</span><span style="color: #007700">) {<br /> if (</span><span style="color: #0000BB">is_array</span><span style="color: #007700">(</span><span style="color: #0000BB">$token</span><span style="color: #007700">)) {<br /> echo </span><span style="color: #DD0000">"Line </span><span style="color: #007700">{</span><span style="color: #0000BB">$token</span><span style="color: #007700">[</span><span style="color: #0000BB">2</span><span style="color: #007700">]}</span><span style="color: #DD0000">: "</span><span style="color: #007700">, </span><span style="color: #0000BB">token_name</span><span style="color: #007700">(</span><span style="color: #0000BB">$token</span><span style="color: #007700">[</span><span style="color: #0000BB">0</span><span style="color: #007700">]), </span><span style="color: #DD0000">" ('</span><span style="color: #007700">{</span><span style="color: #0000BB">$token</span><span style="color: #007700">[</span><span style="color: #0000BB">1</span><span style="color: #007700">]}</span><span style="color: #DD0000">')"</span><span style="color: #007700">, </span><span style="color: #0000BB">PHP_EOL</span><span style="color: #007700">;<br /> }<br />}<br /></span><span style="color: #0000BB">?></span> </span> </code></div> </div> <div class="example-contents"><p>The above example will output something similar to:</p></div> <div class="example-contents screen"> <div class="cdata"><pre> Line 1: T_INLINE_HTML ('/* comment */') </pre></div> </div> </div> Note in the previous example that the string is parsed as <strong><code>T_INLINE_HTML</code></strong> rather than the expected <strong><code>T_COMMENT</code></strong>. This is because no open tag was used in the code provided. This would be equivalent to putting a comment outside of the PHP tags in a normal file. </p> <p class="para"> <div class="example" id="example-5009"> <p><strong>Example #3 <span class="function"><strong>token_get_all()</strong></span> on a class using a reserved word example </strong></p> <div class="example-contents"> <div class="phpcode"><code><span style="color: #000000"> <span style="color: #0000BB"><?php<br /><br />$source </span><span style="color: #007700">= <<<'code'<br /></span><span style="color: #DD0000"><?php<br /><br />class A<br />{<br /> const PUBLIC = 1;<br />}<br /></span><span style="color: #007700">code;<br /><br /></span><span style="color: #0000BB">$tokens </span><span style="color: #007700">= </span><span style="color: #0000BB">token_get_all</span><span style="color: #007700">(</span><span style="color: #0000BB">$source</span><span style="color: #007700">, </span><span style="color: #0000BB">TOKEN_PARSE</span><span style="color: #007700">);<br /><br />foreach (</span><span style="color: #0000BB">$tokens </span><span style="color: #007700">as </span><span style="color: #0000BB">$token</span><span style="color: #007700">) {<br /> if (</span><span style="color: #0000BB">is_array</span><span style="color: #007700">(</span><span style="color: #0000BB">$token</span><span style="color: #007700">)) {<br /> echo </span><span style="color: #0000BB">token_name</span><span style="color: #007700">(</span><span style="color: #0000BB">$token</span><span style="color: #007700">[</span><span style="color: #0000BB">0</span><span style="color: #007700">]) , </span><span style="color: #0000BB">PHP_EOL</span><span style="color: #007700">;<br /> }<br />}<br /></span><span style="color: #0000BB">?></span> </span> </code></div> </div> <div class="example-contents"><p>The above example will output something similar to:</p></div> <div class="example-contents screen"> <div class="cdata"><pre> T_OPEN_TAG T_WHITESPACE T_CLASS T_WHITESPACE T_STRING T_CONST T_WHITESPACE T_STRING T_LNUMBER </pre></div> </div> </div> Without the <strong><code>TOKEN_PARSE</code></strong> flag, the penultimate token (<strong><code>T_STRING</code></strong>) would have been <strong><code>T_PUBLIC</code></strong>. </p> </div> <div class="refsect1 seealso" id="refsect1-function.token-get-all-seealso"> <h3 class="title">See Also</h3> <p class="para"> <ul class="simplelist"> <li class="member"><span class="function"><a href="function.token-name.html" class="function" rel="rdfs-seeAlso">token_name()</a> - Get the symbolic name of a given PHP token</span></li> </ul> </p> </div> </div><hr /><div class="manualnavbar" style="text-align: center;"> <div class="prev" style="text-align: left; float: left;"><a href="ref.tokenizer.html">Tokenizer Functions</a></div> <div class="next" style="text-align: right; float: right;"><a href="function.token-name.html">token_name</a></div> <div class="up"><a href="ref.tokenizer.html">Tokenizer Functions</a></div> <div class="home"><a href="index.html">PHP Manual</a></div> </div></body></html>