<html><head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> <title>str.tokenize.keep.delimiters</title><link rel="stylesheet" type="text/css" href="../reference.css"><meta name="generator" content="DocBook XSL Stylesheets V1.79.2"><link rel="home" href="../index.html" title="DocBook XSL Stylesheets: Reference Documentation"><link rel="up" href="generallibrary.html" title="General Library Templates"><link rel="prev" href="trim.text.html" title="trim.text"><link rel="next" href="apply-string-subst-map.html" title="apply-string-subst-map"><link rel="copyright" href="copyright.html" title="License"></head><body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"><div class="navheader"><table width="100%" summary="Navigation header"><tr><th colspan="3" align="center">str.tokenize.keep.delimiters</th></tr><tr><td width="20%" align="left"><a accesskey="p" href="trim.text.html">Prev</a> </td><th width="60%" align="center">General Library Templates</th><td width="20%" align="right"> <a accesskey="n" href="apply-string-subst-map.html">Next</a></td></tr></table><hr></div><div class="refentry"><a name="str.tokenize.keep.delimiters"></a><div class="titlepage"></div><div class="refnamediv"><h2>str.tokenize.keep.delimiters</h2><p>str.tokenize.keep.delimiters — Tokenize a string while preserving any delimiters</p></div><div class="refsect1"><a name="d0e33490"></a><h2>Description</h2><p>Based on the occurrence of one or more delimiter characters, this function breaks a string into a list of tokens and delimiters, marking up each of the tokens with a <code class="sgmltag-element">token</code> element and preserving the delimiters as text nodes between the tokens.</p><div class="note" style="margin-left: 0.5in; margin-right: 0.5in;"><h3 class="title">Note</h3><p>This function is a very slightly modified version of a function from the <a class="link" href="http://www.exslt.org/" target="_top">EXSLT site</a>. The original is available at: </p><div class="blockquote"><blockquote class="blockquote"><p><a class="link" href="http://www.exslt.org/str/functions/tokenize/str.tokenize.template.xsl" target="_top">http://www.exslt.org/str/functions/tokenize/str.tokenize.template.xsl</a></p></blockquote></div><p> The <code class="function">str.tokenize.keep.delimiters</code> function differs only in that it preserves the delimiters instead of discarding them.</p></div><a name="str.tokenize.keep.delimiters.frag"></a><pre class="programlisting"> <xsl:template name="str.tokenize.keep.delimiters"> <xsl:param name="string" select="''"></xsl:param> <xsl:param name="delimiters" select="' '"></xsl:param> <xsl:choose> <xsl:when test="not($string)"></xsl:when> <xsl:when test="not($delimiters)"> <xsl:call-template name="str.tokenize.keep.delimiters-characters"> <xsl:with-param name="string" select="$string"></xsl:with-param> </xsl:call-template> </xsl:when> <xsl:otherwise> <xsl:call-template name="str.tokenize.keep.delimiters-delimiters"> <xsl:with-param name="string" select="$string"></xsl:with-param> <xsl:with-param name="delimiters" select="$delimiters"></xsl:with-param> </xsl:call-template> </xsl:otherwise> </xsl:choose> </xsl:template> <xsl:template name="str.tokenize.keep.delimiters-characters"> <xsl:param name="string"></xsl:param> <xsl:if test="$string"> <ssb:token><xsl:value-of select="substring($string, 1, 1)"></xsl:value-of></ssb:token> <xsl:call-template name="str.tokenize.keep.delimiters-characters"> <xsl:with-param name="string" select="substring($string, 2)"></xsl:with-param> </xsl:call-template> </xsl:if> </xsl:template> <xsl:template name="str.tokenize.keep.delimiters-delimiters"> <xsl:param name="string"></xsl:param> <xsl:param name="delimiters"></xsl:param> <xsl:variable name="delimiter" select="substring($delimiters, 1, 1)"></xsl:variable> <xsl:choose> <xsl:when test="not($delimiter)"> <ssb:token><xsl:value-of select="$string"></xsl:value-of></ssb:token> </xsl:when> <xsl:when test="contains($string, $delimiter)"> <xsl:if test="not(starts-with($string, $delimiter))"> <xsl:call-template name="str.tokenize.keep.delimiters-delimiters"> <xsl:with-param name="string" select="substring-before($string, $delimiter)"></xsl:with-param> <xsl:with-param name="delimiters" select="substring($delimiters, 2)"></xsl:with-param> </xsl:call-template> </xsl:if> <!-- output each delimiter --> <xsl:value-of select="$delimiter"></xsl:value-of> <xsl:call-template name="str.tokenize.keep.delimiters-delimiters"> <xsl:with-param name="string" select="substring-after($string, $delimiter)"></xsl:with-param> <xsl:with-param name="delimiters" select="$delimiters"></xsl:with-param> </xsl:call-template> </xsl:when> <xsl:otherwise> <xsl:call-template name="str.tokenize.keep.delimiters-delimiters"> <xsl:with-param name="string" select="$string"></xsl:with-param> <xsl:with-param name="delimiters" select="substring($delimiters, 2)"></xsl:with-param> </xsl:call-template> </xsl:otherwise> </xsl:choose> </xsl:template> </pre></div></div><div class="navfooter"><hr><table width="100%" summary="Navigation footer"><tr><td width="40%" align="left"><a accesskey="p" href="trim.text.html">Prev</a> </td><td width="20%" align="center"><a accesskey="u" href="generallibrary.html">Up</a></td><td width="40%" align="right"> <a accesskey="n" href="apply-string-subst-map.html">Next</a></td></tr><tr><td width="40%" align="left" valign="top">trim.text </td><td width="20%" align="center"><a accesskey="h" href="../index.html">Home</a></td><td width="40%" align="right" valign="top"> apply-string-subst-map</td></tr></table></div></body></html>