\section{ost::String\-Tokenizer Class Reference}
\label{classost_1_1_string_tokenizer}\index{ost::StringTokenizer@{ost::StringTokenizer}}
Splits delimited string into tokens.
{\tt \#include $<$tokenizer.h$>$}
\subsection*{Public Member Functions}
\begin{CompactItemize}
\item
{\bf String\-Tokenizer} (const char $\ast$str, const char $\ast$delim, bool skip\-All\-Delim=false, bool trim=false)
\begin{CompactList}\small\item\em creates a new {\bf String\-Tokenizer}{\rm (p.\,\pageref{classost_1_1_string_tokenizer})} for a string and a given set of delimiters. \item\end{CompactList}\item
{\bf String\-Tokenizer} (const char $\ast$s)
\begin{CompactList}\small\item\em create a new {\bf String\-Tokenizer}{\rm (p.\,\pageref{classost_1_1_string_tokenizer})} which splits the input string at whitespaces. \item\end{CompactList}\item
{\bf iterator} {\bf begin} () const
\begin{CompactList}\small\item\em returns the begin iterator \item\end{CompactList}\item
void {\bf set\-Delimiters} (const char $\ast$d)
\begin{CompactList}\small\item\em changes the set of delimiters used in subsequent iterations. \item\end{CompactList}\item
{\bf iterator} {\bf begin} (const char $\ast$d)
\begin{CompactList}\small\item\em returns a begin iterator with an alternate set of delimiters. \item\end{CompactList}\item
const {\bf iterator} \& {\bf end} () const
\begin{CompactList}\small\item\em the iterator marking the end. \item\end{CompactList}\end{CompactItemize}
\subsection*{Static Public Attributes}
\begin{CompactItemize}
\item
static const char $\ast$const {\bf SPACE}
\begin{CompactList}\small\item\em a delimiter string containing all usual whitespace delimiters. \item\end{CompactList}\end{CompactItemize}
\subsection*{Friends}
\begin{CompactItemize}
\item
class {\bf String\-Tokenizer::iterator}
\end{CompactItemize}
\subsection*{Classes}
\begin{CompactItemize}
\item
class {\bf iterator}
\begin{CompactList}\small\item\em The input forward iterator for tokens. \item\end{CompactList}\item
class {\bf No\-Such\-Element\-Exception}
\begin{CompactList}\small\item\em Exception thrown, if someone tried to read beyond the end of the tokens. \item\end{CompactList}\end{CompactItemize}
\subsection{Detailed Description}
Splits delimited string into tokens.
The {\bf String\-Tokenizer}{\rm (p.\,\pageref{classost_1_1_string_tokenizer})} takes a pointer to a string and a pointer to a string containing a number of possible delimiters. The {\bf String\-Tokenizer}{\rm (p.\,\pageref{classost_1_1_string_tokenizer})} provides an input forward iterator which allows to iterate through all tokens. An iterator behaves like a logical pointer to the tokens, i.e. to shift to the next token, you've to increment the iterator, you get the token by dereferencing the iterator.
Memory consumption: This class operates on the original string and only allocates memory for the individual tokens actually requested, so this class allocates at maximum the space required for the longest token in the given string. Since for each iteration, memory is reclaimed for the last token, you MAY NOT store pointers to them; if you need them afterwards, copy them. You may not modify the original string while you operate on it with the {\bf String\-Tokenizer}{\rm (p.\,\pageref{classost_1_1_string_tokenizer})}; the behaviour is undefined in that case.
The iterator has one special method 'next\-Delimiter()' which returns a character containing the next delimiter following this tokenization process or '$\backslash$0', if there are no following delimiters. In case of skip\-All\-Delim, it returns the FIRST delimiter.
With the method 'set\-Delimiters(const char$\ast$)' you may change the set of delimiters. It affects all running iterators.
Example: {\tt \small\begin{alltt}
{\bf StringTokenizer}{\rm (p.\,\pageref{classost_1_1_string_tokenizer})} st("mary had a little lamb;its fleece was..", " ;");
{\bf StringTokenizer::iterator}{\rm (p.\,\pageref{classost_1_1_string_tokenizer_1_1iterator})} i;
for (i = st.begin() ; i != st.end() ; ++i) \{
cout << "Token: '" << *i << "'\(\backslash\)t";
cout << " next Delim: '" << i.nextDelimiter() << "'" << endl;
\}
\end{alltt}\normalsize
}
\begin{Desc}
\item[Author:]Henner Zeller $<${\tt H.Zeller@acm.org}$>$ \end{Desc}
\begin{Desc}
\item[License:]LGPL \end{Desc}
\subsection{Constructor \& Destructor Documentation}
\index{ost::StringTokenizer@{ost::String\-Tokenizer}!StringTokenizer@{StringTokenizer}}
\index{StringTokenizer@{StringTokenizer}!ost::StringTokenizer@{ost::String\-Tokenizer}}
\subsubsection{\setlength{\rightskip}{0pt plus 5cm}ost::String\-Tokenizer::String\-Tokenizer (const char $\ast$ {\em str}, const char $\ast$ {\em delim}, bool {\em skip\-All\-Delim} = {\tt false}, bool {\em trim} = {\tt false})}\label{classost_1_1_string_tokenizer_67c853d283920854b98c41fbe4e84d52}
creates a new {\bf String\-Tokenizer}{\rm (p.\,\pageref{classost_1_1_string_tokenizer})} for a string and a given set of delimiters.
\begin{Desc}
\item[Parameters:]
\begin{description}
\item[{\em str}]{\bf String}{\rm (p.\,\pageref{classost_1_1_string})} to be split up. This string will not be modified by this {\bf String\-Tokenizer}{\rm (p.\,\pageref{classost_1_1_string_tokenizer})}, but you may as well not modfiy this string while tokenizing is in process, which may lead to undefined behaviour.\item[{\em delim}]{\bf String}{\rm (p.\,\pageref{classost_1_1_string})} containing the characters which should be regarded as delimiters.\item[{\em skip\-All\-Delim}]OPTIONAL. true, if subsequent delimiters should be skipped at once or false, if empty tokens should be returned for two delimiters with no other text inbetween. The first behaviour may be desirable for whitespace skipping, the second for input with delimited entry e.g. /etc/passwd like files or CSV input. NOTE, that 'true' here resembles the ANSI-C strtok(char $\ast$s,char $\ast$d) behaviour. DEFAULT = false\item[{\em trim}]OPTIONAL. true, if the tokens returned should be trimmed, so that they don't have any whitespaces at the beginning or end. Whitespaces are any of the characters defined in {\bf String\-Tokenizer::SPACE}{\rm (p.\,\pageref{classost_1_1_string_tokenizer_4018e923303101ebd6d01cdbca10ce38})}. If delim itself is {\bf String\-Tokenizer::SPACE}{\rm (p.\,\pageref{classost_1_1_string_tokenizer_4018e923303101ebd6d01cdbca10ce38})}, this will result in a behaviour with skip\-All\-Delim = true. DEFAULT = false \end{description}
\end{Desc}
\index{ost::StringTokenizer@{ost::String\-Tokenizer}!StringTokenizer@{StringTokenizer}}
\index{StringTokenizer@{StringTokenizer}!ost::StringTokenizer@{ost::String\-Tokenizer}}
\subsubsection{\setlength{\rightskip}{0pt plus 5cm}ost::String\-Tokenizer::String\-Tokenizer (const char $\ast$ {\em s})}\label{classost_1_1_string_tokenizer_d92ba14c987eee96b10f9fa177b5a4e0}
create a new {\bf String\-Tokenizer}{\rm (p.\,\pageref{classost_1_1_string_tokenizer})} which splits the input string at whitespaces.
The tokens are stripped from whitespaces. This means, if you change the set of delimiters in either the 'begin(const char $\ast$delim)' method or in 'set\-Delimiters()', you then get whitespace trimmed tokens, delimited by the new set. Behaves like String\-Tokenizer(s, String\-Tokenizer::SPACE,false,true);
\subsection{Member Function Documentation}
\index{ost::StringTokenizer@{ost::String\-Tokenizer}!begin@{begin}}
\index{begin@{begin}!ost::StringTokenizer@{ost::String\-Tokenizer}}
\subsubsection{\setlength{\rightskip}{0pt plus 5cm}{\bf iterator} ost::String\-Tokenizer::begin (const char $\ast$ {\em d})\hspace{0.3cm}{\tt [inline]}}\label{classost_1_1_string_tokenizer_f3dfeb0e48c9d2fc9629076da65acd86}
returns a begin iterator with an alternate set of delimiters.
\index{ost::StringTokenizer@{ost::String\-Tokenizer}!begin@{begin}}
\index{begin@{begin}!ost::StringTokenizer@{ost::String\-Tokenizer}}
\subsubsection{\setlength{\rightskip}{0pt plus 5cm}{\bf iterator} ost::String\-Tokenizer::begin () const\hspace{0.3cm}{\tt [inline]}}\label{classost_1_1_string_tokenizer_ec7ea216033f881b2bac13777525e415}
returns the begin iterator
\index{ost::StringTokenizer@{ost::String\-Tokenizer}!end@{end}}
\index{end@{end}!ost::StringTokenizer@{ost::String\-Tokenizer}}
\subsubsection{\setlength{\rightskip}{0pt plus 5cm}const {\bf iterator}\& ost::String\-Tokenizer::end (void) const\hspace{0.3cm}{\tt [inline]}}\label{classost_1_1_string_tokenizer_74b3471e3e481c0144758385c29cbfbf}
the iterator marking the end.
\index{ost::StringTokenizer@{ost::String\-Tokenizer}!setDelimiters@{setDelimiters}}
\index{setDelimiters@{setDelimiters}!ost::StringTokenizer@{ost::String\-Tokenizer}}
\subsubsection{\setlength{\rightskip}{0pt plus 5cm}void ost::String\-Tokenizer::set\-Delimiters (const char $\ast$ {\em d})\hspace{0.3cm}{\tt [inline]}}\label{classost_1_1_string_tokenizer_192f7b20b494671b6f69225b4df3e299}
changes the set of delimiters used in subsequent iterations.
\subsection{Friends And Related Function Documentation}
\index{ost::StringTokenizer@{ost::String\-Tokenizer}!StringTokenizer::iterator@{StringTokenizer::iterator}}
\index{StringTokenizer::iterator@{StringTokenizer::iterator}!ost::StringTokenizer@{ost::String\-Tokenizer}}
\subsubsection{\setlength{\rightskip}{0pt plus 5cm}friend class {\bf String\-Tokenizer::iterator}\hspace{0.3cm}{\tt [friend]}}\label{classost_1_1_string_tokenizer_46c74d7c3ebac303a827a759155b5879}
\subsection{Member Data Documentation}
\index{ost::StringTokenizer@{ost::String\-Tokenizer}!SPACE@{SPACE}}
\index{SPACE@{SPACE}!ost::StringTokenizer@{ost::String\-Tokenizer}}
\subsubsection{\setlength{\rightskip}{0pt plus 5cm}const char$\ast$ const {\bf ost::String\-Tokenizer::SPACE}\hspace{0.3cm}{\tt [static]}}\label{classost_1_1_string_tokenizer_4018e923303101ebd6d01cdbca10ce38}
a delimiter string containing all usual whitespace delimiters.
These are space, tab, newline, carriage return, formfeed and vertical tab. (see isspace() manpage).
The documentation for this class was generated from the following file:\begin{CompactItemize}
\item
{\bf tokenizer.h}\end{CompactItemize}
