<HTML><HEAD>
<TITLE>Manual Page - nettee(1)</TITLE>
<BODY bgcolor=#F0F0F0 text=#000000 link=#0000ff vlink=#C000C0 alink=#ff0000><A NAME=top></A>
<!-- .TH "nettee" 1 "MAY 2007" "nettee 0.1.8" "nettee Manual" -->
<CENTER>
<H1><HR><I>Manual Reference Pages - </I><NOBR>nettee (1)</NOBR><HR></H1>
</CENTER>
<P>
<!-- .SH NAME -->
<A name=0>
<H3>NAME</H3>
</A>
<BLOCKQUOTE>
<!-- .TP -->
<TABLE cellpadding=3>
<TR valign=top><TD colspan=2>
nettee - a network "tee" program
</TD></TR><TR valign=top><TD width=6%> </TD><TD>
<P>
<!-- .SH SYNOPSIS -->
</TD></TR>
<TR></TR></TABLE></BLOCKQUOTE>
<A name=contents></A><H3>CONTENTS</H3></A>
<BLOCKQUOTE>
<A HREF=#1>Synopsis</A><BR>
<A HREF=#2>Description</A><BR>
<A HREF=#3>Options</A><BR>
<A HREF=#4>Related Programs</A><BR>
<A HREF=#5>Copyrights</A><BR>
<A HREF=#6>License</A><BR>
<A HREF=#7>Author</A><BR>
</BLOCKQUOTE>
<A name=8>
<H3>SYNOPSIS</H3>
</A>
<BLOCKQUOTE>
<!-- .B nettee -->
<B>nettee</B> <!-- .I [options] -->
<I>[options]</I> <P>
<!-- .SH DESCRIPTION -->
</BLOCKQUOTE>
<A name=2>
<H3>DESCRIPTION</H3>
</A>
<BLOCKQUOTE>
<P>
<!-- .B nettee -->
<B>nettee</B> passes a data stream to one or more child nodes
using a daisychain method. On each node
<!-- .B nettee -->
<B>nettee</B> may also direct
the stream to a file or pipe.
<!-- .B nettee -->
<B>nettee</B> allows large amounts of data to be quickly distributed to multiple
nodes on a network at a rate limited only by the network bandwidth.
The distribution chain is typically linear for each network switch
but may branch when nodes utilize multiple switches. For maximum throughput
only one instance of
<!-- .B nettee -->
<B>nettee</B> should utilize each network interface.
<P>
<P>
When
<!-- .B nettee -->
<B>nettee</B> starts it waits for a connection from the upstream node
before attempting to connect to its downstream nodes. Consequently
<!-- .B nettee -->
<B>nettee</B> may be started on the nodes in any order (by a script, rsh, ssh, and so forth.)
Typically only the node that reads the data stream for stdin or a file
will be set to log messages, so that the progress of the transfer may
be monitored. Transmission errors are detected by comparing the total
number of bytes read by each child node with the number of bytes transmitted
to that child.
<P>
<!-- .SS "Error Handling" -->
</BLOCKQUOTE>
<A name=>
<H4> Error Handling</H4>
</A>
<BLOCKQUOTE>
By default severe errors cause the entire chain to abort. By utilizing
the
<!-- .B -conwf -->
<B>-conwf</B> and
<!-- .B -colwf -->
<B>-colwf</B> options
<!-- .B nettee -->
<B>nettee</B> may be instructed to do its best to continue processing in the event
of certain write failures of the data stream. Note that failures which
occur while the distribution chain is forming are still fatal events.
To allow the program to continue with a truncated or alternate chain
if chain formation errors are encountered utilize the
<!-- .B -connf -->
<B>-connf</B> option, and optionally specify alternate targets in each hostlist.
If the node above the failed node is allowed
to emit messages and errors ( for instance:
<!-- .B -v 5 -->
<B>-v 5</B> ) messages similar to these will be sent to the log destination (
<!-- .B -log -->
<B>-log</B> ):
<!-- .PP -->
<P>
<!-- .B Failures detected in child 0 [node34]: NWF -->
<B>Failures detected in child 0 [node34]: NWF</B> <!-- .br -->
<!-- .B Failures detected in child 1 [node35]: NONE -->
<!-- Need break --><BR>
<B>Failures detected in child 1 [node35]: NONE</B> <!-- .br -->
<!-- .B Failures detected in chain: NWF -->
<!-- Need break --><BR>
<B>Failures detected in chain: NWF</B> <!-- .PP -->
<P>
The first type of message describes the failures that were detected
in the named child node, that is, those named in the
<!-- .B -next -->
<B>-next</B> option. The second message describes failures
that were detected anywhere further on in the chain.
The error codes currently defined are:
<!-- .B NONE -->
<B>NONE</B> no errors,
<!-- .B NWF -->
<B>NWF</B> network write failure,
<!-- .B LWF -->
<B>LWF</B> local write failure,
<!-- .B BBC -->
<B>BBC</B> child returned incorrect byte count,
<!-- .B BSTAT -->
<B>BSTAT</B> child returned unknown or bad status, and
<!-- .B NNF -->
<B>NNF</B> could not connect to (one or more) downstream chain nodes.
<!-- .SS "Exit Status" -->
</BLOCKQUOTE>
<A name=>
<H4> Exit Status</H4>
</A>
<BLOCKQUOTE>
<!-- .B nettee -->
<B>nettee</B> will normally emit an
<!-- .B EXIT_SUCCESS -->
<B>EXIT_SUCCESS</B> status. (0 on Unix.) This is true even if the errors were detected and
handled in the node itself or in a child node.
<!-- .B nettee -->
<B>nettee</B> will emit an
<!-- .B EXIT_FAILURE -->
<B>EXIT_FAILURE</B> status if it was forced to close by an unhandled event such as a timeout,
write failure, or unexpected socket closure.
<P>
<!-- .SH OPTIONS -->
</BLOCKQUOTE>
<A name=3>
<H3>OPTIONS</H3>
</A>
<BLOCKQUOTE>
<P>
<!-- .TP -->
<TABLE cellpadding=3>
<TR valign=top><TD width=6% nowrap>
<!-- .B -h -->
<B>-h</B> </TD><TD valign=bottom>
Print help information.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD colspan=2>
<!-- .B -hexamples -->
<B>-hexamples</B> </TD></TR><TR valign=top><TD width=6%> </TD><TD>
Print examples.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD colspan=2>
<!-- .B -herrors -->
<B>-herrors</B> </TD></TR><TR valign=top><TD width=6%> </TD><TD>
Print error status codes.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .B -i -->
<B>-i</B> </TD><TD valign=bottom>
Print version, license, and copyright information.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .BI -in " <SRC>" -->
<B>-in</B><I> <SRC></I> </TD><TD valign=bottom>
Reads data from
<!-- .I <SRC> -->
<I><SRC></I> which may have one of three values:
<!-- .B "nettee" -->
<B>nettee</B> reads from the upstream node;
<!-- .B "-" -->
<B>-</B> reads from stdin;
<!-- .B "socket" -->
<B>socket</B> read the output of a command from a socket;
<!-- .B "filename" -->
<B>filename</B> reads from a file. If no
<!-- .B -in -->
<B>-in</B> option is present the programs reads data from the upstream node.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .BI -out " <DST>" -->
<B>-out</B><I> <DST></I> </TD><TD valign=bottom>
Writes data locally to
<!-- .I <DST> -->
<I><DST></I> which may have one of three values:
<!-- .B "none" -->
<B>none</B> writes nothing locally;
<!-- .B "-" -->
<B>-</B> writes to stdout;
<!-- .B "socket" -->
<B>socket</B> write the datastream to a command through a socket;
<!-- .B "filename" -->
<B>filename</B> writes to a file. If no
<!-- .B -out -->
<B>-out</B> option is present the program writes data to stdout.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD colspan=2>
<!-- .BI -next " <HOSTLISTS>" -->
<B>-next</B><I> <HOSTLISTS></I> </TD></TR><TR valign=top><TD width=6%> </TD><TD>
Writes data to downstream destination[s]
<!-- .B "hostlist1(,hostlist2(,hostlist3(...)))" -->
<B>hostlist1(,hostlist2(,hostlist3(...)))</B> where the hostlist entries are separated by commas or spaces.
A hostlist consists of either a single hostname, or a comma separated
list of hostnames enclosed in square brackets. Example:
<!-- .B node1,[node2,node3],[node4,node5,node6],node7. -->
<B>node1,[node2,node3],[node4,node5,node6],node7.</B> The bracketed form allows for
automatic failover if unreachable nodes are encountered and if
<!-- .B -connf -->
<B>-connf</B> is specified. The first hostname in the
list is tried, then the next, and so on.
There may be 1-8 hostlists. The number of hostlists controls the
topology of the distribution chain. Use a linear distribution chain (a single hostlist)
when all nodes share a single network switch. Use a forked distribution chain
(multiple hostlist) when nodes are connected to two or more network switches.
The End of Chain condition (no downstream
write) is indicated by a
<!-- .I <HOSTS> -->
<I><HOSTS></I> value of
<!-- .B "." -->
<B>.</B> ,
<!-- .B """"" -->
<B> </B> , or
<!-- .B "_EOC_". -->
<B>_EOC_ .</B> An End of Chain condition is also indicated by the absence of an
<!-- .B -next -->
<B>-next</B> option. If End of Chain is indicated there may not be any other hostslists
specified.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .BI -cmd " <COMMAND>" -->
<B>-cmd</B><I> <COMMAND></I> </TD><TD valign=bottom>
Specifies the command to use in conjunction with an
<!-- .B -in socket -->
<B>-in socket</B> or
<!-- .B -out socket -->
<B>-out socket</B> option. Since only a single
<!-- .B <COMMAND> -->
<B><COMMAND></B> may be specified
<!-- .B socket -->
<B>socket</B> may not be applied to both
<!-- .B -in -->
<B>-in</B> and
<!-- .B -out -->
<B>-out</B> at the same time. When
<!-- .B -cmd -->
<B>-cmd</B> is used with
<!-- .B -in socket -->
<B>-in socket</B> a child process running
<!-- .B <COMMAND> -->
<B><COMMAND></B> reads data from a disk or other device and writes
the resulting data stream to stdout. When
<!-- .B -cmd -->
<B>-cmd</B> is used with
<!-- .B -out socket -->
<B>-out socket</B> a child process running
<!-- .B <COMMAND> -->
<B><COMMAND></B> reads the datastream from stdin and writes the
processed data to a disk or other device.
Typically the
<!-- .B <COMMAND> -->
<B><COMMAND></B> string invokes
<!-- .B tar -->
<B>tar</B> or some other archiving program. In some instances using sockets
and
<!-- .B -cmd -->
<B>-cmd</B> will be faster than using the same command in a pipe due to the larger
buffer size used for the socket. Run
<!-- .B nettee -hexamples -->
<B>nettee -hexamples</B> to see a usage example.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .BI -stm " <EOS>" -->
<B>-stm</B><I> <EOS></I> </TD><TD valign=bottom>
stream text through a
<!-- .B nettee -->
<B>nettee</B> chain until the string
<!-- .B <EOS> -->
<B><EOS></B> is encountered, then exit. This allows short text
messages to traverse the chain without waiting for a buffer to fill.
Since the text message can very rapidly traverse the
<!-- .B nettee -->
<B>nettee</B> chain it can be piped into
<!-- .B execinput -->
<B>execinput</B> (or any other program that will execute its stdin as commands)
to produce essentially simultaneous execution on all target nodes. The
<!-- .B <EOS> -->
<B><EOS></B> string is not passed through the data chain and its length is ignored.
When used to start further
<!-- .B nettee -->
<B>nettee</B> processes on the target nodes
<!-- .B <PORT> -->
<B><PORT></B> values must be chosen to avoid interference.
While this mode may be convenient for setting up Beowulf nodes it is
exceedingly dangerous for general use since any command introduced into
the command stream will execute on all chain nodes
as if submitted by the owner of the nettee process on that node.
Run
<!-- .B nettee -hexamples -->
<B>nettee -hexamples</B> to see a usage example.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD colspan=2>
<!-- .BI -name " <STRING>" -->
<B>-name</B><I> <STRING></I> </TD></TR><TR valign=top><TD width=6%> </TD><TD>
Specify the node name used in messages (<=127 characters). If not supplied
the values of the environmental variables
<!-- .B "MYHOSTNAME" -->
<B>MYHOSTNAME</B> and
<!-- .B "HOSTNAME" -->
<B>HOSTNAME</B> are first checked, and if those are not defined, the result of a
<!-- .B gethostname() -->
<B>gethostname()</B> call is used.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .BI -log " <LDST>" -->
<B>-log</B><I> <LDST></I> </TD><TD valign=bottom>
Errors and messages are written to
<!-- .I <LDST> -->
<I><LDST></I> which may have one of two values:
<!-- .B "-" -->
<B>-</B> writes to stderr or
<!-- .B "filename" -->
<B>filename</B> writes to a file. If no
<!-- .B -log -->
<B>-log</B> option is present the program writes messages to stderr.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD colspan=2>
<!-- .BI -p,-port " <PORT>" -->
<B>-p,-port</B><I> <PORT></I> </TD></TR><TR valign=top><TD width=6%> </TD><TD>
First of two consecutive ports use for communication.
If no
<!-- .B -port -->
<B>-port</B> option is present the program uses the default value of 9997.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .BI -v " <VERBOSE>" -->
<B>-v</B><I> <VERBOSE></I> </TD><TD valign=bottom>
<!-- .B "<VERBOSE>" -->
<B><VERBOSE></B> is a bit mask which controls the types of warning and error
messages which are sent to the
<!-- .B -log -->
<B>-log</B> destination. Bit values indicate:
<!-- .B 1 -->
<B>1</B> show error messages;
<!-- .B 2 -->
<B>2</B> show command line settings;
<!-- .B 4 -->
<B>4</B> show messages;
<!-- .B 8 -->
<B>8</B> show periodic status messages during transfer;
<!-- .B 16 -->
<B>16</B> prepend nodename to all messages.
Use a
<!-- .B "<VERBOSE>" -->
<B><VERBOSE></B> value of 0 to eliminate all messages. If no
<!-- .B -v -->
<B>-v</B> is present the program uses a default
<!-- .B "<VERBOSE>" -->
<B><VERBOSE></B> value of 1.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .BI -q -->
<B>-q</B> </TD><TD valign=bottom>
Suppresss "ignored signal" messages.
<P>
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .BI -t " <WAIT>" -->
<B>-t</B><I> <WAIT></I> </TD><TD valign=bottom>
Wait up to
<!-- .B "<WAIT>" -->
<B><WAIT></B> seconds for a connection from upstream in the chain to form or data to be received.
If neither of these events occur exit with an error.
A value of
<!-- .B 0 -->
<B>0</B> waits forever and will only exit on an end of data condition.
If no
<!-- .B -t -->
<B>-t</B> is present the program uses a default
<!-- .B "<WAIT>" -->
<B><WAIT></B> value of 0. The
<!-- .BI -iconnf "<WAIT>" -->
<B>-iconnf</B><I><WAIT></I> and
<!-- .B -w -->
<B>-w</B> options control timeouts for downstream connections.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .BI -w -->
<B>-w</B> </TD><TD valign=bottom>
Wait for the next node to boot or attach to the network. If not
specified and the next node is not reachable
<!-- .B nettee -->
<B>nettee</B> will exit with an error no matter what the
<!-- .BI -t " <WAIT>" -->
<B>-t</B><I> <WAIT></I> and
<!-- .BI -iconnf " <WAIT>" -->
<B>-iconnf</B><I> <WAIT></I> timeout values are.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .B -colwf -->
<B>-colwf</B> </TD><TD valign=bottom>
Continue on Local Write Failure. Normally the failure of
a write of the data stream to the local output will be fatal
and the entire distribution chain will collapse immediately.
(Typically this happens when data is written to disk and a
partition fills or there is an ownership problem. A complete
disk failure may initially present this way but often goes on
to crash the node, resulting also in a network write failure.)
When
<!-- .B -colwf -->
<B>-colwf</B> is set and a local write failure occurs on a node that node will
continue to relay data down the chain. The node that failed will
not have correctly processed the data stream locally but all other
nodes will be unaffected by this failure. The top node will emit
an error message when this occurs so that a subsequent analysis
with other tools may locate the node(s) which failed. This option
may only be employed on a node that reads data from an upstream node.
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD width=6% nowrap>
<!-- .B -conwf -->
<B>-conwf</B> </TD><TD valign=bottom>
Continue on Network Write Failure. Normally the failure of
a write of the data stream to the next node will be fatal
and the entire distribution chain will collapse immediately.
(Typically
this happens when a node crashes while nettee is running.)
When
<!-- .B -conwf -->
<B>-conwf</B> is set and a network write failure occurs on a node (indicating
that the next node has failed) the node will
continue to process the data stream locally but will make no further
attempts to transfer data to the next node in the chain.
This allows the data transfer to complete on a chain down to the
node above a failed node.
The top node will emit an error message when this occurs so
that a subsequent analysis with other tools may locate the
node(s) which failed. This option
may only be employed on a node that reads data from an upstream node
<P>
<!-- .TP -->
</TD></TR>
<TR valign=top><TD colspan=2>
<!-- .BI -connf " <WAIT>" -->
<B>-connf</B><I> <WAIT></I> </TD></TR><TR valign=top><TD width=6%> </TD><TD>
Continue on Next Node Failure. Give each node in a hostlist
<!-- .B "<WAIT>" -->
<B><WAIT></B> seconds to join the chain. After that each successive host in the hostlist
is given
<!-- .B "<WAIT>" -->
<B><WAIT></B> seconds to join, and if none succeed, no data will be sent to any of those hosts.
If
<!-- .B -connf -->
<B>-connf</B> is not specified or the wait time is set to zero seconds, the program will wait forever
for a connection to the first node in each hostlist.
<P>
<P>
<P>
<!-- .SH RELATED PROGRAMS -->
</TD></TR>
<TR></TR></TABLE></BLOCKQUOTE>
<A name=4>
<H3>RELATED PROGRAMS</H3>
</A>
<BLOCKQUOTE>
<P>
<!-- .B netcat(1). -->
<B>netcat(1).</B> <P>
<!-- .B nettee -->
<B>nettee</B> is derived from Felix Rauch’s
<!-- .B dolly -->
<B>dolly</B> which is available here:
<!-- .B http://www.cs.inf.ethz.ch/CoPs/patagonia/#dolly -->
<B><A HREF="http://www.cs.inf.ethz.ch/CoPs/patagonia/#dolly">http://www.cs.inf.ethz.ch/CoPs/patagonia/#dolly</A></B> <P>
The
<!-- .B nettee -->
<B>nettee</B> home page is:
<!-- .B http://saf.bio.caltech.edu/nettee.html -->
<B><A HREF="http://saf.bio.caltech.edu/nettee.html">http://saf.bio.caltech.edu/nettee.html</A></B> <P>
<P>
<!-- .SH COPYRIGHTS -->
</BLOCKQUOTE>
<A name=5>
<H3>COPYRIGHTS</H3>
</A>
<BLOCKQUOTE>
<P>
<!-- .nf -->
<PRE>
Copyright: 2007 David Mathog and Caltech.
Copyright: Felix Rauch and ETH Zurich
<!-- .fi -->
</PRE>
<P>
<!-- .SH LICENSE -->
</BLOCKQUOTE>
<A name=6>
<H3>LICENSE</H3>
</A>
<BLOCKQUOTE>
Freely distributed under the second GNU General Public License (GPL 2).
<P>
<!-- .SH AUTHOR -->
</BLOCKQUOTE>
<A name=7>
<H3>AUTHOR</H3>
</A>
<BLOCKQUOTE>
<P>
<!-- .nf -->
<PRE>
David Mathog
Biology Division, Caltech
<!-- .fi -->
</PRE>
<P>
<P>
</BLOCKQUOTE>
<P><HR>
<TABLE width=100%><TR> <TD width=33%><I>nettee 0.1.8</I></TD> <TD width=33% align=center>nettee (1)</TD> <TD align=right width=33%><I>MAY 2007</I></TD> </TR></TABLE><FONT SIZE=-1>Generated by <A HREF="http://www.squarebox.co.uk/download/manServer.shtml">manServer 1.07</A> from /usr/common/man/man1/nettee.1 using man macros.</FONT>
</BODY></HTML>
