2010-09-10 S. Thiell <stephane.thiell@cea.fr>
* Version 1.3.2 released.
* clustershell.spec.in: Last fixes (Fedora approved).
* scripts/clush.py: Removed duplicate add_option_group() (Output behaviour
option group was displayed twice in clush -h).
2010-09-03 S. Thiell <stephane.thiell@cea.fr>
* Version 1.3.1 released.
* Worker/Ssh.py: Fix issue with clush -l USER by separating underlying ssh
"-l USER" in two shell arguments (trac ticket #113).
2010-08-31 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Live per-line gathering (-bL mode) improvements.
* Task.py: Fixed Task.timer() when called from another thread - it used to
return None (trac ticket #112).
2010-08-29 S. Thiell <stephane.thiell@cea.fr>
* Task.py: Add docstring for timer's autoclose feature (trac ticket #109).
* Worker/Worker.py: Attribute 'last_errmsg' not properly initialized (trac
ticket #107).
* setup.py: Switch to setuptools.
* clustershell.spec.in: Fix issue on el5 with if condition when defining
python_sitelib.
2010-08-26 S. Thiell <stephane.thiell@cea.fr>
* Packaging automation engineering and improved specfile.
* License files converted to UTF-8.
2010-07-27 S. Thiell <stephane.thiell@cea.fr>
* Version 1.3 released.
2010-07-21 S. Thiell <stephane.thiell@cea.fr>
* Version 1.3 RC 2 released.
* NodeSet.py: Like in some previous version, support None as argument for
most methods (trac ticket #106).
2010-07-16 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Fix uncaught exceptions introduced in 1.3 RC 1 (trac
ticket #105).
2010-07-12 S. Thiell <stephane.thiell@cea.fr>
* Version 1.3 RC 1 released.
* Task.py: Raise proper KeyError exception in Task.key_retcode(key) when
key is not found in any finished workers (trac ticket #102).
2010-07-06 S. Thiell <stephane.thiell@cea.fr>
* Task.py: Added documentation for reserved set_default() and set_info()
keys (trac ticket #101).
* scripts/clubak.py: Merge latest code display changes made on clush to
clubak, including "--color={never,always,auto}" (trac ticket #89). Updated
documentation accordingly.
2010-06-29 H. Doreau <henri.doreau@gmail.com>
* Worker/Pdsh.py: removed obsolete _read() and _readerr() methods that
overrode EngineClient methods without raising an EOFException when
read() reads nothing (trac ticket #97).
2010-06-28 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Centralized handling of exceptions raised from Main
and separate Task thread because some exceptions handled only in Main
thread were not caught (fix btw trac ticket #93).
2010-06-17 S. Thiell <stephane.thiell@cea.fr>
* Version 1.3 beta 6 released.
2010-06-16 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Check for trailing args when using -c/--copy (trac
ticket #88).
* NodeSet.py, NodeUtils.py: Add a way to retrieve all nodes when "all"
external call is missing but "map" and "list" calls are specified (trac
ticket #90).
* Task.py: Add handling of stderr during task.copy().
* Worker/Ssh.py: Add handling of stderr (when needed) during scp.
* scripts/clush.py: Fix display issue with clush --copy when some nodes
are not reachable.
* Version 1.3 beta 5 released.
2010-06-15 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Add --color={never,always,auto} command line option
and color: {never,always,auto} config option (trac ticket #68), defaulting
to `never'. Also did some code refactoring/lightening (created a Display
class). Updated clush and clush.conf man pages.
2010-06-09 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Automatically increase open files soft limit (trac
ticket #61). Handle "Too many open files" exception.
* Task.py: Add excepthook and default_excepthook methods to handle
uncaught exception in a Task thread. Make it compliant with sys.excepthook
also.
2010-06-08 S. Thiell <stephane.thiell@cea.fr>
* Version 1.3 beta 4 released.
* doc/extras/vim/syntax/groupsconf.vim: Improved vim syntax file for
groups.conf (trac ticket #85): now $GROUP and $NODE are keywords.
* scripts/clush.py: Do not wait the end of all commands when using -bL
switches when possible (trac ticket #69).
* MsgTree.py: Added remove(match) method to remove entry from the tree.
* Task.py: Added flush_buffers() and flush_errors() methods.
* Worker/Worker.py: Added flush_buffers() and flush_errors() methods.
2010-05-26 S. Thiell <stephane.thiell@cea.fr>
* Version 1.3 beta 3 released.
* scripts/clush.py: Fixed issue (-g/-X group not working as expected)
found in release 1.3 beta2.
2010-05-25 S. Thiell <stephane.thiell@cea.fr>
* Version 1.3 beta 2 released.
* scripts/clush.py: Added -G, --groupbase to strip group source prefix
when using -r.
* scripts/clubak.py: Added -G, --groupbase to strip group source prefix
when using -r.
* scripts/nodeset.py: Changed -N, --noprefix to -G, --groupbase to avoid
conflict with clush -N.
* scripts/clush.py: Fixed missing support for group source (-s
GROUPSOURCE) when using -a or -g GROUP.
* scripts/nodeset.py: Added --all, -a support (also work is -s
GROUPSOURCE). Almost-silently removed -a for --autostep, I hope nobody's
using it. :)
* Updated man pages of clush, clubak and nodeset to match latest options
changes (trac #58).
* scripts/clubak.py: Added regroup support to clubak (trac ticket #78).
Added -S to specify user settable separator string (trac ticket #62).
2010-05-24 S. Thiell <stephane.thiell@cea.fr>
* tests/NodeSetGroupTest.py: Some cleanup in tests (use setUp, tearDown)
and create temporary groups test files.
* tests/NodeSetRegroupTest.py: Removed (tests moved to
NodeSetGroupTest.py).
* scripts/nodeset.py: Add -N option to avoid display of group source
prefix (trac ticket #79).
* NodeSet.py: Add noprefix boolean option to regroup() to avoid building
nodegroups with group source prefixes. Added test.
* scripts/clush.py: Fix unhandled GroupResolverSourceError exception (part
of trac ticket #74).
* scripts/nodeset.py: Renamed -n NAMESPACE option to -s GROUPSOURCE (or
--groupsource=GROUPSOURCE). Fixed trac ticket #76 so that -f, -e or -c
take -s into account. Improved error handling (trac ticket #74). Added
--groupsources command to list configured group sources (trac #77).
2010-05-20 S. Thiell <stephane.thiell@cea.fr>
* tests/NodeSetRegroupTest.py: added tests for nodeset.regroup().
2010-05-19 S. Thiell <stephane.thiell@cea.fr>
* doc/extras/vim/ftdetect/clustershell.vim: renamed clush.vim to
clustershell.vim.
* doc/extras/vim/syntax/clushconf.vim: renamed clush.vim to clushconf.vim
and cleaned up old external groups keywords.
* doc/extras/vim/syntax/groupsconf.vim: added vim syntax file for
groups.conf (trac ticket #73).
2010-04-08 S. Thiell <stephane.thiell@cea.fr>
* NodeSet.py: Added __getstate__() and __setstate__() methods to support
pickling of NodeSet objects.
* scripts/clush.py: Add option flag -n NAMESPACE to specify groups.conf(5)
namespace to use for regrouping displayed nodeset.
* scripts/clush.py: Add -r (--regroup) option to display default groups
in nodeset when possible.
2010-04-07 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Modified script to support new external "all nodes"
upcall and node groups.
* scripts/nodeset.py: Added command flags -l (list groups), -r (used to
regroup nodes in groups), and also added option flag -n to specify desired
namespace.
* NodeSet.py: Added node group support with the help of the new NodeUtils
module (trac ticket #41). Improved parser to support basic node/nodegroups
arithmetics (trac ticket #44).
* NodeUtils.py: New module that provides binding support to external node
group sources (trac ticket #43).
2010-03-05 S. Thiell <stephane.thiell@cea.fr>
* Worker/*.py: Do not forget to keep last line and generate an ev_read
event when it does not contain EOL (trac ticket #66). Added tests.
2010-02-26 S. Thiell <stephane.thiell@cea.fr>
* Version 1.2 RC 1 released.
2010-02-25 S. Thiell <stephane.thiell@cea.fr>
* Important code cleaning (use absolute imports, remove some unused
imports, remove duplicate code, etc. thanks to pylint).
2010-02-22 S. Thiell <stephane.thiell@cea.fr>
* scripts/nodeset.py: Change command syntax: operations are now specified
inline between nodesets (trac ticket #45). Update doc and tests.
* scripts/clubak.py: Fix TypeError exception raised on unexpected input
and accept 'node:message' line pattern (trac ticket #59).
* scripts/clush.py: Add -B flag (trac ticket #60) to gather with stderr.
* NodeSet.py: NodeSet constructor now raises a NodeSetParseError
exception when unsupported type is used as input (trac ticket #53).
2010-02-21 S. Thiell <stephane.thiell@cea.fr>
* Task.py: Fix a deadlock when a task is resumed two times from another
thread (raise AlreadyRunningError instead). Added test.
* Worker/Worker.py: Improve usage error handling for some methods (trac
ticket #28), raising WorkerError when needed. Add library misusage tests.
2010-02-18 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Disable MsgTree buffering when not performing
any gathering of results (when -b is not used).
* Task.py: Allow disabling of MsgTree buffering (trac ticket #3) via
'stdout_msgtree" and 'stderr_msgtree' Task default keywords, useful if we
don't want MsgTree internal buffering for fully event-based scripts (eg.
clush without -b). When disabled, any Task method accessing MsgTree data
like iter_buffers() will raise a new exception (TaskMsgTreeError).
2010-02-17 S. Thiell <stephane.thiell@cea.fr>
* Version 1.2 beta 5 released.
2010-02-16 S. Thiell <stephane.thiell@cea.fr>
* NodeSet.py: Fix mixed-type comparisons, where, like standard set(),
are allowed, instead of raising TypeError.
2010-02-15 S. Thiell <stephane.thiell@cea.fr>
* Version 1.2 beta 4 released.
* MsgTree.py: Added MsgTreeElem.splitlines() method as alias of lines().
2010-02-14 S. Thiell <stephane.thiell@cea.fr>
* Updated doc/man pages for latest clush changes and added clubak tool.
* Worker/Ssh.py: Fix Ssh worker issue where sometimes stderr buffer could
not be read completely (trac ticket #50).
2010-02-13 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Comply with clubak by adding -L option that allow
switching to alternative line mode display (when using -b). Also, sort
buffers by nodes or nodeset length like clubak (fix trac ticket #54).
2010-02-11 S. Thiell <stephane.thiell@cea.fr>
* Version 1.2 beta 3 released.
* scripts/clush.py: For clush --copy, when --dest is not specified, set
the destination path to the source dirname path and not the source full
path.
* scripts/clush.py: Added option --nostdin to prevent reading from
standard input (fix trac ticket #49).
* Engine/Factory.py: Disable Engine.EPoll automatic selection as an issue
has been found with clush when stdin is a plain file (<file redirections).
Engine.Poll is now the default engine for all platforms. Created trac
ticket #56 to follow this issue.
2010-02-10 S. Thiell <stephane.thiell@cea.fr>
* Worker/Worker.py: Added missing WorkerSimple.last_error() method. Fixed
worker bad argument error exception.
* Worker/Ssh.py: Added command, source and dest public instance variable.
* Worker/Pdsh.py: Added command, source and dest public instance variable.
* scripts/clush.py: Due to set_info() behaviour modifications in
multi-thread mode, change some set_info() for set_default() to modify
task specific dictionary synchronously. Also remove splitlines() where
MsgTreeElem are returned instead of whole buffer after latest MsgTree
improvements.
* scripts/clubak.py: Added clubak utility (trac ticket #47). It provides
dshbak backward-compatibility, but always try to sort buffers by nodes or
nodeset. It also provides additional -L option to switch to alternative
line mode display.
2010-02-09 S. Thiell <stephane.thiell@cea.fr>
* Worker.py: Updated Task/MsgTree dependencies. Added iter_node_errors()
method. Added match_keys optional parameter to iter_node_buffers() and
iter_node_errors(). Added WorkerSimple.error() method (read stderr).
Added tests.
* Task.py: Updated MsgTree dependencies. Factorized most tree data's
access methods.
* MsgTree.py: Merged Msg and _MsgTreeElem in one class MsgTreeElem. All
message objects returned are now instance of MsgTreeElem. Some algorithms
improvements. Renamed main MsgTree access methods: messages(), items() and
walk(). Added more docstring.
* NodeSet.py: Modified NodeSet.__iter__() and __str__() so that nodes are
now always sorted by name/pattern (eg. acluster2, bcluster1).
2010-02-07 S. Thiell <stephane.thiell@cea.fr>
* MsgTree.py: Rewrite of MsgTree module with a better API (part of trac
ticket #47). Adapted library classes. Added specific tests.
2010-02-02 S. Thiell <stephane.thiell@cea.fr>
* Task.py: Add Task.key_error() and its alias node_error() methods for
easy retrieving of error buffers for a specified key/node.
* scripts/clush.py: Fix stdout/stderr separation issue (introduced
in 1.2b2) thanks to the new Task.set_default() method.
* Task.py: As set_info() is now dispatched through the task special port,
and applied only on task.resume() when called from another thread, add two
new methods default() and set_default() to synchronously manage another
task specific dictionary, useful for default configuration parameters.
2010-02-01 S. Thiell <stephane.thiell@cea.fr>
* Version 1.2 beta 2 released.
2010-02-01 A. Degremont <aurelien.degremont@cea.fr>
* NodeSet.py: Added __getslice__() and split() method to RangeSet.
Added split() to NodeSet (trac ticket #18).
2010-02-01 S. Thiell <stephane.thiell@cea.fr>
* NodeSet.py: Added equality comparisons for RangeSet and NodeSet. Fixed
a bug in NodeSet.issuperset().
* mkrpm.sh: Improve RPM build process and allow SRPM package to be
easily rebuilt (trac ticket #51).
2010-01-31 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Fix broken pipe issue (trac ticket #34).
* scripts/clush.py: Fix unhandled NodeSet parse error (trac ticket #36).
* scripts/clush.py: Display uncompleted nodes on keyboard interrupt.
2010-01-29 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Return some error code when -S -u TIMEOUT is used
and some command timeout occurred (trac ticket #48).
* scripts/clush.py: Display output messages on KeyboardInterrupt (trac
ticket #22).
* tests/TaskThreadJoinTest.py: Added test cases for task.join().
* tests/TaskThreadSuspendTest.py: Added test cases for
task.suspend().
* tests/TaskPortTest.py: Added test cases for task.port().
* Task.py: Improved features in multithreaded environments thanks
to new port feature: abort(), suspend(), resume(), schedule(), etc.
are now thread-safe (trac ticket #21).
* Worker/EngineClient.py: Added port feature, a way to communicate between
different tasks.
2009-12-09 A. Degremont <aurelien.degremont@cea.fr>
* scripts/clush.py: Add -X flag to exclude node groups.
Node flags -w/-x/-g/-X can now be specified multiple times.
2009-12-17 S. Thiell <stephane.thiell@cea.fr>
* Engine/Factory.py: Add engine automatic selection mechanism (trac
ticket #10).
* Task.py: Add task_terminate() function for convenience.
2009-12-15 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Fix clush -q/--quiet issue again!
2009-12-09 A. Degremont <aurelien.degremont@cea.fr>
* scripts/nodeset.py: Protect --separator from code injection and
handle gracefully incorrect separtor.
2009-12-09 S. Thiell <stephane.thiell@cea.fr>
* Version 1.2 beta 1 released.
* scripts/clush.py and library: Add -p option when using --copy to
preserve file modification times and modes.
* scripts/clush.py: Fix clush -q/--quiet issue.
* scripts/nodeset.py: Add separator option to nodeset
--expand with -S <string> (trac ticket #39).
* Worker/Pdsh.py: Added copy support for directory (automatic
detection). Added non-reg tests.
2009-12-08 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Added source presence check on copy.
2009-12-07 S. Thiell <stephane.thiell@cea.fr>
* Worker/Ssh.py: Added copy support for directory (automatic
detection).
* Worker/Ssh.py: Fix Scp Popen4->subprocess.popen issue (simple
quote escape not needed).
2009-11-10 S. Thiell <stephane.thiell@cea.fr>
* Version 1.2 beta 0 released. Updated doc and man pages for 1.2.
2009-11-09 S. Thiell <stephane.thiell@cea.fr>
* Engine/EPoll.py: Add stdout/stderr support (still experimental).
* Worker/Pdsh.py: Fix stdout/stderr support.
* Backport recent 1.1-branch improvements: tests code coverage, also
resulting in some fixes (see 1.1 2009-10-28).
2009-11-09 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: Added stdout/stderr support in clush script.
2009-11-04 S. Thiell <stephane.thiell@cea.fr>
* Added optional separate stdout/stderr handling (with 1.1 Task API
compat). Added some tests for that.
* Create a MsgTree class in MsgTree.py and remove this code from
Task.py.
* First changes to use setUp() in test case objects.
2009-08-02 S. Thiell <stephane.thiell@cea.fr>
* clush.py: (1) remove /step in displayed nodeset when using -b (to
allow copy/paste to other tools like ipmipower that doesn't support
N-M/STEP ranges), (2) when command timeout is specified (-u), show
nodes (on stderr) that didn't have time to fully complete, (3) flush
stdio buffers before exiting. [merged from branch 1.1]
2009-07-29 S. Thiell <stephane.thiell@cea.fr>
* tests/NodeSetScriptTest.py: added unit test for scripts/nodeset.py
* NodeSet.py: fixed a problem with intersection_update() when used
with two simple nodes (no rangeset).
* scripts/nodeset.py: merge -i and -X options issue fix from 1.1
branch (#29)
2009-07-28 S. Thiell <stephane.thiell@cea.fr>
* scripts/clush.py: remove DeprecationWarning ignore filter (the
library is now natively Python 2.6/Fedora 11 ready)
* Change all sets to use built in set type available since Python 2.4
(the sets module is deprecated).
* Engine/EPoll.py: added epoll based Engine (Python 2.6+ needed)
* Engine/Poll.py: added _register_specific() and
_unregister_specific() methods to match modified Engine base class.
* Engine/Engine.py: added calls to derived class's
_register_specific() and _unregister_specific() instead of only
_modify_specific()
2009-07-23 S. Thiell <stephane.thiell@cea.fr>
* Replaced popen2.Popen4 (deprecated) by subprocess.Popen (Python 2.4+),
renaming Worker.Popen2 to Worker.Popen.
* clush.py: (backport for 1.1 branch) fix another command_timeout (-u)
issue, now the command_timeout value is passed as the timeout value at
worker level.
* Version 1.1 branched.
2009-07-22 S. Thiell <stephane.thiell@cea.fr>
* Version 1.1 RC 2 released.
* clush.py: change -u timeout behavior, if set it's now the timeout value
passed to task.shell() (and not connect_timeout + command_timeout).
* clush.py: add -o options to pass custom ssh options (#24).
* Worker/Ssh.py: simple quote escape fix (trac ticket #25).
* Worker/Popen2.py: simple quote escape fix (trac ticket #25)
* clush.py: fix options issue when using -f, -u or -t.
2009-07-13 S. Thiell <stephane.thiell@cea.fr>
* Version 1.1 RC 1 released.
* Changed license to CeCILL-C (http://www.cecill.info).
* clush.py (ttyloop): (feature) added '=' special command in
interactive mode to toggle output format mode (standard/gathered).
* Engine/Engine.py (register): (bug) register writer fd to even when
set_writer_eof() has previously been called.
* Worker/EngineClient.py (_handle_write): (bug) don't close writer
when some data remains in write buffer, even if self._weof is True.
2009-07-10 S. Thiell <stephane.thiell@cea.fr>
* clush.py (ttyloop): added a workaround to replace raw_input() which
is not interruptible in Python 2.3 (issue #706406).
2009-07-09 S. Thiell <stephane.thiell@cea.fr>
* NodeSet.py (__contains__): fixed issue that could appear when
padding was used, eg. "node113" in "node[030,113] didn't work.
2009-07-08 S. Thiell <stephane.thiell@cea.fr>
* Version 1.1 beta 6 released.
* clush.py: major improvements (added write support, better
interactive mode with readline, launch task in separate thread to let
the main thread blocking tty input, added Enter key press support
during run, added node groups support (-a and -g) using external
commands defined in clush.conf, added --copy toggle to clush to copy
files to the cluster nodes, added -q option, added progress indicator
when clush is called with gather option -b)
* Added man pages for clush and nodeset commands.
* doc/extras/vim (clush.vim): added vim syntax files for clush.conf
* Engine.py: (feature) added write support to workers
* Worker: (api) created a base class WorkerSimple
2009-04-17 S. Thiell <stephane.thiell@cea.fr>
* Version 1.1 beta 5 released (LUG'09 live update).
* Worker/Worker.py: (bug) update last_node so that user can call
worker.last_node() in an ev_timeout handler callback.
2009-04-17 A. Degremont <aurelien.degremont@cea.fr>
* clush.py: (feature) make use of optparse.OptionParser
2009-04-15 S. Thiell <stephane.thiell@cea.fr>
* Version 1.1 beta 4 released.
2009-04-14 S. Thiell <stephane.thiell@cea.fr>
* Engine/Engine.py (EngineBaseTimer): (bug) fixed issue in timers when
invalidated two times.
2009-04-06 S. Thiell <stephane.thiell@cea.fr>
* Version 1.1 beta 3 released.
* Engine/Engine.py (_EngineTimerQ): (bug) fixed issue in timer
invalidation.
2009-04-03 S. Thiell <stephane.thiell@cea.fr>
* Engine/Engine.py (EngineTimer): (api) added is_valid() method to
check if a timer is still valid.
* Task.py: (api) added optional `match_keys' parameter in Task and
Worker iter_buffers() and iter_retcodes() methods.
2009-03-26 S. Thiell <stephane.thiell@cea.fr>
* Version 1.1 beta 2 released.
2009-03-23 S. Thiell <stephane.thiell@cea.fr>
* Worker/Worker.py: (api) added Worker.did_timeout() method to check
if a worker has timed out (useful for Popen2 workers, other use
DistantWorker.num_timeout()).
2009-02-21 S. Thiell <stephane.thiell@cea.fr>
* Version 1.1 beta 1 released.
2009-02-20 S. Thiell <stephane.thiell@cea.fr>
* NodeSet.py (NodeSet): (api) added clear() method.
(RangeSet): likewise.
* NodeSet.py (NodeSet): added workaround to allow NodeSet to be
properly pickled (+inf floating number pickle bug with Python 2.4)
* NodeSet.py (RangeSet): (bug) don't keep a reference on internal
RangeSet when creating a NodeSet from another one.
2009-02-16 S. Thiell <stephane.thiell@cea.fr>
* Version 1.1 beta 0 released.
* Worker/Ssh.py: (feature) new worker, based on OpenSSH, with fanout
support (thus removing ClusterShell mandatory pdsh dependency).
* Engine/Engine.py: (feature, api) added timer and repeater support.
* 1.0->1.1 internal design changes.
Copyright CEA/DAM/DIF (2009, 2010)
Copying and distribution of this file, with or without modification, are
permitted provided the copyright notice and this notice are preserved.
