2009-03-05 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.17 released. * Fixed low-level scheduler timer computation to take care to monothonic computation. Select returns if timer is null! * VRRP : Fixed vrrp script initialization to use event thread instead of timer thread so that script no longer need to wait until first polling timer fired. * VRRP : Willy and I fixed MII media link failure detection to test SIOCGMIIREG call before fetching BMSR. * VRRP : Resurected VRRP_STATE_GOTO_FAULT. This state is really needed to speed-up convergence and prevent against any issue while using vrrp_sync_group. 2009-02-15 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.16 released. * Code clean-up. * Stefan Rompf, <stefan@loplof.de> extended scheduler to synchronize signal handling by sending the signal number through a self pipe, making signals select()able. Child reaping has been moved to a simple signal synchronous signal handler. Signal shutdown handling has been centralized. * Denis Ovsienko, <pilot@etcnet.org> extended healthchecker framework to support alpha/omega design. It provides virtual service control in a more fine-graned maner. You may have a look to the SYNOPSIS file to have full picture on configation. It addresses the following issues : - A virtual service is considered up even with an empty RS pool. - There is no reliable mean to avoid service regression, when the server pool becomes too small. - There is no mean to escalate any of the above fault/recovery events. - Real servers are assumed alive initially. This leads to unnecessary state flap on keepalived start. - notify_down isn't executed for working real servers on keepalived shutdown. - There is no reliable mean to handle keepalived stop to move the virtual service over another load balancer. * Stephan Mayr, <Mayr.Stefan@swm.de> fixed default value for checker loop... a missing TIMER_HZ. * Merge keepalived.init.suse. * Robin Garner, <robin.garner@scu.edu.au> added support to --log-console facility. * Tobias Klausmann, <klausman@schwarzvogel.de> fixed an openfile leak while performing reload. * Leo Baltus, <Leo.Baltus@omroep.nl> extended pidfile handling to allow keepalived to start using configurated pidfile. * VRRP : Siim Poder, <siim@p6drad-teel.net> fixed IPSEC AH auth to skip IPv4 id field of zero. If zeroed kernel will fill it and lead to an unwanted protocol re-election. * VRRP : Siim Poder, <siim@p6drad-teel.net> fixed reloading issue. New ip addresses are added (from configuration). State is kept instead of starting from whatever is in configuration file. If prios are changed in such a way, state change can occur after reload. * VRRP : Vincent Bernat, <bernat@luffy.cx> extended virtual_route to support virtual "black hole" route as well as multihop route. * VRRP : Stig Thormodsrud, <stig@vyatta.com> fixed a crash while using virtual_router_id set to 255. * VRRP: Jon DeVree, <jadevree@arbor.net> fixed arp handling to to initialize the target hardware address, using 0xff as found in arping. Let scripts work without dealing with weight, if the script fails, VRRP fails. * VRRP : Pierre-Yves Ritschard, <pierre-yves@spootnik.org> removed the GOTO_FAULT state from FSM. * VRRP : Willy Tarreau, <w@1wt.eu> fixed link detection handling to support right ioctl values for recent kernel ! It can lead to issue while running instance on a bonding interface. * VRRP : Willy Tarreau, <w@1wt.eu> extended scheduler to catch time drift. It implements an internal monotonic clock. It maintains an offset between sysclock and monotonic clock, if computed time if anterior to monotonic time then just update offset. If time computed if fare away into the future then limit delay and recompute offset. * VRRP : Willy Tarreau, <w@1wt.eu> fixed autoconf issues. 2007-09-15 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.15 released. * Matthias Saou, <matthias at rpmforge.net> fixed genhash Makefile for man page installation. * Casey Zacek, <keepalived at bogleg.org> provided a patch to check_http to remove buffer minimization while processing stream. It appears some webserver cause healthchecker crash. * Chris Marchesi, <chris.marchesi at canadawebhosting.com> provided a patch for better handling of SSL handshake errors. * Shinji Tanaka, <stanaka at hatena.ne.jp> fixed parser "include" directive to support declaration inside configuration directives, like including file inside vrrp_instance declaration. * Andreas Kotes, <count at flatline.de> fixed HTTP healthchecker while handling MD5SUM result. It appears checker never removed realserver on MD5SUM mismatch !!! whats that crap. * VRRP : Willy Tarreau, <w at 1wt.eu> fixed a missing notifications upon transition from fault to backup. * VRRP : Add support to route metric in virtual_routes definition. 2007-09-13 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.14 released. * Shinji Tanaka, <stanaka at hatena.ne.jp> extended parsing framework to support "include" directives. For more informations and documentation please refer to Shinji website : http://misccs.dyndns.org/index.php?keepalived%20include%20patch * Tobias Klausmann, <klausman at schwarzvogel.de> add error loggin while parsing configuration file. * Merged patches from rpmforge.net on Makefile and redhat specfile. * Create a goodies directory to store nice scripts received from users. Add Steve Milton (milton AT isomedia.com) arpreset script to delete a single ARP entry from a CISCO router. * VRRP : David Woodhouse, <dwmw2 at redhat.com> fixed vrrp_arp includes. * VRRP : Pierre-Yves Ritschard, <pyr at spootnik.org> fixed negative weights in script. * VRRP : Michael Smith, <msmith at cbnco.com> extended virtual_ipaddress setting to support Old-style Linux interface aliases like eth0:1. * VRRP : Ward Wouts, <ward.wouts at gmail.com> add support to vrrp_script logging. 2006-10-11 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.13 released. * VRRP : Added a new notify script to be launch during vrrp instances shutdown. This new notify hook is configured using notify_stop keyword inside vrrp_instance block. * VRRP : Willy Tarreau <w at 1wt.eu> fixed an errno issue in thread_fetch(), errno is lost during set_time_now(). This patch saves it across the call to set_time_now() in order to get the valid error. * VRRP : Willy Tarreau <w at 1wt.eu> extended timer framework to save errno in timer_now() and set_time_now() just in case other functions do not expect these functions to modify it. This is a safer approach than the initial patch to thread_fetch(), while still compatible. * VRRP : Willy Tarreau <w at 1wt.eu> fixed an FSM silent issue. By default, the VRRP daemon stops sending during new MASTER elections. This causes 3 to 4 seconds of silence depending on the local priority, and sometimes causes flapping when the differences in priorities are very low, due to the kernel timer's resolution : sometimes, the old master receives a first advertisement, enters backup, waits 3 seconds, sees nothing and finally becomes master again, which forces a new reelection on the other one. * VRRP : Willy Tarreau <w at 1wt.eu> extended VRRP framework to support floating priority. Replace the priority in each vrrp_instance with a base priority and an effective priority, to prepare the support for floating priorities. The configuration sets the base_priority, and all comparisons use the new effective_priority value. This one is computed in the vrrp_update_priority() thread by adding an offset to base_priority, based on the result of various checks. * VRRP : Willy Tarreau <w at 1wt.eu> extended notify script to add the priority in "$4" when calling a notify script. This is important in labs and datacenters when systems can display the priority on a front LCD, because it allows workers to carefully operate without causing unexpected reelections. * VRRP : Willy Tarreau <w at 1wt.eu> extended interface tracking framework to let interface tracking change the priority by adding a "weight" parameter. If the weight is positive, it will be added to the priority when the interface is UP. If the weight is negative, it will be subtracted from the priority when the interface is down. If the weight is zero (default), a down interface will switch the instance to the FAULT state. * VRRP : Willy Tarreau <w at 1wt.eu> added a new "vrrp_script" section to monitor local processes or do any type of local processing to decide whether the machine is in good enough health to be elected as master. A same script will be run once for all instances which monitor it. If no instance use it, it will not be run, so that it's safe to declare a lot of useful scripts. A weight is associated to the script result. If the weight is positive, it will be added to the priority when the result is OK (exit 0). If the weight is negative, it will be subtracted from the priority when the result is KO (exit != 0). If the weight is zero, the script will not be monitored. The default value is 2. * VRRP : Willy Tarreau <w at 1wt.eu> extended vrrp scheduler so that when a VRRP is part of a SYNC group, it must not use floating priorities, otherwise this may lead to infinite re-election after every advertisement because some VRRPs will announce higher prios than the peer, while others will announce lower prios. The solution is to set all weights to 0 to enable standard interface tracking, and to disable the update prio thread if VRRP SYNC is enabled on a VRRP. * VRRP : Willy Tarreau <w at 1wt.eu> added some documentation and examples for the brand new VRRP tracking mechanisms. * VRRP : Ranko Zivojnovic, <ranko at spidernet.net> fixed vrrp scheduler to execute notify* scripts in transition from the failed state to the backup state. * Nick Couchman, <nick.couchman at seakr.com>, added support for real server upper and lower thresholds. This allows you to set a minimum and maximum number of connections to each real server using the "uthreshold" (maximum) and "lthreshold" (minimum) options in the real_server section of the configuration file. * Chris Caputo, <ccaputo at alt.net> extended autoconf script to support recent move of UTS_RELEASE from linux/version.h to linux/utsrelease.h. * Chris Caputo, <ccaputo at alt.net> extended ipvswrapper 2.4 code to support misc_dynamic weight. 2006-03-09 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.12 released. * VRRP : Christophe Varoqui, <Christophe.Varoqui@free.fr> extended VRRP framework to use virtual_router_id as syncid in LVS mcast datagram while using LVS syncd in VRRP instance. * Kevin Lindsay, <kevinl@netnation.com> and Christophe Varoqui, <Christophe.Varoqui@free.fr> fixed SSL checker to properly use openssl when dealing with asynchronous stream handling. Kevin fixed asynchronous handling during connection stage while Christophe fixed stream handling after connection stage. * Kjetil Torgrim Homme, <kjetilho@ifi.uio.no> extended keepalived spec file to cleanly compile on RedHat enterprise 3 and 4. * Heinz Knutzen, <Heinz.Knutzen@dataport.de> fixed SMTP checker to overwrite default_host while parsing configuration file. A SMTP_CHECK without a "host" section should use the ip of the current real server as default. 2005-03-01 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.11 released. * Asier Llano Palacios, <a.llano@usyscom.com> extended autoconf script to support cross-compilation. * Kevin Lindsay, <kevinl@netnation.com> and I fixed a missing bitwise negation while removing signal from global signal mask. Set this operation before handler is called. This assume that bitwise negation is an atomic code generated from compiler. Since gcc 3.3 this is true. * VRRP : extended ipaddress and iproutes code to return if vip or vroutes is referencing an unknown interface. 2005-02-15 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.10 released. * VRRP : While restoring interface, release iproutes before ipaddresses. Routing daemons needs that order for netlink reflection channel. * VRRP : Bin Guo, <bguo@bluesocket.com> fixed a memory leak while calling script_open. * Kevin Lindsay, <kevinl@netnation.com> fixed some buffer overruns, NULL pointer and dangling pointer references. * Kevin Lindsay, <kevinl@netnation.com> redisigned signal handling. When a signal occurs, a global signal_mask is modified. In the main loop there is a checked to see if the signal_mask has any pending signals. The appropriate signal handler is then run at this time. This is to prevent races when modifying linked lists. * Kevin Lindsay, <kevinl@netnation.com> fixed shadowed declarations. * Christophe Varoqui, <Christophe.Varoqui@free.fr> and I Extended libipvs-2.6 to support syncd zombies handling. Since ip_vs_sync.c kernel code no longer handle waitpid() we fork a child before any ipvs syncd operation in order to workaround zombies generation. * John Ferlito, <johnf@inodes.org> and I Fixed a scheduling race condition while working with low timers. * Updated check_http and check_ssl to use non-blocking socket. * Fixed some race conditions while reloading configuration. Prevent against list gardening if list is empty ! * Fixed recursive configuration parsing function to be clean with stack. Only one recursion level. * Some cosmetics cleanup in Makefiles. 2005-02-07 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.9 released. * VRRP : Chris Caputo, <ccaputo@alt.net> updated keepalived manpage for nopreempt and preempt_delay. * VRRP : Fixed an issue while releasing vrrp socket pool... Just release pool one time ! * VRRP : Fixed netlink framework to properly save netlink socket flags while setting blocking flags. * VRRP : Fixed a regression introduced with previous release while hashing vrrp fd bucket into fd hash index. * Patrick Boutilier, <boutilpj@ednet.ns.ca> fixed an issue in the extract_html function. Read the full html header. * Chris Caputo, <ccaputo@alt.net> and I fixed compilation issue while using --enable-debug configuration option. * Extended both VRRP and Healthchecker framework to support debugging flags. * Removed the watchdog framework. Since scheduling framework support child, we register a child thread for both process VRRP & Healthcheck. When child die or stop prematuraly this launch scheduling callback previously registered. Watchdog is now handled by signaling. (credit goes to Kevin Lindsay, <kevinl@netnation.com> for nice idea). * Some cosmetics cleanup. 2005-01-25 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.8 released. * VRRP : Chris Caputo, <ccaputo@alt.net> added "dont_track_primary" vrrp_instance keyword which tells keepalived to ignore VRRP interface faults. Can be useful on setup where two routers are connected directly to each other on the interface used for VRRP. Without this feature the link down caused by one router crashing would also inspire the other router to lose (or not gain) MASTER state, since it was also tracking link status. * VRRP : Chris Caputo, <ccaputo@alt.net> added "nopreempt" which overrides the VRRP RFC preemption default. This replaces the "preempt" keyword which was not fully implemented. "preempt" is kept around for backward compatibility but is deprecated. * VRRP : Chris Caputo, <ccaputo@alt.net> added "preempt_delay" which allows one to specify number of seconds after startup until VRRP preemption. (range 0 to 1,000 seconds) this is useful because sometimes when a machine recovers it takes a while for it to become usable, such as when it is a router and BGP sessions need to come back up. * Chris Caputo, <ccaputo@alt.net> made it so there is a useful "Date:" in SMTP alert emails. * VRRP : Chris Caputo, <ccaputo@alt.net>. In debug output log gratuitous ARPs with actual IP addresses being ARPed. * VRRP : Chris Caputo, <ccaputo@alt.net>. If started with "--dont-release-vrrp" then try to remove addresses even if we didn't add them during the current run, when it makes sense to do so. * VRRP : Chris Caputo, <ccaputo@alt.net> added a missing free_vrrp_buffer() during VRRP stop. * VRRP : Kees Bos, <k.bos@zx.nl> fixed VRRP sanity check to perform checksum computation over incoming packet and not local router instance memory representation => Better to log 'invalid vip count' instead of 'Invalid vrrp checksum' when the number of configured vips differ in the master and backup server :) * VRRP : Release socket pool during daemon stop and reload * VRRP : Refresh socket pool during reload * VRRP : Extended netlink framework to support blocking operation. During initialization, set blocking netlink channel to wait responses from kernel while parsing result. Kernel netlink reflection are still handled using non-blocking. * Jeremy Rumpf, <rumpf.6@osu.edu> added SMTP checker. It take a special care of smtp server return code. * Merged genhash man page * Chris Caputo, <ccaputo@alt.net> added "misc_dynamic" to a MISC_CHECK which makes it so a script can adjust the weight of a real server. * Fixed some assertion issue in memory framework. * Use router_id instead of lvs_id in the global_def configuration block (lvs_id kept for backward compatibility). * Ronald Wahl <rwa@peppercon.com>, fixed declarations to be only in includes files. * Ronald Wahl <rwa@peppercon.com>, moved the definition of variables to C files * Ronald Wahl <rwa@peppercon.com> and I fixed scanning for header/body separator in HTTP protocol * Ronald Wahl <rwa@peppercon.com> replaced memcpy by memmove where source & destination may overlap * Extended checker API to only register checkers when checker callback is defined. * Jacob Rief, <jacob.rief@tiscover.com> fixed openlog to take care of configured log facility. * Move in_csum to util file. * Extended libraries to support some new facilities (list and vector). * Extended scheduler I/O to use timer decalred on the stack. * Some cosmetics changes. 2004-04-05 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.7 released. * Jacob Rief, <jacob.rief@tiscover.com> added target tarball into root Makefile to facilitate packaging (rpm & tarball). * Jacob Rief, <jacob.rief@tiscover.com> and I unified version handling. Now only the root file VERSION is used by configure to add VERSION_STRING via config.h.in. Added VERSION_DATE included into the VERSION_STRING that reflect the building date into the version banner. * Andres Salomon, <dilinger@voxel.net> wrote the genhash manpage. * VRRP : Added ipvs_start() and ipvs_stop() calls during vrrp child start and stop stage. * Added some assertion test in memory framework to not allocate bucket if no more place. This option is only used if compiled with debug flags. * Some cosmetics patch in Makefiles and autoconf script. 2004-02-23 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.6 released. * VRRP : Fixed scheduling timer update. Global scheduling timer is updated before each thread registering and after scheduling I/O MUX. Since is needed to take care of scheduling jitter introduced by overhead (VRRP is using low low timer so more sensitive to overhead). Thanks to Nathan Neulinger, <nneul@umr.edu> for his quick feedback debugging time. * VRRP : Nathan Neulinger, <nneul@umr.edu> updated vrrp dropping strategy to not reply to incoming bogus adverts. Since this can introduce flooding loop, bogus adverts are now simply silently dropped. * VRRP : Fixed a linkbeat issue while polling NIC flags. * Updated autoconf and Makefile to support 2.6 kernel IPVS code. For code readability, created 2 differents libipvs for 2.4 and 2.6 kernel . Fixed autoconf generated warning. * Extended ipvswrapper to support shared buffer user rule. This increase performances by limiting memory allocation. OTOH, created two new ipvs helpers ipvs_start & ipvs_stop to initialize ipvs subsystem. * Andres Salomon, <dilinger@voxel.net> made some cosmetics update in Makefiles to support $(DESTDIR) and $(BIN)/$(EXEC) path split. 2004-01-25 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.5 released. * Joseph Mack, <mack.joseph@epa.gov> wrote keeplived manpages in doc/man/man5/keepalived.conf.5 and doc/man/man8/keepalived.8. * VRRP : Tsuji Akira, <tsuji@centurysys.co.jp> fixed a length issue while testing password field for auth_pass method. * VRRP : Willy Tarreau, <willy@w.ods.org> fixed a quick loop in the watchdog timer thread. * VRRP : Willy Tarreau, <willy@w.ods.org> extended scheduler to support stable scheduling time. There is now, only one time source updated before and after scheduling event. This solve sliding timer observed on some env, also known as periodically flapping issue (sometime a VRRP election is forced). * VRRP : Willy Tarreau, <willy@w.ods.org> updated the default media link failure detection strategy to perform a ioctl ifflags even if NIC driver are supporting MII or ETHTOOL. Some buggy drivers need this. Anyway the linkwatch patch still the best solution to support efficient and scalable media link failure detection. * Some cosmetics clean-up, removed some dead files, updated autoconf and Makefile prototypes to support dependencies libs like kerberos for RedHat/Fedora distro. To compile keepalived properly on redhat 9 box, for example, run : export CPPFLAGS="-I/usr/kerberos/include" && ./configure Renamed keywords lb_kind to lvs_method and ld_algo to lvs_sched. For compatibility reasons, old keywords are still available. 2003-12-29 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.4 released. * Refresh autoconf script to use autoconf 2.5. * Extended the autoconf script to support linkwatch kernel detection. * To work-around the SMP forking bug, added support to two new daemon starting options : --vrrp -P Only run with VRRP subsystem. --check -C Only run with Health-checker subsystem. Those options extend daemon design to support VRRP & heathchecking subsystem selection. You can now run two Keepalived daemon one invoqued with --vrrp and the other with --check. That way we workaround the forking issue by running one daemon per subsystem. * Tiddy cleanup in the daemon code. * VRRP : Extended the link media failure detection to support asynchronous NIC MII polling. The design use now, one dedicated polling thread per NIC. This reduce scheduling jitter by this way. * VRRP : Added support to kernel linkwatch subsystem. This patch that you will find a copy on the Keepalived website for the kernel 2.4 branch, provides kernel netlink broadcast events drived by NIC link media state event. That way we move from a polling design to an event design. Link events are received throught a kernel netlink broadcast socket in the userspace land. So, NIC media link failure detection is now provided by kernel netlink reflection. You can read the paper attached with the patch for indepth explanations. * VRRP : fixed timer computation to prevent against negative value. 2003-09-29 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.3 released. * Stephan von Krawczynski, <skraw@ithnet.com> extended ip address framework to support broadcast address selection. * Extended the scheduling framework to support plain 'long' timer. Visited the layer4 framework to support this new scheduling scheme. Reviewed the checkers and VRRP framework to support long timer. * VRRP : Removed the timer micro adjust call. Its use is obsolete with the new scheduling 'long' timer support. * Jacob Rief, <Jacob.Rief@tiscover.com> and I added support log level selection for main daemon. A new command line argument has been created : --log-facility -S 0-7 Set syslog facility to LOG_LOCAL[0-7]. (default=LOG_DAEMON) * Extended the HTTP checker to support non blocking read while processing stream. NONBLOCK flags is set before read operation to catch EAGAIN error. * VRRP : Diego Rivera, <lrivera@racsa.co.cr> and I fixed a notify issue while building notify exec string. * VRRP : Diego Rivera, <lrivera@racsa.co.cr> and I extended FSM to support BACKUP state notifiers and smtp_alert call during VRRP initialization. * Jan Vanhercke, <jan.vanhercke@c-cure.be> and I extended scheduling timer computation to support micro-sec second overlap. Extended the whole scheduling framework to support this scheduling scheme while computing thread timers. * Fixed scheduling framework to support child thread timers while computing global scheduling timer. 2003-09-07 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.2 released. * Dominik Vogt, <dominik.vogt@gmx.de> and I extended checker framework to support multiple checkers per realserver. Each checker own a uniq id, each realserver own a list of checkers id. Realserver is considered down if one of the checkers fails. * Dominik Vogt, <dominik.vogt@gmx.de> extended list library to support free_list_element. * Dominik Vogt, <dominik.vogt@gmx.de> and I extended ipwrapper to support multiple checkers test. Created a checker state updater helper function to perform realserver state according to checker state. * Dominik Vogt, <dominik.vogt@gmx.de> extended all checkers code to support multiple checker design (to not perform server state according a single checkers test). * Tobias Klausmann, <klausman@schwarzvogel.de> and I extended layer4 framework to support socket binding to a specific ip address before calling connect(). Extended the TCP, HTTP and SSL checker to support binding selection, creating a new checker keyword named "bindto". look at doc/keepalived.conf.SYNOPSIS for more informations. * VRRP : Extended the ethtool code to be selected only if ETHTOOL_GLINK is available. This is useful for s/390 zSeries users :) since zSerie 2.4 kernel doesn't support ethtool extension. * VRRP : Gatis Peisenieks, <gatis@mt.lv> fixed IPSEC-AH code to exclude ip header id filed while computing AH digest. Fixed AH sequence number to be set in network byte order. * VRRP : Fixed a bug in the static_ipaddress block that caused a noisy crashing startup. * VRRP : Kjetil Torgrim Homme, <kjetilho@ifi.uio.no> and I fixed a daemon crash while reloading configuration due to a vrrp_buffer not freed. * VRRP : Review the watchdog calling location. watchdog listener is reinitialized during a daemon reload. * VRRP : Diego Rivera, <lrivera@racsa.co.cr> extended notify framework to support simple notify script call. Created a new keyword "notify", for both vrrp_instance and vrrp_sync_group. If configured, this notify script is called after FSM state transition notify scripts. look at doc/keepalived.conf.SYNOPSIS for more informations. * Review the checker watchdog calling location like VRRP. * Fixed code selection to exclude VRRP dependencies if code is configured without VRRP framework. * Extended memory lib free function to reset memory location to NULL. * Diego Rivera, <lrivera@racsa.co.cr> extended global parser to support default handlers for lvs_id, smtp_server, smtp_connection_timeout and email_from. default values are : o lvs_id : box local name o smtp_server : localhost o email_from : uid@box_local_name o smtp_connection_timeout : 30s 2003-07-24 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.1 released. * VRRP : Fixed an issue while reloading configuration. Fixed a dereferencing pointer. * Fixed misc checker to perform server state according to checker result !!! 2003-07-22 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.1.0 released. * The release focus is : "High Performance" * Name cleanup for the healthchecking directory. use check instead of healthcheck to be in conformance with watchdog and global software architecture. * updated the SYNOPSIS file for documenting the table arg inside virtual/static_routes declaration. You can set routes refering to a specific TABLE-ID. * Added a dummy debug var in the genhash declaration code to support compilation when compilation is done with debug flag. * Added a set flag inside the real_server declaration correctly relfect the IPVS topology when inhibit_on_failure is used. * fixed a daemon.h include depandency on signal.h * VRRP : Added support to a global shared buffer for incoming advert handling. A new buffer is no longer allocated each time processing incoming advert, instead a shared room is used. * VRRP : Added support to pre-allocated shared buffer for outgoing adverts. Each vrrp instance use a 'one time' allocated buffer instead of a 'all time' one. * VRRP : Extended the socket pool design to support shared fd for the outbound channel. Now, socket pool create a sending socket and affect the fd returned to vrrp instances. This forces instances to use a shared socket instead of creating new socket for each outgoing adverts. The error detection is based on the incoming socket, so that outgoing socket is not created as long as incoming socket can not be created. * Added support to netlink ipaddress as global keyword "static_ipaddress". look at doc/samples/keepalived.conf.static_ipaddress. IP addresses specified into this block will be added during daemon bootstrap and removed during daemon shutdown. Differential conf parsing is enabled for this block, removing/adding static_ipaddress can be done on the fly sending SIGHUP signal to daemon. * VRRP : Extended track_interface to support multiple interface tracking. For those familiar with Nokia monitored circuit, this extention provide the same functionality. look at doc/samples/keepalived.conf.track_interface. * VRRP : The VRRP instance lookup framework has been extended to use a o(1) scheduling design. Rewrote the whole instance lookup to use o(1) lookup instead of previous o(n^2). When receiving incoming adverts vrrp_scheduler performs a lookup over the VRID received to get local instance representation. Since the internal instance representation is an non-sorted linked list, then we run a lookup at o(n^2) complexity that introduce lantency and scheduling jitter side effect when runing large number of instances. To avoid this limitation a static hash table of 255 buckets were created. Since lookup is performed over VRID and since VRID is 8bit fixed, then the hashkey will be VRID. In order to extend code the hashkey is based on incoming fd too. Internally, a NIC is represented by a 2 fds : sending socket and receiving socket. Those fds are NIC specific so we are using them as a hash table lookup collision resolver. With this design we can now use the same VRID on different NICs. The collision design is a linked list so lookup is o(n^2) but due to low number of entries we can consider o(1) speed. But to reach best perf, differents VRID on all instance must be used. The design can be sumed by : VRID hash table : +---+---+---+---+---+---+.........+-----+ | 1 | 2 | 3 | 4 | 5 | 6 |.........| 255 | +---+---+---+---+---+---+.........+-----+ | | +---+ +---+ |fd3| |fd1| +---+ +---+ | +---+ |fd5| +---+ This hash table is filled during configuration parsing and VRRP instances are not duplicated but dynamically pointed to optimize memory. * VRRP : The VRRP synchronization group lookup has been extended. During bootstrap a VRRP instance index is built upon sync_group instance name. This extension speed up synchronization since while synchronizing it perfoms the instance index instead of lookup by instance_name. The previous synchornization code has been rewritten to use this 'list visiting' design for FAULT/BACKUP/MASTER states synchronization. * VRRP : Optimized the vrrp_timer_vrid_timeout(...) to speed up vrid lookup over timeouted fd using a one pass lookup. * Bradley Baetz, <bradley.baetz@optusnet.com.au> extended the scheduler framework to support child process handling. Adding support to new thread child facility for handling child processes, and modifying the scheduling select loop & signal handling to catch SIGCHLD, and call the appropriate process. * Bradley Baetz, <bradley.baetz@optusnet.com.au> fixed the misc_check healthchecker using new thread child scheduling facility. Introduced a new keyword "misc_timeout" to kill processes which take too long time (default is delay_loop). SIGKILL is send to processes if they take too long time to shutdown. * Bradley Baetz, <bradley.baetz@optusnet.com.au> extended daemon framework to block SIGCHLD to only receive it whn its unblocked in the scheduling loop. * Extended healthchecker delay_loop to support long delay (ie: >1000s). * VRRP : Added support to a shared kernel netlink command channel for setting ip address and routes. * Extended the genhash code to support verbose output selection. command arg '-v' will generate a very verbose output. * VRRP : Extended the logging code to select verbose log output or not. This selection is done by passing the '-D' option to command line while starting daemon. By default the output is silent. * VRRP : Extended the gratuitous ARP framework to support shared buffer and shared socket. This increase performances for instances owning a bunch of VIP. * VRRP : Extended the scheduling timer computation to support timer auto-recalibrating. While computing next instance timer, the scheduler will substract the time taken by previous advert handling. This provide software overhead adaptation. The recalibration is performed over usec timer to not pertube global scheduler. * VRRP : Fixed a gratuitous ARP issue. Extended the ipaddress framework to point directly to interface reflected by netlink channel instead of storing device index. Extended the gratuitous ARP code to use new ipaddress structure and for sending garp over device ipaddess belong to. Needed if you run an instance on one device interface and set VRRP VIP on different interface. * Extended watchdog framework to support polling delay selection via daemon command line. Created two new cmdline options : --wdog-vrrp -R Define VRRP watchdog polling delay. (default=5s) --wdog-check -H Define checkers watchdog polling delay. (default=5s) * Extended SMTP code to support bigger buffer while processing remote mta messages. * Erik Barker, <erikb@netnation.com> extended initscript to support native redhat init functions. * Extended the autoconf scripts and Makefile(s) to support code profiling. New configure option : --enable-profile * list library has been extended to support multi-sized list & specific element deletion. Extended to return when list is empty. This reduce duplicated code to test is list is empty while processing. * VRRP : Extended VRRP scheduler to support fd hash table design. Speed up instance lookup while computing instance sands. This offer o(1) design if we consider limited number of instances per device. * VRRP : Extended vrrp new socket creation to replace refreshed instance fd into fd hash table index. * VRRP : Extended vrrp framework to support blank virtual_ipaddress block, can be usefull if someone want to use just the VRRP advert as hello monitoring channel. * Some code cleaning. 2003-05-12 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.0.3 released. * This release has been sponsorized by : Tiscover AG, <www.tiscover.com> Please visit sponsor homepage. I would just like to thanks their IT team for interresting design discussions and testing time, especially Jacob Rief. * This release consist of a major daemon re-design to increase security and availability of Keepalived. The daemon has been splitted into 3 distinct process. The global design is based on a minimalistic parent process responsible for monitoring its forked children process. Then 2 children process, one responsible for VRRP framework and the other for healthchecking. Each children process has its own scheduling I/O multiplexer, that way VRRP scheduling jitter is optimized since VRRP scheduling must be more sensible than healthcheckers. On the other hand this splitted design minimalize for healthchecking the usage of foreign librairies and minimalize its own action down to and idle mainloop in order to avoid malfunctions caused by itself. The parent process monitoring framework has been called watchdog, the design is : each children process open an accept unix domain socket, then while daemon bootstrap, parent process connect to those unix domain socket and send periodic (5s) hello packets to children. If parent cannot send hello packet to remote connected unix domain socket it simply restart children process. This watchdog design offer 2 benefit, first of all hello packets sent from parent process to remote connected children is done throught I/O multiplexer scheduler that way it can detect deadloop in the children scheduling framework. The second benefit is brought by the uses of sysV signal to detect dead children. When running you will see in process list : PID 111 keepalived <-- parent process monitoring child activity 112 \_ keepalived <-- VRRP children 113 \_ keepalived <-- Healthchecking children * Parent : Created a global data and global keyword parser structure. * Healthcheck framework : Defined check_conf_data to handle related checker data structures. Created specific checker framework parser. * VRRP framework : Defined vrrp_conf_data to handle related vrrp data structures. Created specific vrrp framework parser. * Each child process has its own syslog facility. VRRP use LOG_LOCAL1 and Healthchecker LOG_LOCAL2. To split log you can so configure your syslog to log both facilities in a different logfile. * Modularized the configuration parser to limit code duplication. * Created modularized software watchdog. * Extended the recursive stream parser to use sublevel detection while stream processing. Used to skip end-of-block handling if still at keyword root level to prevent against end parsing if unknown block is parsed. * Extended pidfile framework to be more generic. * Extended memory framework to log specific child data. * Fixed a virtual_server_group issue while healthchecker bringing back real_servers. Modularized virtual_server_group API. * Fixed a virtual_server_group issue will reloading configuration. Remove vsgname test from the VS_ISEQ macro. strcmp(...) comparing null pointer... this must have been done in libc :) * ipwrapper : set alive flag after ipvs_cmd(...) has been performed. * VRRP : Extended the netlink framework to support SCOPE selection for both ipaddress and routes fonctionnalities. SCOPE available are site, link, host, nowhere & global. Default value is set to global. look at doc/keepalived.conf.SYNOPSIS for more informations. * Renamed doc/samples/keepalived.conf.routes to doc/samples/keepalived.conf.vrrp.routes. * Updated Makefile include dependencies. 2003-04-14 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.0.2 released. * This release has been sponsorized by : edNET, <www.ednet.co.uk> Please visit sponsor homepage and thanks to them for supporting keepalived project. * Added support to virtual_server_group so that a virtual_server can be either an IP:PORT, a fwmark or group. A group is a set of virtual_server IP:PORT, IP range and fwmark. So, now a real_server can be part of multiple virtual_server without launching multiple time the same healthchecker that finaly flood real_server. This extension is useful for big ISP/ASP configuration using many virtual_server. look at doc/samples/keepalived.conf.virtual_server_group. * Extended differential configuration parser to support diff virtual_server_group entries keeping current entry state as persistent (weight, conn, ...) big work here... * Added support to IP range declaration for virtual_server_group. The IP range has the notation XXX.YYY.ZZZ.WWW-VVV. This will set IPVS virtual_server from WWW to VVV monotonaly incremented by one. look at doc/samples/keepalived.conf.virtual_server_group. * Dominik Vogt, <dominik.vogt@gmx.de> enhanced SIGCHLD handler to reap all zombie child processes. * Created a generic allocation value block with callback handler for block parsing. This remove duplicated code in parser. * VRRP : Jan Holmberg, <jan@artech.net> extended the virtual_routes and static_routes to support source route selection (netlink RTA_PREFSRC). look at doc/samples/keepalived.conf.routes. * Some cosmetics patches to reduce code duplication. 003-03-17 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.0.1 released. * This release has been sponsorized by : Creative Internet Techniques, <www.httpd.net> Please visit sponsor homepage, open minded people here ! * Fixed some Makefile and autoconf code dependence issues. * Move keepalived.conf.SYNOPSIS and samples into "doc" directory. * Enhanced HTTP|SSL check to support large url. Get buffer request is now 2KBytes. * Removed \n in healthchecker smtp_alert call. This cause some troubles with MTA like qmail. Thanks go to John Koyle, <jkoyle@rfpdepot.com>. * Added support to netlink route as global keyword "static_routes". look at doc/samples/keepalived.conf.routes. Routes specified into this block will be added during daemon bootstrap and removed during daemon shutdown. Differential conf parsing is enabled for this block, removing/adding static_route can be done on the fly sending SIGHUP signal to daemon. * VRRP : Added support to "virtual_routes". This is the same as virtual_address. Those routes are set when VRRP instance enter MASTER state and removed otherwise. Differential conf parsing is enabled for this block. This concept extend VRRP and bring dynamic routing as a "route takeover" concept. * VRRP : Rewrote the VRRP vip handling to use template lib list structure. VIP and E-VIP are no longer a simple array reallocated. List library is used to limite code duplication. * VRRP : Extended virtual_ipaddres and virtual_ipaddress_excluded block to support "dev" specification. So that a VIP can be set to a specific interface instead of default runing VRRP instance interface. * VRRP : Added support to "track_interface". Interesting for use with vlan interface. The concept here is to drive VRRP FSM according do both "interface" and "track_interface" state. If tracked interface is down or instance interface is down then VRRP instance transit to FAULT state. For use with vlan, add track to interface vlan belong to. Look at doc/sample/keepalived.conf.track_interface for sample. doc/keepalived.conf.SYNOPSIS for configuration details. * VRRP : Extended FSM FAULT state to keep in fault if track_interface still fault. * VRRP : Extended sync group design to test if group is unary or not. * Some code cleaning and cosmetics enhancements. 2003-01-06 Alexandre Cassen <acassen@linux-vs.org> * keepalived-1.0.0 released. * After fixed all bugs users reported during 2 months, I am glad to announce the first STABLE production ready Keepalived release. * Rename keepalived.init to keepalived RedHat startup script. Fixed some issues to be RedHat release generic. Thanks go to Jeroen Simonetti <jeroens@q-go.com> & Jason Gilbert <jason@doozer.com> * Jason Gilbert, <jason@doozer.com> cleaned keepalived.spec. * Added support to "ha_suspend" for healthcheckers. This option, if set, inform Keepalived to active/suspend checkers according to netlink IP address information reflection. If one IP is removed and this is a virtual_server VIP then the healthcheckers corresponding will be desactivated. (and reciprocity). * Added support to "notify_up" & "notify_down" for realserver config. These options specify a script to be run according to healthchecker activity. If healthchecking fails then "notify_down" script is launched (and reciprocity for healthcheck succeed). This can be usefull for global monitoring system, to send alert to Unicenter TNG or HPOV. * Set default realserver weight to 1. So, realserver will be active if no weight is specified into the configuration file. * Review the layer4.c/tcp_socket_state to return connection in progress only if SOL_SOCKET/SO_ERROR return EINPROGRESS. Thanks go to Mark Weaver, <mark@npsl.co.uk> * Reviewed the global SIGCHLD handler to not suspend execution of the calling process if status is not immediately available for one of the child processes. This remove zombies by reaping. * Extended the parser.c/set_value() code to accept encapsulated quoted string. * Review SMTP DBG() message to LOG_INFO message for more verbose error handling. * Review the check_tcp.c/check_http.c logging messages to be more detailed. * Review the check_tcp.c/check_http.c retry facility to fixes some stalled issues. * VRRP : Added support to sync_group smtp notification in addition to the per instances approach. * VRRP : Fixed some IPSEC-AH seq_num synchronizations issues. Force seq_num sync if vrrp instance is linked to a group. * VRRP : In BACKUP state, force a new MASTER election is received adv. has a lower priority as locale instance. * VRRP : vrrp.c/vrrp_state_master_rx(), sync IPSEC-AH seq_num counter (decrement) if receiving higher prio advert in MASTER state. * VRRP : Reviewed the TSM to be fully filled. Extended speed-up synchronization handling MASTER sync if group is not already synced. * VRRP : Leaving fault state, force MASTER transition is received adv priority is lower than locale. * VRRP : Extended the parser to not be borred with sync_group declaration position in the conf file. vrrp_sync_group can be declared before or after vrrp_instance. Done by adding a reverse instance lookup during parsing. * VRRP : sync_master_election cleanup. * Some cosmetics patches. * Created the keepalived/samples/keepalived.conf.SYNOPSIS to describe all keywords available. 2002-11-20 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.7.6 released. * Created a common library for code modularization. This lib will be used by all Keepalived components (genhash + Keepalived) to reduce repeated and duplicated code. * Rewrote the genhash utility using the common lib. The design is similar to Keepalived core design. * Reviewed the autoconf and Makefiles for new code architecture. * Created a html utility lib for HTTP headers manipulations. * Extended the CHECK_HTTP and CHECK_SSL checkers to support remote webserver HTTP header status_code. HTTP status_code is parsed according to rfc2616.6.1. The keyword created for the new feature is "status_code" inside and "url" declaration. "status_code" feature can be mixed with "digest" feature. See the samples directory keepalived/samples/keepalived.conf.status_code for example. * Review the CHECK_HTTP and CHECK_SSL MD5SUM code to use a common stream handling function. * Matthijs van der Klip, <Matthijs.van.der.Klip@tech.omroep.nl> and I fixed a bug into the HTTP/SSL code that close the socket fd even if remote webserver has not been connected. As a result of fact, next socket created were imediatly closed. As a side effect, this altered the SMTP notification when remote webserver checked fall. No SMTP notification were sent if webserver were detected DOWN. Thanks to Matthijs for time debugging and investigation. * VRRP : Rewrote the previous Gratuitous ARP facility. Created a lib (vrrp_arp.c) dealing with PF_PACKET-SOCK_RAW-ETH_P_RARP and sockaddr_ll. * VRRP : Some cosmetics patch for messages logging. * VRRP : Fixed an issue during VRRP packet building, appending VRRP VIPs to the VRRP packet in the network order form. * VRRP : Reviewed the previous VRRP packet building process to not create the ARP header. Removec the previous hacky PF_PACKET-SOCK_PACKET-0x300 to use AF_INET-SOCK_RAW-PROTO to leave kernel appending ARP header since code doesn t currently support VRRP VMAC. * VRRP : Rewrote the previous vrrp_send_pkt() function to deal with sendmsg(). optimization lazzyness :) * VRRP : Extended the interfaces library to support common utility functions (if_setsockopt_hdrincl, if_setsockopt_bindtodevice, ...) * VRRP : Finally extend the code to support VRRP IPSEC-AH authentication method. Created a IPSEC-AH seq_number syncrhonization mecanism during VRRP MASTER/BACKUP elections. * VRRP : Extended the VRRP TSM to speed up instances syncrhonization during FAULT->BACKUP & FAULT->MASTER state transition. * Some cosmetics patches. This release is proposed as a 1.0.0 STABLE release candidate. 2002-09-17 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.7.1 released. * Fixed a MISC_CHECK issue when registering next timer checker. Must register a new timer thread before forking process. This imply for the user the extra script call must not execute in more than checker->vs->delay_loop. * Extented the ipfwwrapper (for LVS kernel 2.2) to not set ipchains rules if nat_mask is not specified in the configuration file. * VRRP : Added support to delayed gratuitous ARP send. When one instance enter to MASTER state a timer thread is registered. The default delay is 5secs. This delay is configurable per vrrp instance and handle the 'garp_master_delay' keyword. This delay refer to the delay after MASTER state transition we want to launch gratuitous ARP. * VRRP : Force health checker enable flag if VRRP framework is not selected. * VRRP : Review the gratuitous ARP helper function to only send gratuitous ARP if VRRP VIPs are set. * VRRP : Review the FSM to eliminate stalled flapping loop. The state transition diagram implemented is : +---------------+ +----------------| |----------------+ | | Fault | | | +------------>| |<------------+ | | | +---------------+ | | | | | | | | | V | | | | +---------------+ | | | | +--------->| |<---------+ | | | | | | Initialize | | | | | | | +-------| |-------+ | | | | | | | +---------------+ | | | | | | | | | | | | V | | V V | | V +---------------+ +---------------+ | |---------------------->| | | Master | | Backup | | |<----------------------| | +---------------+ +---------------+ The state DUMMY_MASTER state has been removed since it is a fake. * VRRP : In order to handle all possible state transition, a Transition State Matrix design (TSM) has been added. This matrix defines transition state handlers for VRRP sync group extension. The TSM implemented is (cf: vrrp_scheduler.c for more informations) : \ E | B | M | F | S \ | | | | ------+-----+-----+-----+ Legend: B | x 1 2 | B: VRRP BACKUP state ------+ | M: VRRP MASTER state M | 3 x 4 | F: VRRP FAULT state ------+ | S: VRRP start state (before transition) F | 5 6 x | E: VRRP end state (after transition) ------+-----------------+ [1..6]: Handler functions. * VRRP : Set ms_down_timer to 3 * advert_int + TIMER_SKEW when leaving MASTER state. * VRRP : In MASTER state, when incoming advert match or FAULT state is requested then force leaving MASTER state transition. (review the previous election approach). * VRRP : Optimized the leave FAULT state transition. Directly coded into the FSM for speed up recovery or code readability. * VRRP : Extended smtp notifier for BACKUP state. Review the MASTER state notification to only notify when VIPs are set. * some cosmetics patches. * Adam Fletcher, <adamf@rovia.com> created the 'Keepalived+LVS NAT HOWTO' 2002-08-05 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.6.10 released. * Fixed a faked flag during VRRP VIP set. Updated the IP address set flag to reflect netlink return code. * Fixed an autoconf issue during selection of VRRP framework. 2002-07-31 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.6.9 released. * Fixe some code dependence selection during compilation. If autoconf netlink probe fails then unset VRRP code. * Cleanup daemon lib. Added some logging info for the daemon processing, removed some repeated code part. * Added 2 new daemon arguments : --dont-release-vrrp : Dont remove VRRP VIPs on daemon stop --dont-release-ipvs : Dont remove IPVS topology on daemon stop * Review the global scheduling process to clear FD queues on master thread destroy. * Fixed a forking issue in the MISC_CHECK. * Review IPVS wrapper functions to use allocated IPVS rules instead of static referencing pointer. * Fixed the IPVS wrapper to delete IPVS entries according to their 'alive' state. * Added IPVS support to alive flag for VS entries. * Rewrote the previous main.c to support configuration reload on the fly. Extented signal handling to register a conf reload_thread on SIGHUP. The software design used here is a dynamic differential conf file reloading framework. This design offer key decision to add/remove new/old entries to/from low-level framework: IPVS topology and netlink IP addresses entries. This design reduce to the max the global service interruption since only negative diff entries are removed. For VRRP config reload on the fly, if you plan to add/remove many VIPs consider VIP declaration into the virtual_ipaddress_excluded since they are not present into VRRP adverts. * Review the keepalived.init script to support restart and reload arguments. * Fixed some typo issues. 2002-07-16 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.6.8 released. * Alex Kramarov, <alex@incredimail.com> & Remi Nivet, <Remi.Nivet@atosorigin.com> reported an assertion error during smtp notification process. The assertion caused a bad file descriptor registration during in_progress connection handling. Fixed registering an event thread calling upper level SMTP protocol in_progress connection handler. So the SMTP stream handlers use global I/O multiplexer on connection success. * Benoit Gaussen, <ben@trez42.net> and I added support to "inhibit" feature. Added a new keyword called "inhibit_on_failure" for real_server declaration. If specified the real_server will not be removed from the IPVS topology if real_server fail according to checker result. Instead of removing the entry from IPVS topology, the corresponding real_server weight will be set to 0. When real_server will be back, then weight will be set back to original value. See sample directory for example. * Added support to IP_MASQ_CMD_SET_DEST for 2.2 krnl and IP_VS_SO_SET_EDITDEST for 2.4 IPVS code to provide support to "inhibit" feature. * Review Makefile.in to exit on compilation error. * Extended autconf script to check for kernel netlink support. 2002-07-12 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.6.7 released. * Rewrote the previous SMTP notification framework. New code use a strong multi-threaded FSM design. * Moved the SMTP get_local_name() into utils.c * IPVS : updated the code to support IPVS_SVC_PERSISTENT_TIMEOUT. Introduced into the new libipvs coming with ipvs-1.0.4. * VRRP : Extended the mcast membership subscription to handle more robust mcast subscription errors. Removed the previous ugly stalling sleeping call retry for membership subscription. Membership subscriptions are now multi-threaded to not degrade global scheduling timer. * VRRP : Remi Nivet, <Remi.Nivet@atosorigin.com> pointed out a buffer overflow during the sending advert interface binding process. * Some more cosmetics patches. 2002-07-05 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.6.6 released. * added indentation style .indent.pro * Review the previous source tree. Splitted the code into functional subdirs. Added multi-level automake scripts. The source tree looks like : . |-- bin |-- genhash |-- keepalived | |-- core | |-- etc | | |-- init.d | | `-- keepalived | |-- healthcheck | |-- include | |-- libipfwc | |-- libipvs | |-- samples | `-- vrrp `-- lib * Refine autoconf/automake scripts. Added automake support to libipvs and libipfwc. Added code selection compilation for libipvs and libipfwc. * Review Makefile(s) to use more convenient facilities like distclean, ... * Review the Makefile(s) code dependencies. * Added support to modprobe_ipvs if the ip_vs.o module is not loaded. If modprobe fails then IPVS is assumed unavailable. * Refine the IPVS wrapper to be more tolerant. When a VS or RS is already configured don t stop the daemon. The daemon is stopped only on critical IPVS errors. * VRRP : Review the bootstrap sequence to start daemon even if one of the instance want to run on an interface administratively shut. Added extension to FSM to force transition to FAULT state during bootstrap if the interface is shut. * Updated the TODO file. * Some cosmetics patches. 2002-07-01 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.6.5 released. * Fixed a NULL pointer exception while releasing IPVS entries. * Review the Makefile.in to fixe some conventional issue. Fixed a libipvs dependance code selection. * Christophe Varoqui, <Christophe.Varoqui@free.fr> created the rpm spec file. * Roberto Nibali, <ratz@linux-vs.org> helped during OLS with code cleanup. Review the whole code coding style to use more conventional indentation. The one used into LVS and Kernel code. Coding style provided by the following command : find . -name "*.[chS]" -exec indent -kr -i8 -ts8 -sob -l80 -ss -bs -psl \ {} \; && find . -name "*~" -exec rm {} \; * Roberto Nibali and I review the DEBUG logging facility adding global DBG() func declaration. * Roberto Nibali fixed two potential buffer overflow (strcpy). * Richard L. Allbery, <rla@prideindustries.com> pointed out a fwmark issue. Healthcheckers is enabled if virtual service is a fwmark. * Some cosmetics patches. 2002-06-25 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.6.4 released. * Rewrote the previous ip address utilities functions. Review the string to ulong convertion function to support CIDR filtering and more simple handling ("without all hexadecimal and shorthand"), pickted from Paul Vixie code. * VRRP : extended the notify framework to support scripts inside a vrrp_sync_group. view the sample/keepalived.conf.vrrp.sync file. * VRRP : Review the previous vrrp_sync_group block. New declaration is : view the sample/keepalived.conf.vrrp.sync file. * VRRP : fixed a FSM sync_group side effect in FAULT state. * Fixed a Kernel 2.2 code selection issue (ETHTOOL). * Added support to wensong libipvs. * Fixed a sorry_server cleanup side effect. * Alex Kramarov, <alex@incredimail.com> fine the keepalived.init script to be compatible with redhat chkconfig. 2002-06-18 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.6.3 released. * VRRP : Christian Motelet, <cmotelet@canal-plus.com> pointed out a flapping issue when runing vrrp_sync_group on multiple NICs. This have been fixed adding leave FAULT state transition on both FSM state (read & read_to). The group leave fault state if all NIC of each VRRP Instance are functional. * Fixed some issue in the autoconf/automake scripts. 2002-06-16 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.6.2 released. * Andres Salomon, <dilinger@voxel.net> enhanced the autoconf/automake scripts to be more generic and to facilitate cross compilation. Including more efficient IPVS code detection, Kernel version, install script location, ... * Johannes Erdfelt, <johannes@erdfelt.com> fixed a genhash get request length calculation issue. * Johannes Erdfelt, <johannes@erdfelt.com> fixed a wrong printed IP address issue due to a static pivot buffer called multiple times for a single syslog call. * Johannes Erdfelt, <johannes@erdfelt.com> enhanced SMTP notification framework to use more compliant SMTP protocol handling. Enhanced both sending and receiving functions. A nice response code buffer handling calculating remote SMTP server retcode. * Johannes Erdfelt, <johannes@erdfelt.com> fixed a NULL pointer exception into the 2.2 ipvswrapper code. * Aneesh Kumar, <aneesh.kumar@digital.com> fixed a compilation issue for CI-LINUX checker compilation. * Jan Du Caju, <jan@kulnet.kuleuven.ac.be> fixed a compilation dependence selection into the VRRP framework when compiling without LVS support. This disable checkers activity update when compiled without LVS support. * fixed a dereferencing pointer into the parser. * move the dump configuration to printout conf after daemon initialization. * VRRP : Added support to start on complete init. VRRP framework and thus keepalived will start if VRRP instances are properly configured. 2002-06-13 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.6.1 released. * Aneesh Kumar, <aneesh.kumar@digital.com> and I added support to Cluster Infrastructure checkers. Providing HA-LVS for their cluster project (http://ci-linux.sourceforge.net/). The new checker added provide a derivation to the internal CI healthcheck mechanism. * Enhanced the Kernel netlink reflector to drive global healthcheckers activity. The policy implemented here is : If healthchecker is performing test on a service that belong to a VIP not owned by the director, then the healthchecker is suspended. This suspend/active state is particulary usefull if runing VRRP for HA => That way the backup LVS will not charge the realserver pool since LVS VIP is owned by master LVS. * Cosmetics patches into the vector lib. * VRRP : Rewrote the previous VRRP synchronization instance policy. Created a new config block called "vrrp_sync_group" that define VRRP instances synchronization dependences. That way we replace the previous "by-pair" sync approach by this "by-group" approach. This can be useefull for firewall HA with many NICs. Created a dedicated framework to speed up takeover synchronization. * VRRP : Added support to CIDR notation for VRRP VIPs definitions => VRRP VIPs definition like a.b.c.d/e. By default "e" value is set to 32. * VRRP : Added support to multicast source IP address selection => "mcast_src_ip" keyword. Can be usefull for strongly filtered env. The mcast group subscription is done using the NIC default IP after this mcast_src_ip is used if specified. * VRRP : Enhanced the link media failure detection. Added support to the new kernel SIOCETHTOOL probing for ETHTOOL_GLINK command. New drivers use this ETHTOOL interface to report link failure activity. During bootstrap a probe is done to determine the proper polling method to use for link media failure detection. The policy used is : probe for SIOCGMIIREG if not supported then try SIOCETHTOOL GLINK probe, otherwise use a ioctl SIOCGIFFLAGS polling function mirroring kernel NIC flags to localy reflected representation. * Ramon Kagan, <rkagan@YorkU.CA> and I updated the UserGuide.pdf. 2002-05-30 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.5.9 released. * Added support to realserver_group. The work is not yet finished since it introduces new compilation design currently not supported. So please do not use yet. * VRRP : Review the script notification. Moved to a script per VRRP instance state => Created new keywords notify_backup|master|fault to run a specific script during backup|master|fault state transition. * VRRP : Added support to quoted strings for notify_backup|master|fault. Can now launch script passing arguments. See sample directory for examples. * VRRP : Added a protocol extension called "virtual_ipaddress_excluded". This configuration block is similar to "virtual_ipaddress" block => those VIPs (called E-VIPs) are set throught netlink kernel channel and gratuitous arp are sent over each E-VIP. The only difference is that they are not added into VRRP packet adverts. This can be usefull for big env where you want to run many VRRP VIPs (200 for example). VRRP packet lenght are limited to a 20 VIPs, if you want more VRRP VIPs add them to the "virtual_ipaddress_excluded" configuration block. * VRRP : Added more logging facility when setting/removings VIPs & E-VIPs. * VRRP : Created a new FSM state called become_master in charge of VIPs/E-VIPs/notifications handling. The goto_master state is now a state where the instance send an advert to force a new MASTER election setting the instance into a transition mode. If election success its finaly transit to become_master state to own VIPs/E-VIPs and launch scripts. * VRRP : Force a new MASTER election when receiving a lower prio advert. * VRRP : Review the vrrp_scheduler.c to use more conventional FSM design. This reduce and beautifull the code. * VRRP : Fixed a very noisy flapping issue observed on heavy loaded env. Simulating big traffic on a backbone figure out this flapping issue. Added support to a TIMER_MICRO_ADJUST to prevent against timer degradation. This can be view as a DOS protection policy. VRRP MASTER timers are adjusted if they are too degradated, due to heavy loaded networking env introducing latency receiving/sending VRRP protocol adverts. Thanks goes to Paul, <xerox@foonet.net> for pointing it out and providing access to its Internet routing backbone. 2002-05-21 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.5.8 released. * Added an OpenSSL Licence exception to grant Keepalived compilation with OpenSSL Toolkit. Thanks to Andres Salomon, <dilinger@voxel.net> for suggesting. * Added connection port selection for Healthcheckers (TCP_CHECK, HTTP|SSL_GET). Can be usefull for Healthcheck in fwmark LVS topology for grouping service. Thanks to Richard L. Allbery, <rla@prideindustries.com> for suggection. See samples directory for examples. * Fixed some IPVS exclusion code when running --disable-lvs. * Added support to VirtualHost selection when using HTTP|SSL_GET. See samples directory for examples. * Added VirtualHost selection into the genhash utility. * Fixed some IPVS sync daemon initializations issues. * Cometics patches in IPVS wrapper framework. * Added support to quoted string. This can be usefull if you are using MISC_CHECK and you want to pass arguments to called script. See samples. * Prepare work on real_server_group in order to group some realserver declaration. * VRRP : Fixed a password length exception causing an unwanted dropping issue. * VRRP : Enhanced the MASTER state to send gratuitous arp if receiving a remote lower prio advert => This fix a remote stalled ARP cache. Thanks to Simon Kirby, <sim@netnation.com> for discussing this case. 2002-05-02 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.5.7 released. * Review autoconf/automake scripts to be more generic on system and code selection. Added primitives (configure) : --disable-lvs-syncd : Do not use IPVS sync daemon --disable-lvs : Do not use IPVS framework --disable-vrrp : Do not use VRRP framework --enable-debug : Compile with debugging flags * Fixed a SSL stream handling bug. Thanks to Andres Salomon, <dilinger@voxel.net> for pointing the issue. * Added a global memory counter to track global memory used. * Fixed configuration parser. read_line. Remove static allocated temporary read buffer. Only handle stream if line has been spitted into vector. * Limit maximum number of VIPs per VRRP Instance to 20. (for fragmentation, overhead, and others reasons). * Added IPVS wrapper support to persistence granularity. Thanks to Mike Zimmerman, <tarmon@spamcop.net> for the suggestion. * Review smtp notifier to handle VRRP MASTER state transition alert. Thanks to Paul, <xerox@foonet.net> for the suggestion. * Review the UserGuide.pdf to fixe some english issues :) Thanks to Jacques Thomas, <jacktom@noos.fr> for reviewing. 2002-04-13 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.5.6 released. * VRRP : Review in "GOTO_MASTER_STATE" the IP address handling. send protocol adverts before registering IP address to the interface. * VRRP : Review the "LEAVE_MASTER_STATE" to only handle state transition if wanted states are BACKUP or FAULT. * VRRP : Review the BACKUP state to force new protocol election if receiving a lower priority advert. * VRRP : Fixed a BACKUP to MASTER state transition only if interface is reported UP. * VRRP : Fake the "ipvs_syncd_cmd" function if running LVS using a Kernel 2.2. 2002-04-10 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.5.5 released. * Fixed a gratuitous ARP porting bug. * VRRP : Review the data structure to be more generic and clean with the rest of the code. * VRRP : Remove the interface flags (NIC) ioctl functions * VRRP : Created an interface (NIC) library giving access to common interface helpers functions. * VRRP : Created an interface lookup function creating a global interface structure during daemon bootstrap. Consist of a netlink RTM_GETLINK & RTM_GETADDR lookup, so we can work with a userspace interface representation. * VRRP : Create a netlink kernel reflection framework updating dynamically our interface structure according to kernel netlink broadcast. This design is highly inspired from zebra. => Reflection mean : wait for netlink kernel broadcast, if received, wakeup netlink filter to update our userspace representation. Prefer this design instead of a delayed netlink poller. That way we reduce global overhead. * VRRP : VRRP need to detect failure from many places. If netlink can notify for many troubles like mainly IFF_UP|DOWN & IFF_RUNNING, those flags are kernel drivers dependent. To reduce takeover time and performance we need to have informations like : Does the media link is present ?. The fact is that most of the new NICs own embended hardware chip providing such informations. So created a MII transceiver status register thread poller. Monitoring Basic Mode Status Register (BMSR) of the MII status words. Waiting for kernel NIC drivers hackers to support this functionnality through netlink (=> Like a IFF_RUNNING update broadcast). * VRRP : Linked the state machine to the global interface structure. NIC failure/events are handled. * VRRP : Review the whole state machine code to be more realistic. The State transition diagram described into the RFC2338 is an obtimist view. The VRRP state transition diagram implemented here is : +---------------+ +--------->| |<-------------+ | | Initialize | | | +------| |----------+ | | | +---------------+ | | | V V | +---------------+ +---------------+ | |---------------------->| | | Master | | Backup | | |<----------------------| | +---------------+ +---------------+ ^ | | | ^ | | | +---------------+ | | | | +------>| Dummy Master | | | | | +---------------+ | | | | | | | | | V | | | | +---------------+ | | | +------------>| |<----------+ | | | Fault | | +-----------------| |----------------+ +---------------+ * VRRP : Robust multicast handling. Something really strange is : after a NIC failure (in fallback mode) without closing the socket, multicast advert can be sent but not received ? really strange don t know why probably an IGMP resubmit ?. So multicast group is left during failover (media trouble, IFF_DOWN or !IFF_RUNNING). In fallback, we register a new membership and synchronize all the packet dispatcher fds. * VRRP : Fixed a checksum trouble using password authentication. * VRRP : Added support to the LVS sync daemon. This permit LVS sync daemon to be state drived by a specific VRRP instance. * Review the autoconf/automake to be more generic. * Some cosmetics patches. 2002-02-25 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.5.3 released. * Added autoconf / automake generic scripts. * Rewrite the configuration file stream parser. Using a generic keywords tree. Each keyword refer a specific stream handler. The main stream processor is a multilevel recursive function getting file stream and backtracking the keyword tree. Kind of global compiler structure using event driven stream processing. * Re-design the global data structure to be much more generic and to dissociate LVS configuration related to checkers related. Remove static char lenght to use dynamic length strings. * Created a global timer framework. * Created a global vector template, used in cofiguration file parsing (both stream process & keywords tree generation). * Created a global list template, used in most of the code. * Review the global scheduler to remove repeated code. * Created a global checkers API. The design and goal here is to facilitate new checkers creation by localizing specific checker code into a single file without any other global framework integration. * Patched a SSL stream handling race condition finding end of stream. * Jan Holmberg, review MISC checker to use forked process to not degrade global scheduler timer. * Revisited the whole code to use new templates structures. * Fixed a url lentgh bug into the genhash utility. * Fabrice Bucher, <fabrice.bucher@urbanet.ch> fixed a timeout_persistence bug in the IPVS wrapper code. * Bradley McLean, <bradlist@bradm.net> added support to '0' port number service in VS manipulation. Useful for balancing all services (host rather than service). * Matthijs van der Klip, <matthijs.van.der.klip@nos.nl> enhanced smtp framework to use SMTP header and email enclosed with angle brackets. 2001-12-20 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.4.9a released. * Jan and I patched a memory pointer problems in vrrp_scheduler.c Thanks to Negrea Mihai, <mike@umft.ro> for reporting. * Jan Holmberg, patched a memory reallocation pointer exception in memory management framework. * Jan Holmberg, patched a vrrp vip set/remove retry. * Some cosmetics/logging patches. * Created Keepalived UserGuide. 2001-12-10 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.4.9 released. * Jan Holmberg, <jan@artech.net> added a memory managment framework. In debug mode it is used as a memory leak buster. We can so use it to debug quickly memory leaks (buffer overrun, allocation errors, ...). * Jan Holmberg and I added support to SSL. Checker SSL_GET. Can be used with autogenerated cert or with specific cafile, certfile, keyfile. * Use the OpenSSL, <www.openssl.org> library for MD5 & SSL functions. * Jan Holmberg and I Rewrote the HTTP_GET code to use full asynchronous stream handling. The code use a common part for HTTP/SSL stream handling. Review the MD5 digest buffer computation, update MD5 over received buffer. * Patched some memory leaks in smtp handling. * Jan Holmbarg added support to LVS FWMARK. * Added command line option for keepalived. Used the libpopt library. -h, -v, -n, -d, -l, -f. * Jan Holmberg and I added debugging facility on keepalived console. * Added a BOOTSTRAP_DELAY of 1sec when registering checkers during daemon bootstrap. * VRRP : Jan Holmberg added possibility to run an extra script when VRRP Instance become or leave MASTER STATE (=> using a forked process). * Review/fine the whole code to apply cosmetics patch. * Rewrote the genhash utility. * Started checkers API specs. * doc doc doc... 2001-11-20 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.4.8 released. * Rewrite the whole VRRP previous code. * VRRP : Created a hierarchic scheduling framework. Handle VRRP instances multiplexing on the same I/O fd. VRRP I/O events are handled by our global scheduling framework. Then the global sheduling framework call a VRRP I/O instance dispatcher to manage VRRP instances. * VRRP : Created a temporary socket pool to handle register our VRRP thread instances. We create & allocate a socket pool here. The soft design can be sum up by the following sketch : fd1 fd2 fd3 fd4 fdi fdi+1 -----\__/--------\__/---........---\__/--- | ETH0 | | ETH1 | | ETHn | +------+ +------+ +------+ Here we have n physical NIC. Each NIC own a maximum of 2 fds. (one for VRRP the other for IPSEC_AH). All our VRRP instances are multiplexed through this fds. So our design can handle 2*n multiplexing points. * VRRP : Review the multicast socket creating. We bind the socket to a specific NIC. inbound & outbound traffic are bound to the NIC. => why IP_ADD_MEMBERSHIP & IP_MULTICAST_IF doesnt set sk->bound_dev_if themself ??? !!! Needed for filter multicasted advert per interface. => For inbound binding we use SO_BINDTODEVICE kernel option. * VRRP : Created a read dispatcher thread to deal with our sockpool. Handle VRRP states & transition states. * VRRP : Created a VRRP synchronization instance circuit. This functionnality gave you the ability to monitor VRRP instance each other. This mean that if 2 VRRP instances are monitoring themself and if one of this instance change state, the other follow the same state. ex.: With 2 VRRP instances (VI_1 & VI_2) if VI_1 become backup then VI_2 become backup too. (symetricly with master VRRP state). * VRRP : Rewrite the netlink interface to use non blocking socket. * VRRP : Rewrite the ipaddress handling to use the new netlink interface. * VRRP : Remove the VRPP VMAC handling since linux kernel only permit to use one MAC address on a specific NIC. We use gratuitous arp when setting up VRRP VIP, to uptade remote host arp caches. => In certain case this can cause a TCP session renegociation which can cause a permature session end. => To be fully compliant with the VRRP RFC, need to patch the kernel to gave it the possibility to deal with more than one MAC address at a time. Give me clue on it please ! to same me a little time :) * Starting VRRP documentation. * Patch a pidfile handling bug when forking the keepalived daemon. Thanks goes to Gianni D'Aprile for pointing it to me. * Patch a timer race condition into the scheduling framework. This bug caused tcpcheck to respawn quickly... Thanks goes to Gianni D'Aprile for pointing it to me. 2001-11-04 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.3.8 released. * Added support to native IPTABLE LVS CODE => using NAT on 2.4 kernel ipchains kernel support has been removed. * Added support to Direct Routing & Tunneling. * Review the keepalived.init script to be much more generic. 2001-09-14 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.4.1 released. * Added support to LVS kernel 2.4 code 2001-08-23 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.4.0 released. * Patch a race condition into the scheduler timer computation. * Patch a race condition into the tcp checker thread. Only register next timer thread if tcp connection is not in progress. * Patch a race condition into the http checker thread. Handle empty buffer returned from remote http server. * Patch a race condition into the dumping configuration process. A simple dereferencing pointer value...oops... * Eric Jarman, <ehj38230@cmsu2.cmsu.edu> added MISC CHECKER. It Perform a system call to run an extra system or script. => security auditing needed for system call, buffer overflow over script path must be handled. * Added VRRP support using our scheduling I/O multiplexer. VRRP implementation support to IPSEC-AH using HMAC-96bits digest with anti-replay. rfc2402 & rfc2104. * Added routing table fetcher. We ignore route when it is a cloned route from other router, learn by an ICMP redirect or set by kernel. Only UNICAST route are stored. * Added dropping packet support. 2001-07-15 Alexandre Cassen <acassen@linux-vs.org> * keepalived-0.3.5 released. * Rewrite the whole signal handling, registering a terminating thread on signal. * Move logsystem to syslog using facility LOG_INFO & LOG_DEBUG. * Added a daemonization function imported from zebra. * Rewrite the pidfile handling, check if daemon is running, if not remove eventual stalled pidfile and create new pidfile. * Added a strong scheduling framework based on an I/O multiplexer to handle asynchronous process. This code is imported from zebra and have been enhanced for keepalived purposes. Thread types are : . timeouted read on fd. . timeouted write on fd. . timer. . event. . terminate event. => The zebra framework have been enhanced to add support for timeouted read/write fds. => With this framework keepalived use a Boss/Worker thread model design, fetching ready thread from a master threading queues. * Rewrite the configuration file reader to add flexibility on extending. The dynamic data structure has been rewritten to use apropriate types. Right now parsing framework is ready for easy new checker structures integration. * Rewrite the smtp connector. The implementation take advantage of the I/O multiplexer. All read/write operations from/to the remote smtp server are done asynchronously. The implementation is rfc 821 compliant (multiple receiver are handled by a multiple RCPT TO command as specified in rfc821.3.1). * Rewrite the IPFW & IPVS wrappers. * Added support for NAT mask on IP MASQ rules (keyword nat_mask in configuration file). Added support for sorry server facility, so when all the server from a VS server pool are removed, a sorry server is automaticaly added to the VS pool (typically this is used when you have a spare server online). * Rewrite the previous checkers. Checkers are now based on a hierarchic layer stack framework. The protocol implemented for the moment is TCP. All layer 5 checkers are using layer4.c primitives with the same design : . a checker connector thread (creating the socket) registering the connection checker thread. . a connection checker thread testing connection states (error, in_progress, timeout, success). When connection success upper level thread are registered to handle checks. * Delay loop is now checkers specifics since we can use a multithreaded framework. * Update the PDF documentation file.