This file lists noteworthy changes in MooseFS.

* MooseFS 3.0.101-1 (2018-07-12)

  - (master) make under goal replications respect topology (patch made by tclh123)
  - (master) fix inforec undergoal (patch made by tclh123)
  - (cgiserv) fixed handling connection close during data sending (leaving active pipes which caused leaving mfs.cgi subprocesses)
  - (mount) decreased memory usage (fixed handling buffers allocated per thread)
  - (all) setting correct errno when read returns 0 (in sockets.c)
  - (mount) fixed double free of plock_data structure
  - (master) fixed inconsistency in calculating realsize when storage class definition has changed
  - (master+mount) fixed handling read-only filesystems (ro flag in exports and lookup-open lflags)
  - (all) fixed differences in defaults between source,config and manpages
  - (all) added 'nobreak' macro to explicitly inform compiler that intentionally there is no 'break' before following 'case'
  - (mount) added 'mfssuid' and 'mfsdev' options to mount (allows mounting with suid/dev options using system mount on Linux)
  - (cs) added using posix_fadvise(...DONTNEED) for tested chunks
  - (master) fixed HA bug (uninitialized variable might cause everlasting ELECT - PRO only)
  - (cs) fixed metaid handling
  - (master) added clearing SNAPHSOT eattr on file/directory modifications
  - (master) added optional same rack preference for destination server during undergoal replications
  - (mount+tools) increased masterproxy timeout from 1s to 20s

* MooseFS 3.0.100-1 (2018-01-24)

  - (cs) fixed rare segfault during chunk sending to master after disconection
  - (mount) added thread for cleaning released files with delay
  - (mount) added assertion for lru cache (sustained open files)
  - (master+mount) added chunks data cache invalidation after chunksever disconnection and reconnection
  - (mount) changed (lowered) number of connection retries (depends on I/O retry counter)

* MooseFS 3.0.99-1 (2017-11-23)

  - (master) disconnect all clients when exports has changed
  - (mount) do not show message about 'negative travel time' when the measured time is zero (happens in virtual machines)
  - (all) fixed get16bit function on big-endian machines

* MooseFS 3.0.98-1 (2017-10-12)

  - (mount) changed default cache mode on OS X from 'direct' to 'auto' due to problems with mmap
  - (cgiserv) added seteuid to 'mfs' or 'nobody'
  - (cs+tools) use readdir instead of readdir_r on glibc 2.24+
  - (debian) added support for systemd in debian packages
  - (all) fixed library dependencies (for libpcap,libz and libm)
  - (mount) fixed race between defered 'open' and 'locks'
  - (master) more chunk debugs and better handling chunk status change
  - (cs) fixed handling mark for removal (after reload sometimes cs could send chunks marked for removal as normal chunks)
  - (master) added file with metadata checksums (for future use)
  - (master) fixed way of sending gids to changelog/follower
  - (master) added inode checksum for some changes sent to changelog/follower
  - (metadump) added '0x' prefix for fileds dumped as hex numbers
  - (master) fixed ACL mask synchronization issues (restoring from changelog/leader->follower)

* MooseFS 3.0.97-1 (2017-08-01)

  - (tools) fixed trash/sustained size desynchronization after file length changed by chunk creation
  - (mount) fixed handling invalid arguments in posix lock command

* MooseFS 3.0.96-1 (2017-07-25)

  - (tools) fixed segfault in mfsscadmin

* MooseFS 3.0.95-1 (2017-07-17)

  - (mount+master) added update master version during reconnection to master (after updating master server)
  - (master) fixed winattr field size in setattr command
  - (mount) fixed dirattrcache (freezes or inefficiences during listing directories - introduced in 3.0.93)
  - (cgi) fixed displaying master charts without leader (pro only)
  - (mount) added info about using open dir cache in oplog/ophisotry

* MooseFS 3.0.94-1 (2017-06-22)

  - (mount) fixed support for CREATE/REPLACE flags in setxattr
  - (mount) added headers for flock defines
  - (all) added check for poll.h header file and use it instead of sys/poll.h if possible
  - (master) added test for WRITE access on directory during moving between different parents
  - (master) added clearing suig/sgid during write
  - (master+mount) increased max symlink path size to 4096 (posix compliance)

* MooseFS 3.0.93-1 (2017-06-01)

  - (master) redesigned xattr storage (much faster and uses less memory)
  - (master) improved hash map in xattr and acl (static -> dynamic)
  - (cgi+cli) added possibility to define master group as set of IP adresses divided by comma or semicolon
  - (cs) fixed using fsync on closed descriptors (after change FSYNC_BEFORE_CLOSE option)
  - (master) fixed removing default acl
  - (master+mount) added support for basic Windows attributes (hidden,ro,system,archive)
  - (cgi+cli) fixed working with followers only
  - (all) added api for reading config parameters from master and chunkserver
  - (tools) increased timeout from 10 seconds to 20 seconds (needed for huge snapshots)

* MooseFS 3.0.92-1 (2017-04-27)

  - (master+tools) added chunk slices to mfsappendchunks
  - (tools) added archive mode tools
  - (master+mount) fixed getfacl (unnecessary check for read rights for uid/gid)
  - (master) fixed changing acl mask during setattr

* MooseFS 3.0.91-1 (2017-04-07)

  - (all) silence false warnings generated by static analyzers (clang and cppcheck)
  - (master) fixed quota testing routine used during 'move/rename'
  - (all) added README.md to distribution
  - (cs+mount) decreased delay in conncache
  - (mount) fixed reading done by many (20+) threads using one descriptor (premature removing requests from queue during cleaning)

* MooseFS 3.0.90-1 (2017-03-15)

  - (master) fixed 'move/rename' with quota
  - (master) fixed chunk counters shown in cli/cgi
  - (master+tools) added option 'preserve hardlinks' to mfsmakesnapshot
  - (master) fixed acl copying during mfsmakesnapshot

* MooseFS 3.0.89-1 (2017-02-21)

  - (mount) added fixing file length in all inodes after write
  - (cs) split bitfiled into separate bytes (fixed potential race condition)
  - (master) setting operation to NONE before sending status (silence unnecessary messages in some cases)
  - (cs) increased verbosity of crc-error messages
  - (cs) fixed invalidating preserved block in case of truncate down followed by truncate up
  - (mount) increased master-proxy buffer sizes

* MooseFS 3.0.88-1 (2017-02-08)

  - (mount) added read cache clean on write (same file access using different descriptors)
  - (mount) added missing cond_destroy in readdata.c (fix sent by Jakub Ratajczak)
  - (master) fixed initializing packet size for reading 'sustained' directory
  - (all) fixed zassert for printing correct statuses in case of pthread functions

* MooseFS 3.0.87-1 (2017-02-01)

  - (mount) fix fleng in finfo after truncate (patched by Davies Liu)
  - (mount) fix overlapped read (patched by Davies Liu)
  - (mount) fixed invalidating chunk cache after truncate
  - (mount) fixed fleng handling in read worker
  - (mount) fixed handling BREAK state in read worker
  - (mount) changed handling data invalidation in read module (sometimes could be less efficient, but it is much more safer)
  - (tools) fixed number parsing (patched by Pawel Gawronski)
  - (cli) fixed printed host/port options
  - (mount) moved pipes from requests to workers (read and write - huge decrease of descriptors used by mount)
  - (mount) changed signal to broadcast in rwlock (fixed very rare read/write deadlock)
  - (mount) fixed 'open descriptors' leak (lookup(with data for open)->open(with cached data)->close)
  - (mount) fixed potential 'race condition' - free 'csdata' during access
  - (master) split (only internally) sustained folder into 256 subfolders (too many sustained files caused socket timeouts in master)

* MooseFS 3.0.86-1 (2016-11-30)

  - (master) fixed leader-follower resynchronization after reconnection (pro only)
  - (masrer) fixed rehashing condition in edge/node/chunk hashmaps (change inspired by yinjiawind)

* MooseFS 3.0.85-1 (2016-11-17)

  - (mount) fixed memory leak (also efficiency) on Linux and potential segfaults on FreeBSD (negative condition)
  - (mount) fixed race condition for inode value in write module
  - (mount) better descriptors handling (lists of free elements, inode hash)
  - (mount) better releasing descriptors on FreeBSD
  - (cs) fixed time condition (patch send by yinjiawind)

* MooseFS 3.0.84-1 (2016-10-06)

  - (master) fixed setting acl-default without named users or named groups
  - (master) fixed master-follower synchronization after setting storage class

* MooseFS 3.0.83-1 (2016-09-30)

  - (cs) changed header size from 5k to 8k (due to 4k-sector hard disks)

* MooseFS 3.0.82-1 (2016-09-28)

  - (all) silenced message about using deprecated parameter 'oom_adj'
  - (mount) fixed FreeBSD delayed release mechanism
  - (mount) added rwlock for chunks

* MooseFS 3.0.81-1 (2016-07-25)

  - (mount) fixed oom killer disabling (setting oom_adj and oom_score_adj)
  - (cli) fixed displaying inactive mounts
  - (mount) added fsync before removing any locks
  - (daemons) added disabling oom killer (Linux only)
  - (all) small fixes in manpages
  - (mount) fixed handling nonblocking lock commands (unlock and try) in both locking mechanisms
  - (daemons+mount) changed default settings for limiting malloc arenas (Linux only)

* MooseFS 3.0.80-1 (2016-07-13)

  - (master) fixed chunk loop (in some cases chunks from the last hash position might be left unchecked)
  - (master) fixed storage class management (fixed has_***_labels fields)

* MooseFS 3.0.79-1 (2016-07-05)

  - (master) fixed 'flock' (change type of lock SH->EX and EX->SH caused access to freed memory and usually SEGV)

* MooseFS 3.0.78-1 (2016-06-14)

  - (cs) fixed serious error that may cause data corruption during internal rebalance (intr. in 3.0.75)

* MooseFS 3.0.77-1 (2016-06-07)

  - (mount) added assertions after packet allocation in master communication module
  - (manpages) fixes related to storage classes
  - (all) improved error messages used for storage classes

* MooseFS 3.0.76-1 (2016-06-03)

  - (master) fixed resolving multi path for root inode
  - (master) fixed licence expiration behaviour (pro only)

* MooseFS 3.0.75-1 (2016-04-20)

  - (all) fixed cppcheck warnings/errors (mostly false negatives)
  - (cs) added file sparsification during chunk write (also chunk replication, chunk duplication etc.)
  - (mount) fixed write data inefficiency when I/O was performed by root
  - (master) fixed removing too early locked chunks from data structures
  - (tools) fixed reusing local ports (connection to master returned EADDRNOTAVAIL)
  - (all) changed default action from restart to start
  - (all) added exiting when user defined config (option -c) cannot be loaded
  - (cs) changed handling chunk filename (dynamic filename generation - cs should use less memory)
  - (netdump) changed 'moose ports only' option to 'port range'
  - (mount) limited number of files kept as open after close
  - (cs) changed subfolder choosing algorithm
  - (mount) changed mutex lock to rw-lock for I/O on the same descriptor
  - (mount) added link to mfsmount from '/sbin/mount.moosefs' (Linux only)
  - (all) introduced "storage classes" (new goal/labels management)
  - (master+cs) introduced 'temporary maintenance mode' (automatic maintenance mode after graceful stop of cs)
  - (master+mount) added fix for bug in FreeBSD kernel (kernel sends truncate before first close - FreeBSD only)
  - (cs) fixed setting malloc pools on Linux

* MooseFS 3.0.74-1 (2016-03-08)

  - (master) fixed rebalance replication (check for all chunk copies for destination - not only valid)
  - (master+mount) new mechanism for atime+mtime setting during I/O
  - (master+mount) new I/O synchronization between different mounts (with cache invalidation)
  - (master+mount) new chunk number/version cache (with automatic invalidation from master)
  - (master) added mapping chunkserver IP classes (allow to have separate network for I/O and separate for other activity)
  - (master) fixed status returned by writechunk after network down/up
  - (master) changed trashtime from seconds to hours
  - (master) added METADATA_SAVE_FREQ option (allow to save metadata less frequently than every hour)
  - (master) added using in emergency (endangered chunks) all available servers for replication (even overloaded and being maintained)
  - (master) added using chunkservers in 'internal rebalance' state in case of deleting chunks
  - (all) spell check errors fixed (patch contributed by Dmitry Smirnov)

* MooseFS 3.0.73-1 (2016-02-11)

  - (master) fixed restoring ARCHCHG from changelog
  - (cli+cgi) fixed displaying master list with followers only (pro version only)
  - (master) added using size and length quota to fix disk usage values (statfs)
  - (master) fixed xattr bug which may lead to data corruption and segfaults (intr. in 2.1.1)
  - (master) added 'node_info' packet
  - (tools) added '-p' option to 'mfsdirinfo' - 'precise mode'
  - (master) fixed edge renumeration
  - (master) added detecting of wrong edge numbers and force renumeration in such case

* MooseFS 3.0.72-1 (2016-02-04)

  - (master+cgi+cli) added global 'umask' option to exports.cfg
  - (all) changed address of FSF in GPL licence text
  - (debian) removed obsolete conffiles
  - (debian) fixed copyright file
  - (mount) fixed parsing mfsmount.cfg (system options like nodev,noexec etc. were omitted)
  - (master) changed way how cs internal rebalance state is treated by master (as 'grace' state instead of 'overloaded')
  - (mount) fixed bug in read module (setting etab after ranges realloc)
  - (tools) removed obsoleted command 'mfssnapshot'

* MooseFS 3.0.71-1 (2016-01-21)

  - (master) fixed emptying trash issue (intr. in 3.0.64)
  - (master) fixed possible segfault in chunkservers database (intr. in 3.0.67)
  - (master) changed trash part choice from nondeterministic to deterministic

* MooseFS 3.0.70-1 (2016-01-19)

  - (cgi+cli) fixed displaying info when there are no active masters (intr. in 3.0.67)
  - (mount+common) refactoring code to be Windows ready
  - (mount) added option 'mfsflattrash' (makes trash look like before version 3.0.64)
  - (mount) added fixes for NetBSD (patch contributed by Tom Ivar Helbekkmo)

* MooseFS 3.0.69-1 (2016-01-12)

  - (mount) fixed rare case when request memory was used after being freed.

* MooseFS 3.0.68-1 (2016-01-08)

  - (cgi+cli) fixed UNKNOWN state for "marked for removal" readiness
  - (mount) added protection from releasing inode memory too early in reading module
  - (mount) use direct I/O as a default mode on Mac OS X (due to keep_cache bug in kernel/fuse)

* MooseFS 3.0.67-1 (2016-01-07)

  - (cgi+cli) added much more info for master in ELECT state (pro version only)
  - (master+cgi+cli) added temporarily removing chunkservers in master in ELECT state (pro version only)
  - (cgi) fixed bug in ChunkServers list (heavy load/internal rebalance)

* MooseFS 3.0.66-1 (2016-01-04)

  - (mount) use direct I/O as a default mode on FreeBSD (due to keep_cache bug in kernel/fuse)

* MooseFS 3.0.65-1 (2015-12-23)

  - (master+cs) added minimal version for master supervisors in chunkservers (pro version only)
  - (cs) chunkserver can be started with empty mfshdd.cfg and work as a voter (pro version only)
  - (master) mfsrmsnapshot will remove files without using trash
  - (master+cs) added optional authorization

* MooseFS 3.0.64-1 (2015-12-21)

  - (master+mount) split trash into 4096 separate sub-trashes

* MooseFS 3.0.63-1 (2015-12-17)

  - (mount) fixed rare race condition in read (not dangerous)
  - (mount) better read/write synchronization in read
  - (master) new chart (used/total space)

* MooseFS 3.0.62-1 (2015-12-11)

  - (cs) added ability to start with metaid read from connected hard drives (from .metaid file)
  - (cs) added protection from using disks filled in more than 99.9% when there are other disks
  - (master+cs+cli+cgi) added 'rebalance in progress' state to chunkservers (treated as heavy load state)
  - (master) added ATIME_MODE option to set atime modification behaviour

* MooseFS 3.0.61-1 (2015-11-30)

  - (master) fixed lookup in case of missing chunks (lookup returned ENXIO in such case)
  - (mount) added mfstimeout option to force timeout for all I/O operations
  - (autotools) fixed problem with 'undefined reference to rpl_malloc/rpl_realloc'

* MooseFS 3.0.60-1 (2015-11-23)

  - (systemd) added TimeoutStartSec=1800 to master
  - (master) fixed "parse error" message for broken network changelogs

* MooseFS 3.0.59-1 (2015-11-06)

  - (all) fixed debug symbols
  - (master+supervisor) added metaid check (pro version only)
  - (master) added rejection of followers with incorrect meta version (pro version only)
  - (mount) change type of data stored in kernel from pointers to indexes (more robust)
  - (cli) fixed show exports in plain mode
  - (master) added sending metaid after switching from ELECT to LEADER (pro version only)
  - (master) removed option '-e' from GPL edition (only makes sense in pro version)
  - (master+cs) added sending metaid to cs in ELECT state (pro version only)
  - (master) improved metaid generation method
  - (mount) added mfsoomdisable option (Linux only, turned on by default)
  - (mount) added minimum retry counter to log messages in I/O modules
  - (mount) reduced memory consumption by reducing thread stack sizes
  - (mount) changed default malloc arena count to 2 (Linux only)

* MooseFS 3.0.58-1 (2015-10-30)

  - (mount) added condition for requests in read data (request should begin before EOF)
  - (systemd) fixed typo in mfscgiserv service file

* MooseFS 3.0.57-1 (2015-10-27)

  - (metalogger) added 1 second timeout when connecting to master
  - (macosx) fixed packages to be compatible with OS X 10.11+

* MooseFS 3.0.56-1 (2015-10-26)

  - (mount) fixed reading scenario: (read from empty chunk -> write chunk -> read this chunk again)

* MooseFS 3.0.55-1 (2015-10-20)

  - (cs,master) added 1 second timeout when connecting to master

* MooseFS 3.0.54-1 (2015-10-16)

  - (master) fixed setting version of new chunks registered as 'marked for removal'
  - (master) added stronger condition for deleting invalid chunks
  - (cs) changed condition for number of blocks to change to mark disk as damaged (allow changes up to 10%)

* MooseFS 3.0.53-1 (2015-10-13)

  - (cs) fixed typo (cnunk)
  - (mount) create in deleted directory returns EACCES only in OS X (ENOENT in other systems)

* MooseFS 3.0.52-1 (2015-10-09)

  - (mount) added new mechanism for sustaining working directories (replaces mechanism added in 3.0.40)
  - (cs) force disconnection from master couple seconds after term signal (frozen I/O threads can prevent CS from termination)
  - (cs) when RO/RW status or total blocks changes then device is automatically marked as damaged
  - (master) added support for root inode end deleted inodes in MASS_RESOLVE_PATHS
  - (cli) fixed error displaying disconnected chunkservers
  - (rpm) added network-online.target to Wants and After in systemd service files (startup issues after reboot)

* MooseFS 3.0.51-1 (2015-09-24)

  - (mount) fixed segfault during umount
  - (mount) changed fsync before close to be turned on by default
  - (mount) fixed read-write-read scenario on files created as a snapshots

* MooseFS 3.0.50-1 (2015-09-22)

  - (master+mount) for files with EATTR_NODATACACHE mounts will always use direct io
  - (all) changed suffix of sample configuration files from '.dist' to '.sample'

* MooseFS 3.0.49-1 (2015-09-18)

  - (mount) added DIRECT as a cachemode (force using direct io - also disables cache - mainly for FreeBSD)

* MooseFS 3.0.48-1 (2015-09-17)

  - (mount) minor changes in mastercomm module (disconnection handle)

* MooseFS 3.0.47-1 (2015-09-16)

  - (mount) fixed memory leak in readchunkdata module

* MooseFS 3.0.46-1 (2015-09-15)

  - (mount) fixed chunk-data cache bug (repeated network timeouts could lead to reading freeze)

* MooseFS 3.0.45-1 (2015-09-10)

  - (mount) fixed cache for extra inode data (usage of non thread-safe allocation functions - leads to hangups on lookup/open etc.)
  - (mount) fixed data-cache issue (delete only directories from kerenel dentry cache)
  - (mount) inserting into xattr cache "nonexistent" xattr "security.capability" after file creation. (speed up writing small files)
  - (master) fixed scenario causing deleting chunks from chunkservers marked for removal
  - (cgi+cli) fixed color of reported exports checksum for non-pro masters

* MooseFS 3.0.44-1 (2015-08-13)

  - (master+mount) fixed synchronization between mounts (write+close on one and then read this data on another)
  - (mount) fixed chunk localization cache (snapshot case - on snapshot chunkid may change)

* MooseFS 3.0.43-1 (2015-08-10)

  - (netdump) added support for device 'any' with LINUX_SLL and PKTAP frames

* MooseFS 3.0.42-1 (2015-08-07)

  - (mount) added support for negative answers in 'removexattr' from xattr cache

* MooseFS 3.0.41-1 (2015-08-06)

  - (master) added creating first chunk after file creation

* MooseFS 3.0.40-1 (2015-08-05)

  - (master+mount) added using fuse's forgets in inode number reusage (fixes very rare EBUSY case when inode number is reused after unlink of directory which is still open or used as a current working directory)
  - (master+mount) sped up reading of small files
  - (mount) increased chunk location cache (sped up reading files)

* MooseFS 3.0.39-1 (2015-07-23)

  - (master) fixed truncate bug (wrong behaviour when chunk was locked - not dangerous)
  - (cgi+cli) fixed python3 incompatibilities

* MooseFS 3.0.38-1 (2015-07-13)

  - (master+cgi+cli) added check for servers with disks marked for removal (if disks can be safely removed)

* MooseFS 3.0.37-1 (2015-07-08)

  - (mount) performance fixes in read module

* MooseFS 3.0.36-1 (2015-07-07)

  - (mount) fixed bug in read module (intr. in 3.0.35)

* MooseFS 3.0.35-1 (2015-07-06)

  - (mount) fixed state automaton in read module (in very rare cases zeros might be read instead of data - intr. in 3.0.23)

* MooseFS 3.0.34-1 (2015-06-26)

  - (master) fixed small follower-desync issue (pro version only - intr. in 3.0.33)
  - (master) fixed session restore/synchronization issue (bad info length after metadata restore)
  - (mount) fixed setting maxfleng on truncate

* MooseFS 3.0.33-1 (2015-06-25)

  - (master) added metaid synchronization between masters (pro version only)
  - (cgi+cli) fixed "Metadata Info" table (intr. in 3.0.32).

* MooseFS 3.0.32-1 (2015-06-23)

  - (master) added exports checksum
  - (master) changed sessions parameters refresh policy (refresh all parameters after 'exports' reload)
  - (master) fixed truncate bug (close file without writing data may cause file truncate - dangerous !!!, introduced in 3.0.25)

* MooseFS 3.0.31-1 (2015-06-22)

  - (master) changed metadata storing policy - when there are no followers, then store metadata every hour (pro version only)

* MooseFS 3.0.30-1 (2015-06-19)

  - (master) added 'empty' chunk count to mfscheckfile (fixed mfsfileinfo on sparse files)
  - (master) added 'mfsrmsnapshot' (remove whole directory created as a result of mfsmakesnapshot)
  - (tools) fixed memory overrun error during mfssetgoal
  - (all) improved reloading cfg files (commented out options should be treated the same as options set to default values)

* MooseFS 3.0.29-1 (2015-06-18)

  - (master) fixed truncate bug (freezing mount during truncate in some rare scenarios)

* MooseFS 3.0.28-1 (2015-06-17)

  - (master) fixed lock issue on empty (chunkid==0) chunk

* MooseFS 3.0.27-1 (2015-06-16)

  - (master) fixed memory allocation size bug in chunk's "ftab"

* MooseFS 3.0.26-1 (2015-06-15)

  - (master+tools) added reportig invalid chunks to mfsfileinfo
  - (cs) added protection against using '.chunkdb' when folders were changed by user
  - (master) fixed free inodes list issue (can happen only during first 60 seconds after starting master with new metadata)

* MooseFS 3.0.25-1 (2015-06-11)

  - (cs) fixed sending disk sizes after chunk deletions
  - (tools) added displaying chunk information in "mfsfileinfo" for chunks exceeding file size
  - (cli+cgi) added "missing chunk type" column in "missing files" table
  - (master) added automatic fixing missing chunks exceeding file size
  - (cs) fixed "got unknown message (type:212)" issue
  - (cs) added ip and port to connection error message
  - (master+mount) added length and write/read synchronization between nodes
  - (mount) removed waiting for data sync on mfs_release
  - (master+mount) added not cacheable ENOENT status

* MooseFS 3.0.24-1 (2015-06-01)

  - (master) changed two highest replication priorities (fixes problem with slow replication from "marked for removal" disks)
  - (cs) added option '!' to paths in mfshdd.cfg - ignore 'metaid'

* MooseFS 3.0.23-1 (2015-05-19)

  - (configure) added python as dependency for mfscgiserv
  - (mount) readdata module improvements (masterdata cache)

* MooseFS 3.0.22-1 (2015-05-15)

  - (mount) fixed dircache invalidation

* MooseFS 3.0.21-1 (2015-05-14)

  - (master) fixed bug in flock (bsd) locks

* MooseFS 3.0.20-1 (2015-05-12)

  - (cli) fixed "last save time" in master general info (-SIG)
  - (master) added "last seen paths" to TRASH and SUSTAINED files in inode to path resolver
  - (mount) fixed "race conditions" in new reading module (intr. in 3.0.19)
  - (mount) added "mfsnoxattrs","mfsnoposixlocks" and "mfsnobsdlocks" options
  - (master) added support for "DEFAULT" options in mfsexports.cfg

* MooseFS 3.0.19-1 (2015-05-08)

  - (master) removed missing chunks/files from messages
  - (cli+cgi) added presenting missing chunks/files in separate table
  - (mount) improved reading performance
  - (master) added missing chunks/files log
  - (master) added fast delete during rebalance

* MooseFS 3.0.18-1 (2015-04-29)

  - (master) fixed bug in chunk unlocking mechanism (few writes to the same chunk may cause master freezing)

* MooseFS 3.0.17-1 (2015-04-20)

  - (master) fixed 'df' for subnodes with quota set on ancestor nodes

* MooseFS 3.0.16-1 (2015-04-15)

  - (all) changed freebsd ports
  - (cgiserv) fixed POST handling
  - (mount) fixed handling read-ahead upper limit

* MooseFS 3.0.15-1 (2015-04-10)

  - (master) added 'admin' flag to default mfsexport.cfg
  - (master) fixed setgoal bug (user couldn't change goal without admin privilege)

* MooseFS 3.0.14-1 (2015-04-09)

  - (mount) fixed rare getgroup race condition

* MooseFS 3.0.13-1 (2015-04-03)

  - (tools,mount) fixed man pages

* MooseFS 3.0.12-1 (2015-03-31)

  - (netdump) fixed man page

* MooseFS 3.0.11-1 (2015-03-18)

  - (all) changed licence of standard version to GPLv2
  - (all) changed name moosefs-ce to just moosefs

* MooseFS 3.0.10-1 (2015-03-10)

  - (master) fixed possible reference to freed memory during metadata recovery
  - (master) added minimal priority queue length
  - (cs) fixed removing damaged disks (lock file issue)
  - (master) treat servers in maintenance mode as overloaded (no chunk creations, replications etc.)
  - (master+tools) added tools for manipulating archive flag
  - (master+mount) added preferred labels for I/O
  - (master) added option for setting length of chunks priority queues

* MooseFS 3.0.9-1 (2015-02-19)

  - (all) added separated goal for "create", "keep" and "archive"
  - (all) added "create mode" (loose/standard/strict)
  - (master) fixed check for ENOSPC condition
  - (mount) added "chunkready" condition before close (alternative to full fsync)

* MooseFS 3.0.8-1 (2015-02-11)

  - (master) fixed descriptor leak
  - (mount) new delayed ops scheduler

* MooseFS 3.0.7-1 (2015-02-06)

  - (cs+master) added heavy-loaded status

* MooseFS 3.0.6-1 (2015-02-05)

  - (master) removed chunk preallocation
  - (master) added write-in-progress counters
  - (master) added experimental "new chunk" algorithm

* MooseFS 3.0.5-1 (2015-02-03)

  - (master) changed preallocation algorithm
  - (all) changed assert macros (added source file and line number to syslog msg)
  - (mount) made fsync before close optional (added option mfsfsyncbeforeclose)

* MooseFS 3.0.4-1 (2015-02-02)

  - (cs) made fsync before close optional
  - (all) added missing options to manpages
  - (mount+master) changed atime/mtime setattr handling (avoid fsync on mtime change)

* MooseFS 3.0.3-1 (2015-01-30)

  - (cs) disabled Nagle's algorithm in mainserv and replicator

* MooseFS 3.0.2-1 (2015-01-29)

  - (master) added experimental functions in preallocation

* MooseFS 3.0.1-1 (2015-01-28)

  - (master) changed preallocation formula
  - (master) changed choosing privlileged servers for rebalance replication
  - (cs) added new xor-replication

* MooseFS 3.0.0-1 (2015-01-22)

  - (master+mount) fixed small posix incompatibility (utime with NULL arg)
  - (all) change version to 3.0

* MooseFS 2.1.12-1 (2015-01-13)

  - (master+cs) monotonic chart counters
  - (cs) fixed error handling in writing data (mainserv)

* MooseFS 2.1.11-1 (2015-01-12)

  - (master) better state switching (pro version only)

* MooseFS 2.1.10-1 (2015-01-09)

  - (cs) fixed handling .metaid file
  - (cs) fixed showing incorrect I/O error messages
  - (cs) fixed drive removals

* MooseFS 2.1.9-1 (2015-01-08)

  - (master) force rebalancing when some servers are almost full (regardless of undergoal priority queues)
  - (master) fixed adding chunks to priority queues (when replication limits are not reached then undergoal chunks shouldn't be added to queues)
  - (master) fixed potential double decreasing number of sockets in session data

* MooseFS 2.1.8-1 (2015-01-07)

  - (master) fixed leader-follower synchronization issue (preallocated chunk deletion - pro version only)
  - (master) fixed setting goal to 9

* MooseFS 2.1.7-1 (2014-12-24)

  - (master) fixed serious incompatibility with mounts 2.1+
  - (master) added chunk preallocation
  - (master) added deleting wrong versioned chunks that do not belong to any file
  - (mount) fixed printed informations about mapall (thanks to Tom Ivar Helbekkmo)
  - (master) fixed rmdir mapall/maproot rights check (thanks to Tom Ivar Helbekkmo)

* MooseFS 2.1.6-1 (2014-12-22)

  - (master) added rollback after client disconnection

* MooseFS 2.1.5-1 (2014-12-19)

  - (master) fixed algorithm responsible for choosing servers for new chunks
  - (master) rollback changes when no chunkservers finished chunk ops (like create,duplicate etc.)

* MooseFS 2.1.4-1 (2014-12-17)

  - (cgi+cli) added labelset tables
  - (master) minor changes in replication algorithm

* MooseFS 2.1.3-1 (2014-12-15)

  - (master) fixed replication algorithm (no replication for labeled chunks when one or more servers are disconnected)
  - (cgi+cli) fixed typo (number of column for disconnected chunkservers in cli mode)

* MooseFS 2.1.2-1 (2014-12-12)

  - (master) added quota-licences (pro version only)
  - (master) restrict label manipulations to mounts with 'admin' property in exports file

* MooseFS 2.1.1-1 (2014-12-10)

  - (tools) added tools for copying mfs attributes (mfscopygoal,mfscopytrashtime,mfscopyeattr and mfscopyquota)

* MooseFS 2.1.0-1 (2014-12-04)

  - (mount+master) added support for global flock locks (Linux, kernel 3.8+, libfuse 2.9+ only)
  - (mount+master) added support for global posix locks (Linux only)
  - (cgi+cli) added resources tab with open files and acquired locks
  - (master) better 'wrong version' chunks handling
  - (mount) changes in 'write' procedures (much faster random writes)
  - (all) added support for labels (georeplication)

* MooseFS 2.0.44-1 (2014-11-27)

  - (master) fixed read rebalance limit

* MooseFS 2.0.43-1 (2014-10-31)

  - (master) introducing new licence model (new licences with size limit for pro version)
  - (all) fixed portability issues with usleep (thanks to Tom Ivar Helbekkmo)
  - (all) setting socket options after listen (thanks to Tom Ivar Helbekkmo)

* MooseFS 2.0.42-1 (2014-10-29)

  - (tools) fixed printing ip numbers in mfsfileinfo
  - (mount) changed format of ip numbers in messages from hex to standard
  - (master) changed format of ip numbers in some messages from hex to standard

* MooseFS 2.0.41-1 (2014-10-21)

  - (mount) fixed buffer overrun (FreeBSD and OS X only)

* MooseFS 2.0.40-1 (2014-10-15)

  - (cs) fixed serious bug (new chunks were created on disks 'marked for removal') - introduced in 1.7.31
  - (cs) fixed bug (segfault on very rare occasions during disk scanning just after startup - caused by damaged chunk files)
  - (mount) fixed memory leak (supplementary group tabs) - intr. in 2.0.1
  - (rpm) fixed systemd scripts
  - (daemons) added increasing limit of core dump size (debugging purposes)

* MooseFS 2.0.39-1 (2014-10-06)

  - (mount) added ability to mount from /etc/fstab on FreeBSD

* MooseFS 2.0.38-1 (2014-10-02)

  - (mount) fixed memory leak in workaround on FreeBSD early-release bug (FreeBSD only)

* MooseFS 2.0.37-1 (2014-09-23)

  - (cs) added internal open files limit
  - (master) changed chunkserver choosing algorithm

* MooseFS 2.0.36-1 (2014-09-18)

  - (cgi) fixed sorting masters by ip
  - (master) fixed acl posix compatibility (two tests from pjd)
  - (cs) fixed compatibility issues with master 1.6

* MooseFS 2.0.35-1 (2014-09-12)

  - (freebsd) fixed cgiserv and cli port
  - (mount) fixed setting acl

* MooseFS 2.0.34-1 (2014-08-21)

  - (mount) workaround for bug in FreeBSD Fuse version
  - (master) added setting correct metaid in "new" followers (pro version only)

* MooseFS 2.0.33-1 (2014-08-18)

  - (cs,supervisor) more reliable network code in supervisor module (pro version only)
  - (cs,master) added check for metaid in chunkservers (also hard drives) and secondary masters

* MooseFS 2.0.32-1 (2014-08-13)

  - (cs) changed delay handling method (delayed file close,free crc block etc.)
  - (mount) fixed truncate loop (fixed bug intr. in 2.0.26)

* MooseFS 2.0.31-1 (2014-08-12)

  - (cs) fixed bug: write data error not causing exit from write loop - sometimes could lead to segfault.
  - (master) fixed setting data cache drop bit.
  - (cli) fixed printed options for mfssupervisor

* MooseFS 2.0.30-1 (2014-07-09)

  - (daemons) added writing pid to lockfile
  - (all) added freebsd port-maker

* MooseFS 2.0.29-1 (2014-07-03)

  - (cs) moved fsync to background (improved write performance)

* MooseFS 2.0.28-1 (2014-06-30)

  - (master) fixed syslog message about metaloggers in masters (pro version only)

* MooseFS 2.0.27-1 (2014-06-27)

  - (tools) added to mfsfileinfo checksum digest and file signature calculation

* MooseFS 2.0.26-1 (2014-06-26)

  - (cs) fixed bug in replicator
  - (mount) fixed truncate loop
  - (mount) fixed potential descriptor leak
  - (master) fixed bug with duplicated chunkserver id

* MooseFS 2.0.25-1 (2014-06-23)

  - (mount) better handling error counters
  - (master) fixed replication status handling

* MooseFS 2.0.24-1 (2014-06-23)

  - (master) added unfinished replications detection
  - (master) added source ip and port in replication status message
  - (cs) fixed handling job buffers
  - (daemons) added ignoring profiling signals

* MooseFS 2.0.23-1 (2014-06-18)

  - (mount) changed chunkserver choosing algorithm
  - (master) added checks for busy/operation mismatch
  - (master) added checks for busy status during duplicate/duplicate+truncate operations
  - (cs) added ignoring 'marked for removal' disks in internal rebalance

* MooseFS 2.0.22-1 (2014-06-17)

  - (master) add chunks to priority queues after chunk damage, chunk lost and set goal

* MooseFS 2.0.21-1 (2014-06-17)

  - (supervisor+cstool) fixed version shown
  - (supervisor) fixed 'DEAD' master recognition routine
  - (master) fixed segfault introduced in 2.0.20

* MooseFS 2.0.20-1 (2014-06-16)

  - (cs) fixed handling unfinished jobs on socket timeout
  - (master) fixed missing chunks CHUNKDEL LEADER/FOLLOWER race condition (pro version only)
  - (master) postpone write to undergoal chunks during registration process (wait for all cs)
  - (master) undergoal replications have higher priority than write chunk
  - (all) added check for EINTR status after poll in sockets module

* MooseFS 2.0.19-1 (2014-06-13)

  - (master+cs) added support for 'get version' packet
  - (all) fixed licence text in source files

* MooseFS 2.0.18-1 (2014-06-12)

  - (cs) fixed bandwidth charts

* MooseFS 2.0.17-1 (2014-06-12)

  - (daemons) added 'info' handler (for future use)
  - (master) changed median to arithmetic mean in 'new chunk' algorithm

* MooseFS 2.0.16-1 (2014-06-11)

  - (master+cs) added 'maxentries' to chart data packet
  - (master) improved median usage calculations

* MooseFS 2.0.15-1 (2014-06-10)

  - (master) fixed floating replication limits

* MooseFS 2.0.14-1 (2014-06-10)

  - (master) replication limits can be defined as floating point numbers
  - (master+cgi+cli) changed store status (background,download and foreground)
  - (master+cgi+cli) changed option "canchangequota" to "admin"
  - (master) added rename/unlink "metadata.mfs" in case of using '-a' option
  - (master+cgi+cli) last save duration changed from seconds to miliseconds

* MooseFS 2.0.13-1 (2014-06-09)

  - (master) added separate replication limits for different cases
  - (cs) deleting already deleted chunk returns status ok.

* MooseFS 2.0.12-1 (2014-06-06)

  - (mount) changed readworker behaviour in case of having no data to read
  - (master) added block version check during metadata read
  - (metadump) added support for newest metadata format
  - (cs) changed format of reported ip in replicator module

* MooseFS 2.0.11-1 (2014-06-03)

  - (cgi) fixed missing master condition
  - (netdump) new binary - mfsnetdump
  - (cs) new binary - mfschunktool
  - (master+cgi+cli) added maintenance mode to chunkservers
  - (cli) fixed cs-commands bug

* MooseFS 2.0.10-1 (2014-06-02)

  - (cgi) added colors for versions
  - (master) fixed dirinfo counters update on write in master-follower (pro version only)

* MooseFS 2.0.9-1 (2014-05-30)

  - (all) fixed compatibility with very old compilers
  - (cgi) added avg line and tooltips to '%used' bars

* MooseFS 2.0.8-1 (2014-05-29)

  - (master) even better replication priority algorithm

* MooseFS 2.0.7-1 (2014-05-28)

  - (master) improved replication priority algorithm

* MooseFS 2.0.6-1 (2014-05-27)

  - (master) added replication priority "pseudomissing->endangered->undergoal->rebalance"
  - (master) added option for setting metadata download frequency (pro version only)

* MooseFS 2.0.5-1 (2014-05-26)

  - (master) fixed finding best metadata file (-a option)
  - (cs) fixed infinite loop condition in sockets module

* MooseFS 2.0.4-1 (2014-05-22)

  - (master+mount) fixed "chunk lost" condition during cs registration
  - (mount) fixed support for "tail -f"
  - (master) fixed xattr snapshot bug (attr name length not copied)

* MooseFS 2.0.3-1 (2014-05-20)

  - (master) autorepair chunk version for standard scenarios
  - (master) fixed dealing with chunks that doesn't exist in metadata
  - (master) fixed status returned by CHUNKADD/CHUNKDEL
  - (all) faster endian conversions

* MooseFS 2.0.2-1 (2014-05-15)

  - (mount) turn on posix-ACL's by default
  - (mount) add release number to version reporting (in df -i)

* MooseFS 2.0.1-1 (2014-05-14)

  - (master) download meta file from followers
  - (master) added "get paths" and "get parents" commands
  - (master) added ability to send directories in parts
  - (master) keep network connections alive during long operations
  - (all) keep CL-CS and CS-CS connections alive between operations
  - (mount) fixed race-condition in read module
  - (mount) fixed local buffer lock in 'oplog'
  - (cs) fixed race condition in resolver
  - (mount) fixed truncate in read module
  - (master) fixed new chunk/delete chunk constants
  - (master+mount) implementing full support for posix-ACL's
  - (master+mount+tools) move permission checking from kernel to master

* MooseFS 1.7.31 (2014-04-01)

  - (cs) fixed choosing disk algorithm
  - (cs) removed (some) unimportant syslog messages
  - (cs) tuning glibc malloc arenas
  - (cs) fixed protocol autonegotiation
  - (cs) fixed CS-CS and CS-CL communication charts

* MooseFS 1.7.30 (2014-03-26)

  - (cs) fixed compatibility with mfs 1.6
  - (mount) fixed "no xattr" bit bug (intr. in 1.7.25)

* MooseFS 1.7.29 (2014-03-26)

  - (cs) fixed descriptor leak (intr. in 1.7.25)

* MooseFS 1.7.28 (2014-03-11)

  - (all) fixed thread creation routine (introduced in 1.7.24)

* MooseFS 1.7.27 (2014-03-11)

  - (cs) added memory chart
  - (master+cs) added virtual memory usage to memory charts
  - (cs) fixed race condition in charts (introduced in 1.7.25)
  - (cgi+cli) fixed cs charts

* MooseFS 1.7.26 (2014-03-05)

  - (master+mount) create file using one command

* MooseFS 1.7.25 (2014-02-27)

  - (all) fixed another bug in new network module (introduced in 1.7.21 and 1.7.22)
  - (cs) new I/O model
  - (master) added xattr flag
  - (master) added copying xattrs during snapshot (cp-like version)
  - (master) changed metadata version handling
  - (master) changed node structure (RAM and HDD)
  - (master+cs) added chunkserver id (identyfying cs in master)
  - (master+cs) new registration algorithm (vers. 6)
  - (master+mount) "no xattr" bit in file attr

* MooseFS 1.7.24 (2014-02-19)

  - (mount) fixed xattr cache bug
  - (master) fixed ENOSPC condition after switching
  - (cgi+cli) fixed sortkeys for hdd write bandwidth
  - (master) turn off internal supervisor if there are connections from external supervisor
  - (cs) added common routine for thread creation

* MooseFS 1.7.23 (2014-02-06)

  - (master) bug fix version (bugs introduced in 1.7.21 and 1.7.22)

* MooseFS 1.7.22 (2014-02-05)

  - (master) fixed cs disconnection (make master responsive during disconnection)

* MooseFS 1.7.21 (2014-02-05)

  - (master) fixed master<->chunkserver network module (make master responsive during chunk registration)

* MooseFS 1.7.20 (2014-02-03)

  - (mount) added negative lookup cache

* MooseFS 1.7.19 (2014-01-30)

  - (mount) added xattr cache

* MooseFS 1.7.18 (2014-01-29)

  - (mount) setting FUSE capabilities (improved efficiency)
  - (cs) fixed disk choosing algorithm

* MooseFS 1.7.17 (2014-01-28)

  - (cs) fixed mutex double-lock error (introduced in 1.7.14)
  - (master) improved calculation of init hash size

* MooseFS 1.7.16 (2014-01-22)

  - (master+cgi+cli) memory usage detailed report
  - (master+cs+ml) fixed starvation scenario in network modules

* MooseFS 1.7.15 (2014-01-15)

  - (master+cs) new charts prepared for mfs 2.0
  - (cs) options for transferring data between disks

* MooseFS 1.7.14 (2014-01-08)

  - (master) added simple supervisor (allow start masters without manual supervisor)
  - (cs) fixed internal rebalance algorithm
  - (cgi+cli) new class for mfs communication
  - (cgi+cli) introduced pro/ce version (for forthcoming mfs 2.0)

* MooseFS 1.7.13 (2013-12-12)

  - (cs) added rebalance utilization limit option
  - (cs) added error tolerance options

* MooseFS 1.7.12 (2013-12-11)

  - (master) fixed metadata synchronization bug (HUP detected to early)
  - (master) fixed metadata synchronization bug (settrashtime,setgoal,seteattr)
  - (master+cs) changed cs/disk choosing algorithm (random->rr and using only servers/disks close to median)
  - (cs) added rebalancing of disks usage

* MooseFS 1.7.11 (2013-11-22)

  - (mount) fixed read/write bug (status EAGAIN on read/write caused EIO)
  - (cli) added user defined separator
  - (cgi+cli) display licence data

* MooseFS 1.7.10 (2013-11-21)

  - (master) added licences for version pro
  - (mount) fixed very rare race condition (fuse context)

* MooseFS 1.7.9 (2013-11-12)

  - (master) fixed chunk deletions during chunkserver disconnection
  - (daemons) fixed 'stop' status (ok when daemon is already stopped)
  - (daemons) added 'try-restart' action
  - (all) fixed status returned from socket operations
  - (mount) add ignoring SIGPIPE
  - (all) improved crc32 calculation (1.7 times faster)

* MooseFS 1.7.8 (2013-11-05)

  - (master) new 'open files' module (also fixed some bugs)
  - (master) better management of chunks deletion in master
  - (master) fixed bug (adding chunkservers by ELECT)
  - (cgi+cli) added displaying disconnected sessions
  - (master+cgi+cli) added option to remove disconnected sessions

* MooseFS 1.7.7 (2013-10-22)

  - (master+cs) added ability to change leader on demand
  - (mount) fixed couple of small bugs in new "read data" module
  - (supervisor) fixed connecting bugs
  - (master) added progress info during metadata restoring
  - (master) fixed net timeouts (fixing time after long operations)
  - (mount) fixed connection problems when one master is dead

* MooseFS 1.7.6 (2013-09-20)

  - (master) imporoved chunk hashmap

* MooseFS 1.7.5 (2013-09-12)

  - (master) fixed network timeouts for data transfer
  - (master) changed hashmaps
  - (cgi) BUSY state detection (mainly during data transfer)

* MooseFS 1.7.4 (2013-08-14)

  - (mount) read procedures have been rewritten (with read ahead support)
  - (mount) read+write dynamic number of workers
  - (cs) dynamic number of workers
  - (cs) fixed segfault condition during closing application
  - (master) improved memory allocation
  - (master) improved hashmaps
  - (master) fixed bug (sending changelogs to desynchronized followers)
  - (master) fixed bug (cs added to metadata in desynchronized followers)
  - (master) imporoved sending old changelogs (sending in groups)

* MooseFS 1.7.3 (2013-07-30)

  - (man) automatically set release date and version
  - (cgiserv) fixed lock-file issue (early truncate - introduced in 1.7.1)
  - (master+mount) when chunk is definitely lost then do not wait for it in clients
  - (cs) reduced amount of space reports to master (inspired by Davies Liu)
  - (mount) imporved write efficiency (inspired by Davies Liu)
  - (master) fixed 'rename/mv' bug in metarestore
  - (master) fixed csdb cleanup bug
  - (master) added SYNC/DESYNC to state description in syslog
  - (supervisor) imporved connecting to masters

* MooseFS 1.7.2 (2013-07-12)

  - (cgi+cli) fixed recognition of using wrong DNS name
  - (cgi+cli) fixed commands bug in cli mode
  - (cgi+cli) added inputs for master host name and ports
  - (all) fixed distclean (automake strange behavior)

* MooseFS 1.7.1 (2013-07-11)

  - (cgi+cli) simple support for master-ha
  - (cgi+cli) fixed path bug introduced in 1.7.0
  - (all) added default-mastername and default-portbase options to configure script
  - (mount) fixed reconnecting to ELECT
  - (cgiserv) fixed pidfile bug

* MooseFS 1.7.0 (2013-07-05)

  - (cgi+cli) mfs.cgi can be used also in CLI mode (for monitoring scripts etc.)
  - (dist) CGI permissions fix in RPM packaging
  - (dist) preliminary systemd support inspired by Arch Linux
  - (master+tools) added building files from particular chunks (manual file recovery)
  - (cs) join master tasks queue and client tasks queue into one tasks queue.
  - (cgi+cli) added chunkserver 'load' charts.
  - (cgi+cli) added 'load' column in chunkservers table
  - (master+cs) simple chunkserver overload detection
  - (master) 'grace' state for chunkservers - after overload detection given chunkserver is in 'grace' state for some time - new chunks are not created on that server in this state
  - (cgi+cli+master) turn off 'grace' state
  - (master) in 'noowner' i-nodes mode copy 'owner' bits to 'group' and 'other' bits
  - (common) increase time events frequency
  - (master) increase freqency of chunk-loop task (1 per second to 50 per seconds)
  - (mount) fixed invalidate open-dir dentry cache
  - (master) fixed sustaining old sessions by clients who can't properly register
  - (cs) improved efficiency (removed lock in network thread)
  - (master) fixed chunkserver disconnection handling (remove server data form chunks 'in background')
  - (master) changed directory size (from GB to 'floating point' numbers, size nxxxxyy = xxxx.yy * 2^(10*n), f.e. 2053189 = 531.89MB)
  - (master+tools) added mfssnapshot option (cp-like mode) - set files ownership to user who makes snapshot.
  - (cgi+cli) python scripts where made Python 3 compatible
  - (all) added support for quota (per directory)
  - (all) added support for xattr's (without special ACL treatment)
  - (master) fixed dirinfo calculation bug (symbolic link snapshots)
  - (all) added support for high availability
  - (master) added meterestore option (mfsmetarestore is now included in mfsmaster)

* MooseFS 1.6.27 (2012-08-09)

  - (metarestore) fixed bug - freeing filenames memory too early
  - (all) added initial support for extra attributes (xattr), which will be introduced in upcoming version 1.7
  - (master+metalogger) better change log synchronization (storage in master memory and sending expected version in metalogger - inspired by Davies Liu)
  - (master) acceptable difference of percent of hdd usage added to configuration (up to this version this parameter was constantly set to 0.01% - patch contributed by Davies Liu)
  - (master) added emergency store metadata to some other places on errors during standard hourly store (inspired by Davies Liu)
  - (cs) default space to be left (256MB) moved to config file (inspired by Davies Liu)
  - (cs) added extra limits in mfshdd.cfg (minimum space to be left, maximum space to be used)
  - (cs) fixed charts overflow issue (overflow in net transfer on about 575Mbps and hdd transfer on about 77MBps)
  - (metalogger) fixed issue: file variable was not clear after close (on rare occasions might cause segfault after disconnecting from master)
  - (all) cfg files moved form PREFIX/etc/ to PREFIX/etc/mfs/
  - (cgiserv) improved CGI handle (added support for custom http responses, such as "302 Found")
  - (master+cgi) showing disconnected chunkservers in "Servers" tab.
  - (deb+rpm) mfscgiserv moved from mfscgi to separate package, changes in init scripts
  - (mount) added option 'mfsdelayedinit' - for being run from fstab/init scripts
  - (master) optimized goal management in chunks
  - (master) fixed rare race-condition on clear/preserve cache during open in mount
  - (mount) fixed compiling problems on Mac OS X
  - (all) changed code to be more compatible with new gcc 4.7 (gcc 4.7 has too strong optimizations - it can generate unpredictable code)
  - (master) sustain session time could be defined in mfsmaster.cfg

* MooseFS 1.6.26 (2012-02-01)

  - (all) fixed signal handling in multithreaded modules
  - (master) added goal and trashtime limits to mfsexport.cfg
  - (metalogger) added simple check for downloaded metadata file (inspired by Davies Liu)
  - (master) better handle disk full (inspired by Davies Liu)
  - (master+metalogger) added keeping previous copies of metadata (inspired by Davies Liu)
  - (all) reload all settings on "reload" (SIGHUP)
  - (cs) disk scanning in background
  - (cs) fixed long termination issue (found by Davies Liu)
  - (master) fixed modify/open cache race

* MooseFS 1.6.25 (2011-12-30)

  - (metadump) fixed dumping big files (>2TiB)
  - (metarestore) fixed bug: nonexisting changelog file caused segv
  - (master+mount) added 'sugidclearmode' and 'mkdircopysgid' compatibility options
  - (master) improved chunk deletion algorithm (soft/hard limits per server)
  - (all) ready for new metadata file format, which will be introduced in upcomoing version 1.7
  - (all) ready for quota handling, which will be introduced in upcoming version 1.7

* MooseFS 1.6.24 (2011-12-06)

  - (master+mount) proxy in mount for mfstools (fixes problems with frequent connect to master)

* MooseFS 1.6.23 (2011-11-08)

  - (master+mount) removed directory cache (didn't increase performance as expected and caused many troubles)
  - (metarestore) added option (-i) - ignore some structure inconsistencies
  - (metarestore) added option (-b) - in case of errors save the best metadata file
  - (mount) more dynamic write cache management (changed condition ib<tb/5 to ib<3*fb where: ib - inode blocks in cache, tb - total blocks in cache, fb - free block in cache)
  - (master) save metadata file to alternative locations in case of error
  - (all) increased file length limit from 2TiB to 128PiB
  - (cgiserv) fixed directory traversal vulnerability
  - (cgiserv) added lockfile/pidfile and actions such as 'start', 'stop', 'restart' and 'test'.
  - (mount) fixed parsing file with defaults

* MooseFS 1.6.22 (2011-05-09)

  - (mount) added resolving master hostname whenever connection has failed
  - (cs) removed getting attributes of every chunk during initialization - speeds up starting of chunkserver
  - (cs) changed calculating of total/used space. Superuser space now is subtracted from total space instead of adding it to used space
  - (master+mount) fixed directory cache.
  - (debian) rewritten init scripts to use mfscommon commands (start/stop/restart/reload) instead of start-stop-daemon (where stop caused killing all instances of daemon)
  - (debian) changed init scripts to bail out early if MFS*_ENABLE is not true (i.e. given daemon is not scripts-controlled)

* MooseFS 1.6.21 (2011-04-14)

  - (mount) added support of default config file (mfsmount.cfg)
  - (metarestore) fixed snapshot bug
  - (metarestore) improved tolerance for damaged changelog files
  - (master,mount) added full directory (with attributes) cache on client (mfsmount) side
  - (mount) added symlink cache on client (mfsmount) side
  - (mount) added hidden files '.oplog' and '.ophistory' with detailed info about current/historical operations performed by mfsmount
  - (master) added simple net topology support
  - (all) added -D_POSIX_PTHREAD_SEMANTICS to CFLAGS - needed by Solaris-like OSes
  - (cs) fixed detection of 'damaged disk' condition
  - (mount) fixed error caused segfaults during umount on certain conditions
  - (daemon) added 'test' command - checks if process is running and returns its PID

* MooseFS 1.6.20 (2011-01-14)

  - (cs) fixed "packet too big" issue during register to master (split big register packet with all chunks info into small packets)
  - (cs,metalogger,master) added minimum socket timeout (ten seconds)
  - (mount) accepting nop packets during write (for future use)

* MooseFS 1.6.19 (2010-12-15)

  - (debian) separated mfs-common and mfs-metalogger subpackages, added init scripts (contributed by Christopher Lewis)
  - (daemon) fixed return values (return non zero on error)
  - (cs) fixed chunk testing bug (any error during chunk opening caused assigning whole disk as damaged)
  - (cs,metalogger) added resolving master name when connection failed (patch contributed by Davies Liu)
  - (mount) added creating new session when prevoius is lost (inspired by Davies Liu)
  - (cs) added for unused chunks week delay before deletion (inspired by Davies Liu)
  - (cgi) added switching between name and IP in 'path' column in 'Disks' table (inspired by Davies Liu)
  - (master) do not update ctime when goal, trashtime or extra attributes are not changing

* MooseFS 1.6.18 (2010-11-08)

  - (metalogger) added sending metadata after metalogger startup
  - (master,metalogger) added sending two change logs together with metadata
  - (metarestore) imporved merging change logs
  - (all) added a lot of assertions (mainly NULL pointers, and unsuccessful thread functions)
  - (all) fixed some minor bugs and potential race conditions (makes valgrind happy)
  - (cs) added ability to use read-only disks in "marked for removal" mode (to retrieve missing chunks from damaged disks)
  - (cs) added showing scanning progress during startup
  - (all/master) added releasing used resources at the end
  - (all) changed some malloc/free allocations into mmap/munmap
  - (all) new error messages routines
  - (daemons) added proper handling SIGINT in "foreground" mode
  - (cgiserv) fixed small bug (malformed request could crash cgiserv)

* MooseFS 1.6.17 (2010-07-20)

  - (master,tools) added automatic data cache management
  - (master,tools) new flag "nodatacache"

* MooseFS 1.6.16 (2010-07-19)

  - (master) added clearing suid/sgid bits
  - (master) added check for "sticky" flag during rename and unlink operations
  - (master) fixed posix compatibility (removing empty directory on rename)
  - (master) fixed posix compatibility (proper changing ctime)
  - (master) fixed some constants (better support for larger systems - with millions of files and chunks)
  - (master) fixed error logging (logging is turning off when there are too many messages)
  - (all) fixed some OpenSolaris compile issues (mainly added -D__EXTENSIONS__ to compile flags)
  - (all) fixed OpenSolaris pthreads issue (errno doesn't work correctly without proper compiler flags)
  - (man) added mfscgiserv man page, added "BIND" options descriptions to man pages
  - (cs) fixed hdd stats (overflow on negative time difference)

* MooseFS 1.6.15 (2010-04-09)

  - (daemons) fixed "double free" error
  - (metarestore) fixed bugs in "REPAIR" and "SNAPSHOT" entries
  - (master) fixed bug in "snapshot" command ("mfsmakesnapshot dir dir/" caused master to hung-up)
  - (master) preserving atime and mtime during "snapshot" operation (makes "snapshot" to work more like "cp -Rp" than "cp -R")
  - (cs) ommit "marked for removal" disks during chunk test loop

* MooseFS 1.6.14 (2010-03-19)

  - (all) improved messages
  - (cs) prevent from using localhost as a master IP address
  - (all) remove some BSDisms from code
  - (metalogger) added BIND_HOST option (as in mount and cs)

* MooseFS 1.6.13 (2010-02-08)

  - (master) fixed poll events bug
  - (autotools) added m4 module for posix threads detection
  - (master) added exporting "meta" to default mfsexports.cfg
  - (master,tools) removed "allowdatacache" flag (will be managed automatically in the future)
  - (master,cs) added support for not compressed PNG images in chart module (when build without zlib)

* MooseFS 1.6.12 (2010-01-25)

  - (mount) fixed locking in read module
  - (mount,cs) added ability to bind outgoing sockets to specific IP

* MooseFS 1.6.11 (2009-12-31)

  - (cs) removed dirent.d_type test
  - (cs) fixed some memory leaks
  - (mount) improved attribute cache

* MooseFS 1.6.10 (2009-12-09)

  - fixed errors introduced in 1.6.9
  - added 'reload' command to master
  - split chunk counters into 'all' and 'regular' (new approach to disks removal - since this version chunks on 'marked for removal' disks are not deleted)

* MooseFS 1.6.9 (2009-11-24)

  - updated manpages
  - added support for mlockall (prevent from swapping)
  - redesigned charts module
  - redesigned locking mechanizm (daemons)
  - added HDD I/O stats
  - removing unfinished jobs from queue on socket close
  - added 'cache file' flag

* MooseFS 1.6.8 (2009-10-26)

  - added options to set nice level and number of I/O retries in mfsmount

* MooseFS 1.6.7 (2009-10-05)

  - fixed error: damaged disk caused SIGSEGV in mfschunkserver

* MooseFS 1.6.6 (2009-10-02)

  - fixed errors introduced in 1.6.5
  - changed directory structure in CS (from 16 subfolders to 256 subfolders)

* MooseFS 1.6.5 (2009-09-23)

  - fixed some performance issues
  - added new module "mfsmetalogger"

* MooseFS 1.6.4 (2009-08-25)

  - rewrite mfs to use poll instead of select
  - fixed some errors in new write module
  - limit cache usage by inode in write module

* MooseFS 1.6.3 (2009-08-24)

  - reduced version increasing
  - increased open files limit in chunkserver
  - decreased time window in damage disk detection

* MooseFS 1.6.2 (2009-08-19)

  - fixed "race" in writedata module
  - increased session timeout in mfsmaster for new sessions

* MooseFS 1.6.1 (2009-08-14)

  - Added 'mapall' option to exports
  - Added flags 'noattrcache' and 'noentrycache'
  - Rewritten "Writedata" module in mfsmount
  - Added source (read) limit to replications
  - Prefer localhost during some operations (reduce network usage)

* MooseFS 1.6.0 (2009-07-01)

  - (all) Removed duplicate source files.
  - (all) Strip whitespace at the end of configuration file lines.
  - (cs) Rewritten in multi-threaded model.
  - (cs) Added periodical chunk testing functionality (HDD_TEST_FREQ option).
  - (cs) New -v option (prints version and exits).
  - (master) Added "noowner" objects flag (causes objects to belong to current user).
  - (master) Maintaining `mfsdirinfo` data online, so it doesn't need to be calculated on every request.
  - (master) Filesystem access authorization system (NFS-like mfsexports.cfg file, REJECT_OLD_CLIENTS option) with ro/rw, password and maproot functionality.
  - (master) New -v option (prints version and exits).
  - (mount) Rewritten options parsing in mount-like way, making possible to use standard FUSE mount utilities (see mfsmount(8) manual for new syntax). Note: old syntax is no longer accepted and mountpoint is mandatory now (there is no default).
  - (mount) Updated for FUSE 2.6+.
  - (mount) Added password, file data cache, attribute cache and entry cache options. By default attribute cache and directory entry cache are enabled, file data cache and file entry cache are disabled.
  - (mount) opendir() no longer reads directory contents - it's done on first readdir() now; fixes "rm -r" on recent Linux/glibc/coreutils combo.
  - (mount) Fixed mtime setting just before close() (by flushing file on mtime change); fixes mtime preserving on "cp -p".
  - (mount) Added statistics accessible through MFSROOT/.stats pseudo-file.
  - (mount) Changed master access method for mfstools (direct .master pseudo-file replaced by .masterinfo redirection); fixes possible mfstools race condition and allows to use mfstools on read-only filesystem.
  - (tools) Units cleanup in values display (exact values, IEC-60027/binary prefixes, SI/decimal prefixes); new options: -n, -h, -H and MFSHRFORMAT environment variable - refer to mfstools(8) manual for details).
  - (tools) mfsrgetgoal, mfsrsetgoal, mfsrgettrashtime, mfsrsettrashtime have been deprecated in favour of new "-r" option for mfsgetgoal, mfssetgoal, mfsgettrashtime, mfssettrashtime tools.
  - (tools) mfssnapshot utility replaced by mfsappendchunks (direct descendant of old utility) and mfsmakesnapshot (which creates "real" recursive snapshots and behaves similar to "cp -r").
  - (tools) New mfsfilerepair utility, which allows partial recovery of file with some missing or broken chunks.
  - (cgi) First public version of CGI scripts allowing to monitor MFS installation from WWW browser.

* MooseFS 1.5.12 (2009-01-28)

  - Fixed CRC cache in chunkserver

* MooseFS 1.5.11 (2008-12-18)

  - Added CRC-block and file descriptor cache in chunkserver.
  - Removed compilation warnings on 64-bit machines.
  - Optimized chunkserver choice in mfsmount.
  - Better replication algorithm.

* MooseFS 1.5.10 (2008-12-03)

  - Fixed replication algorithm.

* MooseFS 1.5.9 (2008-11-19)

  - Improved rebalancing algorithm.
  - Protections against overriding metadata.mfs.back by older metadata.mfs.
  - Fixes to build mfstools, mfsmaster and mfsmetarestore with Large File Support on Linux (previously only mfsmount had LFS enabled).
  - Less verbose logging in mfsmaster.
  - Log chunkserver disconnections in mfsmaster.

* MooseFS 1.5.8 (2008-08-26)

  - Better memory allocation in Master's chunks module.
  - Syslog messages used only for debugging purposes were commented out.

* MooseFS 1.5.7 (2008-08-01)

  - Fix for stack buffer overrun (by 4 bytes) in mfsmount.

* MooseFS 1.5.6 (2008-07-25)

  - Memory allocation fix in mfsmaster.

* MooseFS 1.5.5 (2008-07-10)

  - Mounting options fix for FreeBSD in mfsmount.

* MooseFS 1.5.4 (2008-07-08)

  - Blocking rebalance replications when undergoal chunks exist.
  - Minor bugfixes.
  - Build system improvements.
  - MooseFS and mfsmount version numbers synchronized.

* MooseFS 1.5 (2008-05-30)

  - First version released as Open Source.