lrzip-0.551 Compressing from stdin should be unbroken again. Compression values returned at the end of stdin work. lzma failing to compress a block will not cause a failure. lrzip-0.550 Speed up compression on large files that take more than one pass by overlapping work on successive streams, thus using multiple CPUs better. Fix for failures to decompress large files. Decompression will be slightly slower but more reliable. Faster lzma compression by default, less prone to memory failures, but at slight compression cost. Recover from multithreaded failures by serialising work that there isn't enough ram to do in parallel. Revert the "smooth out spacing" change in 0.544 as it slowed things down instead of speeding them up. Larger compression windows are back for 32 bits now that memory usage is kept under better control. Fixed some memory allocation issues which may have been causing subtle bugs. lrzip-0.544 Hopefully a fix for corrupt decompression on large files with multiple stream 0 entries. Fix for use under uclibc. Fix for memory allocation errors on large files on 32 bits. Smooth out spacing of compression threads making better use of CPU on compress and decompress. Fix for using -U on ultra-small files. Use bzip2 on blocks that lzma fails to compress to make sure they are still compressed. lrzip-0.543 A fix for when large files being decompressed fail with multithreaded decompression. Slight speedup on multithreaded workloads by decreasing the nice value of the main process compared to the back end threads as it tends to be the rate limiting component. Fixed lzma compression windows being set way too small by default. lrzip-0.542 Lrzip will now try to select sane defaults for memory usage in cases where the virtual memory heavily overcommits (eg. Linux) as this seriously slows down compression. For compression windows larger than 2/3 ram, lrzip will now use a sliding mmap buffer for better performance. The progress output is more informative in max verbose mode, and will no longer do more passes than it estimates. 32 bit machines should be able to use slightly larger windows. The sliding mmap not working on 2nd pass onwards has been fixed which should speed up the slowdown of death. lrzip-0.540 MASSIVE MULTITHREADING on the decompression phase. Provided there are enough chunks of data in the archived file, lrzip will use as many threads as there are CPUs for the backend decompression. Much like the multithreading on the compression side, it makes the slower compression algorithms speed up the most. Fixed output from being scrambled and consuming a lot of CPU time on threaded zpaq compression. Further fixes to ensure window sizes work on 32 bit machines. Be more careful about testing for how much ram lrzip can use. Minor build warning fixes. Minor tweaks to screen output. Updated benchmarks. lrzip-0.530 MASSIVE MULTITHREADING on the compression phase. Lrzip will now use as many threads as you have CPU cores for the back end compression, and even continue doing the rzip preprocessing stage as long as it can which the other threads continue. This makes the slower compression algorithms (lzma and zpaq) much faster on multicore machines, to the point of making zpaq compression almost as fast as single threaded lzma compression. -p option added to allow you to specify number of processors to override the built-in test, or if you wish to disable threading. -P option to not set permissions has now been removed since failing to set permissions is only a warning now and not a failure. Further improvements to the progress output. Updated benchmarks and docs. lrzip-0.520 Just changed version numbering back to 2 point. lrzip-0.5.2 Fixed the Darwin build again. Fix the corner case of big ram usage on 32 bit zpaq failing due to the compression window not being limited by limiting zpaq to 600MB windows on 32 bits as well. Some previous failures now only induce warnings. Improved progress output. lrzip-0.5.1 Fixed the build on Darwin. Rewrote the rzip compression phase to make it possible to use unlimited sized windows now, not limited by ram. Unfortunately it gets progressively slower in this mode the bigger the file gets but you can compress a file of any size as one big compression window with it using the new -U option. Suggest you try the new improved -M mode first or in combination. See the docs for more information. Changed the memory selection system to simply find the largest reasonable sized window and use that by default instead of guessing the window size. Setting -M now only affects the window size, trying to find the largest unreasonably sized window that will still work. The default compression level is now 9 and affects the rzip compression stage as well as the backend compression. Fixed some potential failures during compression. Improved screen output with more reporting in verbose mode, and chunk size percentage update. Fixed file size reporting on compressed files generated from stdin. Changed to 3 point releases in case we get more than 9 subversions ;) lrzip-0.50 Rewrote the file format to be up to 5% more compact and slightly faster. Made the memory initialisation much more robust, with attempted fallback to still work even when initial settings fail. Updated a lot of the stdin code. The most common scenario of compression from stdin now works without temporary files. Lots more meaningful warnings if failure occurs. May be able to decompress files on 32 bit machines that were compressed on 64 bit machines with >2GB windows now if there is enough ram. lrzip-0.46 Added lrzuntar which works the same as lrztar -d. Con Kolivas May 2010 lrzip-0.45 Added docs for lrztar and lrunzip. Added distclean and maintainer-clean make targets. Created git repo: http://github.com/ckolivas/lrzip Con Kolivas March 2010 lrzip-0.44 Added an lrztar wrapper to compress / decompress whole directories (finally). Added -i option to give information about a compressed file. lrzip-0.43 Darwin support updated. Should build on OSX v10.5+ Finally, stdin/stdout support. Test archive integrity support. ZPAQ support in config files. lrzip-0.42 ZPAQ compression update now shows which rzip stream it's currently compressing making the update more useful. It also doesn't update unnecessarily with every byte compressed which was slowing it down a LOT. lrzip-0.41 ZPAQ compression backend! ZPAQ is from the family of "paq" compressors that have some of the best compression ratios around, but at the cost of extremely long compression and equally long decompression times. This can be enabled with the -z option and makes lrzip archives made with this not backwardly compatible. lrzip-0.40 Compression windows should be limited by available ram now on 64bit. The limit on 32bit is still 2GB. The compression advantages on large files on 64bit machines with large ram should be substantially better. The file format is no longer compatible with earlier versions of lrzip. Support for decompressing older formats is present, but all new files will be generated in the new format. Minor speedups. Decompression should no longer stall at 4GB boundaries for extended periods making decompression much faster on files >4GB in size. Documentation and benchmark updates galore. lrzip-0.31 The window size limit is now 2GB on both 32bit and 64bit. While it appears to be smaller than the old windows, only 900MB was being used on .30 even though it claimed to use more. This can cause huge improvements in the compression of very large files. Flushing of data to disk between compression windows was implemented to minimise disk thrashing of read vs write. Con Kolivas November 2009 lrzip-0.30 -P option to not set permissions on output files allowing you to write to braindead filesystems (eg fat32). Probably other weird and wonderful bugs have been introduced. Con Kolivas November 2009 lrzip-0.24 has updated functionality FEATURE ENHANCEMENTS lrzip.conf file may be used to set default parameters. Omit conf using environment: LRZIP=NOCONFIG lrzip..... LRZIP environment variable may be used in the future to store certain types of parameters. LZMA SDK has been upgraded to version 4.63. This version fixes some problems certain users observed, and is much simpler using a C-only wrapper interface. lrzip now is able to compute an ETA for completion. In order to do this, the file must be larger than one compression window in size. That is, is the compression window is 500MB, and the file is 1GB, then after the first pass, an ETA will be computed. If the file is smaller, then no estimate can be made. lrzip is now able to compute MB/s transfer speeds for both compression and decompression. CLEANUPS Some file cleanups have been done. Peter Hyman January 2009 pete@peterhyman.com lrzip-0.22 update FEATURE ENHANCEMENTS -g option. Now supports gzip compression. Very fast! Expanded dictionary buffer size in lzma compressor. Variable, expanded dictionary size buffer in both lzma compressor and decompressor. Improved output during compression when using -vv. Multi-threading supprt when using multiple processors or dual core processors when using lzma compression. This results in a nearly 2x speed improvement. Assembler module support to speed up CRC checking. Improvements in autotools usage, system detection and Makefile enhancements. Lrzip now has a timer that will print total time at the end of a compression or decompression if -q command line option is not used. BUG FIX!!! Even though lrzip uses a compression threshold to prevent the lzma compressor from getting data that may not be compressible, there was still a possibility that lrzip could hang. This was because a data chunk could contain an uncompressible segment and if the lzma compressor got it, it would hang. THANKS TO LASSE COLLIN for uncovering the error in the lzma wrapper code that was causing the hangup. January 2008 Peter Hyman pete@peterhyman.com