HDF5 version 1.4.4 released on Tue Jul 2 11:38:00 CDT 2002
==============================================================================
INTRODUCTION
This document describes the differences between HDF5-1.4.3 and
HDF5-1.4.4, and contains information on the platforms tested and
known problems in HDF5-1.4.4. For more details check the HISTORY.txt
file in the HDF5 source.
The HDF5 documentation can be found on the NCSA ftp server
(ftp.ncsa.uiuc.edu) in the directory:
/HDF/HDF5/docs/
For more information, see the HDF5 home page at:
http://hdf.ncsa.uiuc.edu/HDF5/
If you have any questions or comments, please send them to:
hdfhelp@ncsa.uiuc.edu
CONTENTS
- New Features
- Bug Fixes since HDF5-1.4.3
- Performance Improvements
- Documentation
- Platforms Tested
- Supported Configuration Features
- Known Problems
New Features
============
o Configuration
================
* The H4 to H5 tools have been removed from the main source and placed
in a separate package. You can get these tools from the HDF ftp site
(ftp://hdf.ncsa.uiuc.edu/). The "--with-hdf4" command-line option
during configure is no longer valid. BW - 2002/06/25
o Library
=========
o General
---------
* Fill-value forward-compatibility with release 1.5 was added. SLU -
2002/04/11
* A new query function H5Tget_member_index has been added for compound
and enumeration data types. This function retrieves a member's index
by name. SLU - 2002/04/05
* Added serial multi-gigabyte file size test. "test/big -h" shows
the help page. AKC - 2002/03/29
o APIs
------
* The F90 subroutines h5dwrite_f, h5dread_f, h5awrite_f, and h5aread_f
were overloaded with a "dims" argument of type INTEGER(HSIZE_T) to
specify the size of the array. We recommend using these subroutines
with the new type; module subroutines that accept "dims" as an i
INTEGER array of size 7 will be deprecated in release 1.6.
EIP - 2002/05/06
o Performance
-------------
* Added internal "small data" aggregation, which can reduce the number of
actual I/O calls made, improving performance. QAK - 2002/06/05
* Improved internal metadata aggregation, which can reduce the number of
actual I/O calls made, improving performance. Additionally, this can
reduce the size of files produced. QAK - 2002/06/04
* Improved internal metadata caching, which can reduce the number of
actual I/O calls made by a substantial amount, improving
performance. QAK - 2002/06/03
o Parallel library
==================
* Fixed bug in parallel I/O routines where a collective I/O which used
MPI derived types, followed by an independent I/O would cause the library
to hang. QAK 2002/06/24
* Added environment variable flag to control whether creating MPI derived
types is preferred or not. This can affect performance, depending on
which way the MPI-I/O library is optimized. The default is set to
prefer MPI derived types for collective raw data transfers; setting the
HDF5_MPI_PREFER_DERIVED_TYPES environment variable to "0" (i.e.:
"setenv HDF5_MPI_PREFER_DERIVED_TYPES 0") changes the preference to avoid
using them whenever possible. QAK - 2002/06/19
* Changed MPI I/O routines to avoid creating MPI derived types (and thus
needing to set the file view) for contiguous selections within datasets.
This should result in some performance improvement for those types of
selections. QAK - 2002/06/18
* Changed MPI type support for collective I/O to be enabled by default.
This can be disabled by setting the HDF5_MPI_OPT_TYPES environment
variable to the value "0". QAK - 2002/06/14
* Allowed chunks in chunked datasets to be cached when parallel file is
opened for read-only access (bug #709). QAK - 2002/06/10
* Changed method for allocating chunked dataset blocks to only allocate
blocks that don't already exist, instead of attempting to create all the
blocks all the time. This improves performance for chunked
datasets. QAK - 2002/05/17
* Allowed the call to MPI_File_sync to be avoided when the file is going to
immediately be closed, improving performance. QAK - 2002/05/13
* Allowed the metadata writes to be shared among all processes, easing the
burden on process 0. QAK - 2002/05/10
o Tools
=======
* h5redeploy utility was added. It updates HDF5 compiler tools
after the HDF5 software has been installed in a new location.
o Support for new platforms and languages
=========================================
* Parallel Fortran Library works now on HP-UX B.11.00 Sys V.
EIP - 2002/05/06
* Intel C++ and F90 compilers Version 6.0 are supported on Linux 2.4.
* Intel C++ compilers Version 6.0 are supported on Windows 2000.
o Misc.
=========================================
* zlib has been moved out of the Windows source release. Users should go to
the ZLIB homepage(http://www.zlib.org) to download the corresponding
zlib library.
* The Windows binary release is built with the old version of the zlib
library. We expect users to use zlib 1.1.4 to build with the source
release.
* In the Windows-specific install document, we specify how to test backward
compatibility. However, in this release, we are not testing the backward
compatibility of HDF5.
Bug Fixes since HDF5-1.4.3 Release
==================================
* Fixed bug in chunking routines where they were using internal allocation
free routines, instead of malloc/free, preventing user filters from
working correctly. Chunks are now allocated/freed with malloc/free and
so should the chunks in user filters. QAK 2002/06/18
* Fixed bug where regular hyperslab selection could get incorrectly
transferred when the number of elements in a row did not fit evenly
into the buffer provided. QAK 2002/06/12
* Fixed bug (#499) which allowed an "empty" compound or enumerated datatype
(one with no members) to be used to create a dataset or to be committed
to a file. QAK - 2002/06/11
* Fixed bug (#777) which allowed a compound datatype to be inserted into
itself. QAK - 2002/06/10
* Fixed bug (#789) where creating 1-D dataset region reference caused the
library to go into infinite loop. QAK - 2002/06/10
* Fixed bug (#699, fix provided by a user) where a scalar dataspace was
written to the file and then subsequently queried with the
H5Sget_simple_extent_type function; type was reported as H5S_SIMPLE
instead of H5S_SCALAR. EIP - 2002/06/04
* Clear symbol table node "dirty" flag when flushing symbol tables to
disk, to reduce I/O calls made & improve performance. QAK - 2002/06/03
* Fixed bug where an object's header could get corrupted in certain
obscure situations when many objects were created in the
file. QAK - 2002/05/31
* Fixed bug where read/write intent in file IDs created with H5Freopen
was not being kept the same as the original file. QAK - 2002/05/14
* Fixed bug where selection offsets were not being used when iterating
through point and hyperslab selections with
H5Diterate(). QAK - 2002/04/29
* Fixed bug where the data for several level deep nested compound &
variable-length datatypes used for datasets were getting corrupted when
written to the file. QAK - 2002/04/17
* Fixed bug where selection offset was being ignored for certain hyperslab
selections when optimized I/O was being performed. QAK - 2002/04/02
* Fixed limitation in h5dumper with object names which reached over 1024
characters in length. We can now handle arbitrarily larger sizes for
object names. BW - 2002/03/29
* Fixed bug where variable-length string type did not behave as a
string. SLU - 2002/03/28
* Fixed bug in H5Gget_objinfo() which was not setting the 'fileno'
of the H5G_stat_t struct. QAK - 2002/03/27
* Fixed data corruption bug in hyperslab routines when contiguous
hyperslab that spans entire dimension and is larger than type
conversion buffer is attempted to be read. QAK - 2002/03/26
Performance Improvements
========================
This release of the HDF5 library has been extensively tuned to improve
performance, especially to improve parallel I/O performance.
Most of the specific information for particular performance improvements
is mentioned in the "New Features" and "Bug Fixes since HDF5-1.4.3" sections
of this document, but in general, the library should make fewer and larger
I/O requests when accessing a file. Additionally, improvements to the parallel
I/O portions of the library should have reduced the communications and barriers
used in various internal algorithms, improving the performance of the library.
However, with the extensive changes to some portions of the library that
were required for these improvements, some errors or unanticipated results may
have been introduced also. Please report any problems encountered to our
support team at hdfhelp@ncsa.uiuc.edu.
Hopefully these improvements will benefit all HDF5 applications, but if
there are particular I/O patterns that appear to be slower than necessary,
please send e-mail to hdfhelp@ncsa.uiuc.edu with a sample program showing the
problem behavior; we will look into the issue to see if it is possible to
address it.
Documentation
=============
* Documentation was updated for the hdf5-1.4.4 release.
* A new "HDF5 User's Guide" is under development. See
http://hdf.ncsa.uiuc.edu/HDF5/doc_dev_snapshot/H5_NewUG/current/.
* A "Parallel HDF5 Tutorial" is available at
http://hdf.ncsa.uiuc.edu/HDF5/doc/Tutor/.
* The "HDF5 Tutorial" is not distributed with this release. It is
available at http://hdf.ncsa.uiuc.edu/HDF5/doc/Tutor/.
Platforms Tested
================
AIX 4.3.3.0 (IBM SP powerpc) xlc 5.0.2.0
mpcc_r 5.0.2.0
xlf 07.01.0000.0002
mpxlf 07.01.0000.0002
AIX 4.3 (IBM SP RS6000) C for AIX Compiler, Version 5.0.2.0
xlf 7.1.0.2
poe 3.1.0.12 (includes mpi)
AIX 5.1 xlc 5.0.2.0
xlf 07.01.0000.0002
mpcc_r 5.0.2.0; mpxlf_r 07.01.0000.0002
Cray T3E sn6711 2.0.5.57 Cray Standard C Version 6.5.0.3
Cray Fortran Version 3.5.0.4
Cray SV1 10.0.1.1 Cray Standard C Version 6.5.0.3
Cray Fortran Version 3.5.0.4
FreeBSD 4.6 gcc 2.95.4
g++ 2.95.4
HP-UX B.10.20 HP C HP92453-01 A.10.32.30
HP F90 v2.3
HP-UX B.11.00 HP C HP92453-01 A.11.01.20
HP F90 v2.4
HP-UX B.11.00 SysV HP C HP92453-01 A.11.01.20
HP F90 v2.4
HP MPI [not a product] (03/24/2000) B6060BA
IRIX 6.5 MIPSpro cc 7.30
IRIX64 6.5 (64 & n32) MIPSpro cc 7.3.1.3m
F90 MIPSpro 7.3.1.3m (64 only)
Linux 2.4.9-31smp gcc 2.95.3
g++ 2.95.3
Intel(R) C++ Version 6.0
Intel(R) Fortran Compiler Version 6.0
MPICH 1.2.2
Linux 2.2.18smp gcc 2.95.2
gcc 2.95.2 with mpich 1.2.1
g++ 2.95.2
pgf90 3.2-4
OSF1 V5.1 Compaq C V6.4-014
Compaq Fortran V5.5-1877-48BBF
gcc version 3.0 for C++
SunOS 5.7 WorkShop Compilers 5.0 98/12/15 C 5.0
(Solaris 2.7) WorkShop Compilers 5.0 98/12/15 C++ 5.0
WorkShop Compilers 5.0 98/10/25
FORTRAN 90 2.0 Patch 107356-04
SunOS 5.8/32 Sun WorkShop 6 update 1 C 5.2 2000/09/11
(Solaris 2.8) Sun WorkShop 6 update 1 Fortran 95 6.1
Patch 109503-07 2001/08/11
Sun WorkShop 6 update 1 C++ 5.2 Patch
109508-04 2001/07/11
SunOS 5.8/64 Sun WorkShop 6 update 1 C 5.2 2000/09/11
(Solaris 2.8) Sun WorkShop 6 update 1 Fortran 95 6.1
Patch 109503-07 2001/08/11
Sun WorkShop 6 update 1 C++ 5.2 Patch
109508-04 2001/07/11
TFLOPS r1.0.4 v4.2.2 i386 pgcc Rel 3.1-4i with mpich-1.2.3 with
local modifications
IA-32 Linux 2.4.9 cc Intel 5.0.1
gcc 2.96
Intel(R) C++ Version 6.0
Intel(R) Fortran Compiler Version 6.0
IA-64 Linux 2.4.16 ia64 gcc version 2.96 20000731
Intel(R) C++ Version 6.0
Intel(R) Fortran Compiler Version 6.0
Windows 2000 (NT5.0) MSVC++ 6.0
DEC Visual Fortran 6.0
Windows NT4.0 MSVC++ 6.0
DEC Visual Fortran 6.0
Windows NT4.0 Code Warrior 6.0
Supported Configuration Features Summary
========================================
In the tables below
y = tested and supported
n = not supported or not tested in this release
x = not working in this release
( ) = footnote appears below second table
Platform C C F90 F90 C++ Shared zlib Tools
parallel parallel libraries(5)
Solaris2.6 y n y n y y y y
Solaris2.7 64 y y (1) y n y y y y
Solaris2.7 32 y y (1) y n y y y y
Solaris2.8 64 y n y n y y y y
Solaris2.8 32 y n y n y y y y
IRIX6.5 y y (1) n n n y y y
IRIX64_6.5 64 y y (2) y y n y y y
IRIX64_6.5 n32 y y (2) n n n y y y
HPUX10.20 y n y n n y y y
HPUX11.00 y n y n n y y y
HPUX11 SysV y y y y n y y y
OSF1 v5.1 y n y n y y y y
T3E y y y y n n y y
SV1 y n y n n n y y
TFLOPS n y (1) n n n n y y (4)
AIX-4.3 y y y y y n y y
AIX-5.1 y y y y n n y y
WinNT/2000 y n y n y y y y
WinNT CW y n n n n n y y
FreeBSD y n n n y y y y
Linux 2.2 y y (1) y n y y y y
Linux 2.4 y y (1) n n y y y y
Linux 2.4 Intel(6) y n y n y n y y
Linux 2.4 IA32 y n y n n n y y
Linux 2.4 IA64 y n y n n n y y
Platform 1.2 static- Thread- SRB GASS STREAM-
compatibility exec safe VFD
Solaris2.6 y x y n n y
Solaris2.7 64 y x y n n y
Solaris2.7 32 y x y n n y
Solaris2.8 64 y y n n n y
Solaris2.8 32 y x y n n y
IRIX6.5 y x n n n y
IRIX64_6.5 64 y x y n y y
IRIX64_6.5 n32 y x y n y y
HPUX10.20 y y n n n y
HPUX11.00 y x n n n y
HPUX11 SysV y x n n n y
OSF1 v5.1 y y n n n y
T3E y y n n n y
SV1 y y n n n y
TFLOPS y y n n n n
AIX-4.3 y y (3) n n n y
AIX-5.1 y y n n n y
WinNT/2000 y y n n n n
WinNT CW n n n n n n
FreeBSD y y y n n y
Linux 2.2 y y y n n y
Linux 2.4 y y y n n y
Linux 2.4 Intel(6) y y n n n y
Linux 2.4 IA32 y y n n n y
Linux 2.4 IA64 y y n n n y
Footnotes: (1) Using mpich.
(2) Using mpt and mpich.
(3) When configured with static-exec enabled, tests fail in
serial mode.
(4) No HDF4-related tools.
(5) Shared libraries are provided only for the C library,
except on Windows where they are provided for all languages.
(6) Linux 2.4 with Intel compilers.
Known Problems
==============
* Datasets or attributes which have a variable-length string datatype are
not printing correctly with h5dump and h5ls.
* When a dataset with the variable-length datatype is overwritten,
the library can develop memory leaks that cause the file to become
unnecessarily large. This is planned to be fixed in the next release.
* On the SV1, the h5ls test fails due to a difference between the
SV1 printf precision and the printf precision on other platforms.
* The h5dump tests may fail to match the expected output on some
platforms (e.g. SP2 parallel, Windows) where the error messages
directed to "stderr" do not appear in the "right order" with output
from stdout. This is not an error.
* The --enable-static-exec configure flag fails to compile for HP-UX
11.00 platforms.
* The executables are always dynamic on IRIX64 6.5(64 and n32) and
IRIX 6.5 even if they are configured with --enable-static-exec.
* IRIX 6.5 fails to compile if configured with --enable-static-exec.
* The HDF5_MPI_OPT_TYPES optimization code in the parallel HDF5 will cause
a hang in some cases when chunked storage is used. This is now set to
be off by default. One may turn it on by setting the environment
variable HDF5_MPI_OPT_TYPES to a non-zero value such as 1.
* On IA32 and IA64 systems, if you use a compiler other than GCC (such as
Intel's ecc or icc compilers), you will need to modify the generated
"libtool" program after configuration is finished. On or around line 104
of the libtool file, there are lines which look like:
# How to pass a linker flag through the compiler.
wl=""
Change these lines to this:
# How to pass a linker flag through the compiler.
wl="-Wl,"
* To build the Fortran library using Intel compilers, one has to
x modify the source code in the fortran/src directory to remove the
!DEC and !MS compiler directives.
x The build will fail in the fortran/test directory and then in the
fortran/examples directory; to proceed, edit the work.pcl files in
those directories to contain two lines
work.pc
../src/work.pc
* To build the Fortran library on IA64 use
setenv CC "ecc -DIA64"
setenv F9X "efc -cl,work.pcl"
before running configure and see the steps described above.
