Instructions for the Installation of HDF5 Software
==================================================
This file provides instructions for installing the HDF5 software.
If you have any problems with the installation, please see The HDF Group's
support page at the following location:
http://www.hdfgroup.org/services/support.html
1. Obtaining HDF5
2. Quick installation
2.1. UNIX platforms
2.2. Windows and Cygwin
3. HDF5 dependencies
Szip (optional)
3.4. MPI and MPI-IO
4. Full installation instructions for source distributions
4.1. Unpacking the distribution
4.1.1. Non-compressed tar archive (*.tar)
4.1.2. Gzip'd tar archive (*.tar.gz)
4.1.3. Bzip'd tar archive (*.tar.bz2)
4.2. Source versus build directories
4.3. Configuring
4.3.1. Specifying the installation directories
4.3.2. Using an alternate C compiler
4.3.3. Configuring for 32 or 64-bit support
4.3.4. Additional compilation flags
4.3.5. Compiling HDF5 wrapper libraries
4.3.6. Specifying other programs
4.3.7. Specifying other libraries and headers
4.3.8. Static versus shared linking
4.3.9. Optimization versus symbolic debugging
4.3.10. Parallel versus serial library
4.3.11. Threadsafe capability
4.3.12. Backward compatibility
4.4. Building
4.5. Testing
4.6. Installing HDF5
5. Using the Library
6. Support
A. Building and testing with other compilers
Building and testing with Intel compilers
Building and testing with PGI compilers
*****************************************************************************
1. Obtaining HDF5
The latest supported public release of HDF5 is available from
ftp://ftp.hdfgroup.org/HDF5/current/src.
For Unix and UNIX-like
platforms, it is available in tar format compressed with gzip.
For Microsoft Windows, it is in ZIP format.
The HDF team also makes snapshots of the source code available on
a regular basis. These snapshots are unsupported (that is, the
HDF team will not release a bug-fix on a
rather any bug fixes will be rolled into the next snapshot).
Furthermore, the snapshots have only been tested on a few
machines and may not test correctly for parallel applications.
Snapshots, in a limited number of formats, can be found on The
HDF Group's development FTP server:
ftp://ftp.hdfgroup.uiuc.edu/pub/outgoing/hdf5/snapshots
2. Quick installation
2.1. UNIX platforms
For those who don'-) the following steps can be used
to configure, build, test, and install the HDF5 Library, header files,
and support programs. For example, to install HDF5 version X.Y.Z at
location /usr/local/hdf5, use the following steps.
$ gunzip < hdf5-X.Y.Z.tar.gz | tar xf -
$ cd hdf5-X.Y.Z
$ ./configure --prefix=/usr/local/hdf5
$ make check
# run test suite.
$ make install
$ make check-install
# verify installation.
Some versions of the tar command support the -z option. In such cases,
the first step above can be simplified to the following:
$ tar zxf hdf5-X.Y.Z.tar.gz
Some versions even auto-recognize the file is in compressed format
and will do uncompression automatically. The step above can be
further simplified to the following:
$ tar xf hdf5-X.Y.Z.tar.gz
above refers to the configure flags appropriate
to your installation.
For example, to install HDF5 with the
Fortran and C++ interfaces and with SZIP compression, the
configure line might read as follows:
$ ./configure --prefix=/usr/local/hdf5 --enable-fortran \
--enable-cxx --with-szlib=PATH_TO_SZIP
In this case, PATH_TO_SZIP would be replaced with the path to the
installed location of the SZIP library.
2.2. Windows and Cygwin
Users of Microsoft Windows should see the INSTALL_Windows files for
detailed instructions.
INSTALL_Cygwin also exists for those platforms.
3. HDF5 dependencies
The building of HDF5 Library employs some Gnu Make (gmake)
features. Either gmake or compatible make is needed for the building
and installation of the Library.
The HDF5 Library includes a predefined compression filter that
uses the "deflate" method for chunked datasets. If zlib-1.1.2 or
later is found, HDF5 will use it.
Otherwise, HDF5's predefined
compression method will degenerate to a no- the compression
filter will succeed but the data will not be compressed.
3.3. Szip (optional)
The HDF5 Library includes a predefined compression filter that
uses the extended-Rice lossless compression algorithm for chunked
datasets. For more information about Szip compression and license
terms, see http://hdfgroup.org/doc_resource/SZIP/.
The Szip source code can be obtained from the HDF5 Download page
http://www.hdfgroup.org/HDF5/release/obtain5.html#extlibs. Building
instructions are available with the Szip source code.
The HDF Group does not distribute separate Szip precompiled libraries,
but the HDF5 binaries available from
http://www.hdfgroup.org/HDF5/release/obtain5.html include
the Szip encoder enabled binary for the corresponding platform.
To configure the HDF5 Library with the Szip compression filter, use
the '--with-szlib=/PATH_TO_SZIP' flag. For more information, see
section 4.3.7, "Specifying other libraries and headers."
Please notice that if HDF5 configure cannot find a valid Szip library,
con in this case, the compression filter will
not be available to the applications.
To check if Szip compression was successfully configured in,
check the "I/O filters (external):" line in the configure output,
summary section, printed to the standard output.
3.4. MPI and MPI-IO
The parallel version of the library is built upon the foundation
provided by MPI and MPI-IO. If these libraries are not available
when HDF5 is configured, only a serial version of HDF5 can be built.
4. Full installation instructions for source distributions
4.1. Unpacking the distribution
The HDF5 source code is distributed in a variety of formats which
can be unpacked with the following commands, each of which creates an
'hdf5-X.Y.Z' directory, where X.Y.Z is the HDF5 version numbers.
4.1.1. Non-compressed tar archive (*.tar)
$ tar xf hdf5-X.Y.Z.tar
4.1.2. Gzip'd tar archive (*.tar.gz)
$ gunzip < hdf5-X.Y.Z.tar.gz | tar xf -
$ tar zxf hdf5-X.Y.Z.tar.gz
$ tar xf hdf5-X.Y.Z.tar.gz
4.1.3. Bzip'd tar archive (*.tar.bz2)
$ bunzip2 < hdf5-X.Y.Z.tar.bz2 | tar xf -
$ tar jxf hdf5-X.Y.Z.tar.bz2
$ tar xf hdf5-X.Y.Z.tar.bz2
4.2. Source versus build directories
On most systems the build can occur in a directory other than the
source directory, allowing multiple concurrent builds and/or
read-only source code. In order to accomplish this, one should
create a build directory, cd into that directory, and run the
`configure' script found in the source directory (configure
details are below). For example,
$ mkdir built-fortran
$ cd build-fortran
$ ../hdf5-X.Y.Z/configure --enable-fortran ...
4.3. Configuring
HDF5 uses the GNU autoconf system for configuration, which
detects various features of the host system and creates the
Makefiles. On most systems it should be sufficient to say:
$ ./configure
$ sh configure
The configuration process can be controlled through environment
variables, command-line switches, and host configuration files.
For a complete list of switches type:
$ ./configure --help
The host configuration files are located in the `config'
directory and are based on architecture name, vendor name, and/or
operating system which are displayed near the beginning of the
`configure' output. The host config file influences the behavior
of configure by setting or augmenting shell variables.
4.3.1. Specifying the installation directories
The default installation location is the HDF5 directory created in
the build directory. Typing `make install' will install the HDF5
Library, header files, examples, and support programs in hdf5/lib,
hdf5/include, hdf5/doc/hdf5/examples, and hdf5/bin. To use a path
other than hdf5, specify the path with the `--prefix=PATH' switch:
$ ./configure --prefix=/usr/local
If shared libraries are being built (the default), the final
home of the shared library must be specified with this switch
before the library and executables are built.
HDF5 can be installed into a different location than the prefix
specifi see section 4.6, "Installing HDF5,"
for more details.
4.3.2. Using an alternate C compiler
By default, configure will look for the C compiler by trying
`gcc' and `cc'. However, if the environment variable "CC" is set
then its value is used as the C compiler.
For instance, one would
use the following line to specify the native C compiler on a system
that also has the GNU gcc compiler (users of csh and derivatives
will need to prefix the commands below with `env'):
$ CC=cc ./configure
A parallel version of HDF5 can be built by specifying `mpicc' as
the C compiler.
Using the `mpicc' compiler will insure that the
correct MPI and MPI-IO header files and libraries are used.
$ CC=/usr/local/mpi/bin/mpicc ./configure --enable-parallel
4.3.3. Configuring for 64 or 32 bit support
Some machine architectures support 32-bit or 64-bit binaries.
The options below describe how to enable support for different options.
Users compiling on 64-bit Linux systems may generate 32-bit binary
with the following flags:
$ CC='gcc -m32' ./configure
$ CFLAGS='-m32' ./configure
Users compiling on older Solaris machines using the Sun compiler and
desiring to build the distribution with 64-bit support may need to
specify the compiler 'cc' with the appropriate flag:
$ CC='cc -m64' ./configure
To configure AIX 64-bit support including the Fortran and C++ APIs,
(Note: need to set $AR to 'ar -X 64'.)
$ CFLAGS=-q64 FCFLAGS=-q64 CXXFLAGS=-q64 AR='ar -X 64'\
./configure --enable-fortran
Parallel: (C++ not supported with parallel)
$ CFLAGS=-q64 FCFLAGS=-q64 AR='ar -X 64'\
./configure --enable-parallel --enable-fortran
4.3.4. Additional compilation flags
If addtional flags must be passed to the compilation commands,
specify those flags with the CFLAGS variable. For instance,
to enable symbolic debugging of a production version of HDF5, one
might say:
$ CFLAGS=-g ./configure --enable-production
4.3.5. Compiling HDF5 wrapper libraries
One can optionally build the Fortran and/or C++ interfaces to the
HDF5 C library. By default, both options are disabled. To build
them, specify `--enable-fortran' and `--enable-cxx', respectively.
$ ./configure --enable-fortran
$ ./configure --enable-cxx
Additionally, --enable-fortran --enable-fortran2003 enables Fortran
2003 APIs. Configuration will halt if a working Fortran 90/95 compiler
(or a working Fortran 2003 compiler in the case of
--enable-fortran2003) was specified or C++ compiler is not found.
Currently, the Fortran configure tests for these compilers in order:
gfortran ifort pgf90 pathf90 pathf95 xlf90 xlf95 xlf2003 f90 epcf90 f95
fort lf95 g95 ifc efc gfc. To use an alternate compiler specify it with
the FC variable:
$ FC=/usr/local/bin/gfortran ./configure --enable-fortran --enable-fortran2003
Note: The Fortran and C++ interfaces are not supported on all the
platforms the main HDF5 Library supports. Also, the Fortran
interface supports parallel HDF5 while the C++ interface does
4.3.6. Specifying other programs
The build system has been tuned for use with GNU make which is
the preferred version. Other versions of make may or may not work
completely.
If the `make' command runs a non-GNU version but a GNU
version is available under a different name (perhaps `gmake'), then
HDF5 can be configured to use it by setting the MAKE variable. Note
that whatever value is used for MAKE must also be used as the make
command when building the library:
$ MAKE=gmake ./configure
The `AR' and `RANLIB' variables can also be set to the names of
the `ar' and `ranlib' (or `:') commands to override values
detected by configure.
The HDF5 Library, include files, and utilities are installed
during `make install' (described below) with a BSD-compatible
install program detected automatically by configure. If none is
found, the shell script bin/install-sh is used. Configure does not
check that the install s if a bad install is
detected on your system you have two choices:
1. Copy the bin/install-sh program to your $HOME/bin
directory, name it `install', and make sure that $HOME/bin
is searched before the system bin directories.
2. Specify the full path name of the `install-sh' program
as the value of the INSTALL environment variable. Note: do
not use `cp' or some other program in place of install
because the HDF5 makefiles also use the install program to
change file ownership and/or access permissions.
4.3.7. Specifying other libraries and headers
Configure searches the standard places (those places known by the
systems compiler) for include files and header files. However,
additional directories can be specified by using the CPPFLAGS
and/or LDFLAGS variables:
$ CPPFLAGS=-I/home/robb/include \
LDFLAGS=-L/home/robb/lib \
./configure
HDF5 uses the zlib library to support the HDF5 deflate
data compression filter.
Configure searches the standard places
(plus those specified above with the CPPFLAGS and LDFLAGS variables)
for the zlib headers and library. The search can be disabled by
specifying `--without-zlib' or alternate directories can be specified
with `--with-zlib=INCDIR,LIBDIR' or through the CPPFLAGS and LDFLAGS
variables:
$ ./configure --with-zlib=/usr/unsup/include,/usr/unsup/lib
$ CPPFLAGS=-I/usr/unsup/include \
LDFLAGS=-L/usr/unsup/lib \
./configure
HDF5 includes Szip as a predefined compression method (see 3.3).
To enable Szip compression, the HDF5 Library must be configured
and built using the Szip Library:
$ ./configure --with-szlib=/Szip_Install_Directory
4.3.8. Static versus shared linking
The build process will create static libraries on all systems and
shared libraries on systems that support dynamic linking to a
sufficient degree. Either form of the library may be suppressed by
saying `--disable-static' or `--disable-shared'.
$ ./configure --disable-shared
Shared C++ and Fortran libraries will be built if shared libraries
are enabled.
To build only statically linked executables on platforms which
support shared libraries, use the `--enable-static-exec' flag.
$ ./configure --enable-static-exec
4.3.9. Optimization versus symbolic debugging
The library can be compiled to provide symbolic debugging support
so it can be debugged with gdb, dbx, ddd, etc., or it can be
compiled with various optimizations. To compile for symbolic
debugging (the default for snapshots), say `--disable-production';
to compile with optimizations (the default for supported public
releases), say `--enable-production'. On some systems the library
can also be compiled for profiling with gprof by saying
`--enable-production=profile'.
$ ./configure --disable-production
#symbolic debugging
$ ./configure --enable-production
#optimized code
$ ./configure --enable-production=profile
#for use with gprof
Regardless of whether support for symbolic debugging is enabled,
the library can also perform runtime debugging of certain packages
(such as type conversion execution times and extensive invariant
condition checking). To enable this debugging, supply a
comma-separated list of package names to to the `--enable-debug'
See "Debugging HDF5 Applications" for a list of package
http://www.hdfgroup.org/HDF5/doc/H5.user/Debugging.html
Debugging can be disabled by saying `--disable-debug'.
The default debugging level for snapshots is a subset of the
the default for supported releases is no
debugging (debugging can incur a significant runtime penalty).
$ ./configure --enable-debug=s,t
#debug only H5S and H5T
$ ./configure --enable-debug
#debug normal packages
$ ./configure --enable-debug=all
#debug all packages
$ ./configure --disable-debug
#no debugging
HDF5 can also print a trace of all API function calls, their
arguments, and the return values. To enable or disable the
ability to trace the API say `--enable-trace' (the default for
snapshots) or `--disable-trace' (the default for public releases).
The tracing must also be enabled at runtime to see any output
(see "Debugging HDF5 Applications," reference above).
4.3.10. Parallel versus serial library
The HDF5 Library can be configured to use MPI and MPI-IO for
parallelism on a distributed multi-processor system.
file INSTALL_parallel for detailed explanations.
4.3.11. Threadsafe capability
The HDF5 Library can be configured to be thread-safe (on a very
large scale) with the `--enable-threadsafe' flag to the configure
Some platforms may also require the '-with-pthread=INC,LIB'
(or '--with-pthread=DIR') flag to the configure script.
For further details, see "HDF5 Thread Safe Library":
http://www.hdfgroup.org/HDF5/doc/TechNotes/ThreadSafeLibrary.html
4.3.12. Backward compatibility
The 1.8 version of the HDF5 Library can be configured to operate
identically to the v1.6 library with the
--with-default-api-version=v16
configure flag. This allows existing code to be compiled with the
v1.8 library without requiring immediate changes to the application
source code.
For addtional configuration options and other details,
see "API Compatibility Macros in HDF5":
http://www.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html
4.4. Building
The library, confidence tests, and programs can be built by
saying just:
Note that if you have supplied some other make command via the MAKE
variable during the configuration step, that same command must be
used here.
When using GNU make, you can add `-j -l6' to the make command to
compile in parallel on SMP machines. Do not give a number after
the `-j' since GNU make will turn it off for recursive invocations
$ make -j -l6
4.5. Testing
HDF5 comes with various test suites, all of which can be run by
$ make check
To run only the tests for the library, change to the `test'
directory before issuing the command. Similarly, tests for the
parallel aspects of the library are in `testpar' and tests for
the support programs are in `tools'.
The `check' consists of two sub-tests, check-s and check-p, which
are for serial and parallel tests, respectively.
Since serial tests
and parallel tests must be run with single and multiple processes
respectively, the two sub-tests work nicely for batch systems in
which the number of processes is fixed per batch job.
One may submit
one batch job, requesting 1 process, to run all the serial tests by
"make check-s"; and submit another batch job, requesting multiple
processes, to run all the parallel tests by "make check-p".
Temporary files will be deleted by each test when it completes,
but may continue to exist in an incomplete state if the test
fails. To prevent deletion of the files, define the HDF5_NOCLEANUP
environment variable.
The HDF5 tests can take a long time to run on some systems.
To perform
a faster (but less thorough) test, set the HDF5TestExpress environment
variable to 2 or 3 (with 3 being the shortest run).
To perform a
longer test, set HDF5TestExpress to 0.
1 is the default.
4.6. Installing HDF5
The HDF5 Library, include files, and support programs can be
installed in a (semi-)public place by saying `make install'. The
files are installed under the directory specified with `--prefix=DIR'
(default is 'hdf5' in the build directory) in directories named `lib',
`include', and `bin'. The directories, if not existing, will be created
automatically, provided the mkdir command supports the -p option.
If `make install' fails because the install command at your site
somehow fails, you may use the install-sh that comes with the
source. You will need to run ./configure again.
$ INSTALL="$PWD/bin/install-sh -c" ./configure ...
$ make install
If you want to install HDF5 in a location other than the location
specified by the `--prefix=DIR' flag during configuration (or
instead of the default location, `hdf5'), you can do that
by running the deploy script:
$ bin/deploy NEW_DIR
This will install HDF5 in NEW_DIR.
Alternately, you can do this
manually by issuing the command:
$ make install prefix=NEW_DIR
where NEW_DIR is the new directory where you wish to install HDF5.
If you do not use the deploy script, you should run h5redeploy in
NEW_DIR/bin directory.
This utility will fix the h5cc, h5fc and
h5c++ scripts to reflect the new NEW_DIR location.
The library can be used without installing it by pointing the
compiler at the `src' and 'src/.libs' directory for include files and
libraries. However, the minimum which must be installed to make
the library publicly available is:
The library:
./src/.libs/libhdf5.a
The public header files:
./src/H5*public.h, ./src/H5public.h
./src/H5FD*.h except ./src/H5FDprivate.h,
./src/H5api_adpt.h
The main header file:
./src/hdf5.h
The configuration information:
./src/H5pubconf.h
The support programs that are useful are:
./tools/h5ls/h5ls
(list file contents)
./tools/h5dump/h5dump
(dump file contents)
./tools/misc/h5repart
(repartition file families)
./tools/misc/h5debug
(low-level file debugging)
./tools/h5import/h5import
(imports data to HDF5 file)
./tools/h5diff/h5diff
(compares two HDF5 files)
./tools/gifconv/h52gif
(HDF5 to GIF converter)
./tools/gifconv/gif2h5
(GIF to HDF5 converter)
5. Using the Library
Please see the "HDF5 User's Guide" and the "HDF5 Reference Manual":
http://www.hdfgroup.org/HDF5/doc/
Most programs will include
and link with -lhdf5.
Additional libraries may also be necessary depending on whether
support for compression, etc., was compiled into the HDF5 Library.
A summary of the HDF5 installation can be found in the
libhdf5.settings file in the same directory as the static and/or
shared HDF5 Libraries.
6. Support
Support is described in the README file.
*****************************************************************************
APPENDICES
*****************************************************************************
A. Building and testing with other compilers
Building and testing with Intel compilers
When Intel compilers are used (icc or ecc), you will need to modify
the generated "libtool" program after configuration is finished.
On or around line 104 of the libtool file, there are lines which
look like:
# How to pass a linker flag through the compiler.
Change these lines to this:
# How to pass a linker flag through the compiler.
UPDATE: This is now done automatically by the configure script.
However, if you still experience a problem, you may want to check this
line in the libtool file and make sure that it has the correct value.
Building and testing with PGI compilers
When PGI C and C++ compilers are used (pgcc or pgCC), you will need to
modify the generated "libtool" program after configuration is finished.
On or around line 104 of the libtool file, there are lines which
look like this:
# How to pass a linker flag through the compiler.
Change these lines to this:
# How to pass a linker flag through the compiler.
UPDATE: This is now done automatically by the configure script. However,
if you still experience a problem, you may want to check this line in
the libtool file and make sure that it has the correct value.
To build the HDF5 C++ Library with pgCC (version 4.0 and later), set
the environment variable CXX to "pgCC -tlocal"
setenv CXX "pgCC -tlocal"
before running the configure script.