Description of VisIt libraries

The various libraries that VisIt uses can probably be grouped into two or three kinds. The first might be called core libraries in that VisIt cannot operate at all (or at least not to a minimum level of service users have come to expect) without them. Examples of the core libraries in VisIt are Qt, VTK, Mesa and/or Gl. These libraries are used either throughout VisIt or are essential to one or more of VisIt's main executable components. For example, the GUI and the viewer both require Qt. Are MPI or Python core libraries? In theory, no they are not. VisIt can be compiled and used without either MPI or Python. However, in practice, these libraries are indeed essential to utilize many of VisIt's useful features. MPI is required for dealing with big data. Python is required for doing sophisticated data analysis or movie making.

A second kind of library VisIt uses might be called optional in that it is non-essential for VisIt to operate to the minimum level of functionality users commonly expect. Typically, optional libraries are used in a much more focused way within VisIt. For example, they may be specific to a certain (plot, operator or database) plugin or to a given feature. For example, the Silo library is specific to the Silo database plugin. The hdf5 library is specific to the various hdf5 database plugins VisIt supports. In addition, some optional libraries may be more for the purposes of facilitating development rather than providing new features to users. Examples of these include tcmalloc library or the google perftools library.

A third kind of library VisIt uses is known as a bultin library. The key different between builtin libraries and the other two is that builtin libraries are, for better or worse, part of VisIt's source code tree. They are in the third_party_builtin directory. Typically, but not always, builtin libraries get compiled whenever VisIt gets compiled. Builtin libraries are for functionality that is considered too important to risk being left out of an installation and that have a high probability of compiling without error everywhere VisIt needs to be compiled. Nonetheless, although builtin libraries are compiled with VisIt by default, they can also be disabled. Examples of builtin libraries are zlib and glew.

One thing that is common to core, optional and builtin libraries is that they are not system libraries. By this, we mean we do not assume that the specific (version of the) library is going to be available as part of the set of libraries that come standard on the system VisIt is being compiled on. So, in this sense, core, optional and builtin libraries are all treated as third party libraries in that development and support of these libraries is handled outside of VisIt development. For convenience, release tarballs for most if not all of the third party libraries VisIt uses are maintained in the top-level, third_party directory, of the VisIt repository.

The preceding paragraph hints at the fact that there exists a fourth kind of library VisIt uses and that is the various system libraries such as the math library (-lm), dynamic linking library (-ldl), and socket library (-lsock). Indeed, VisIt does use such libraries. However, we assume these libraries are present as part of the standard enviornment in which VisIt is compiled and so do not, for example, maintain release tarballs for such libraries that users can download, compile and install.

Why can't I use the Qt, Python, VTK, Mesa/Gl, etc. that came on my system?

As much as we might like to believe it, large, complex libraries like Qt, Python and VTK are rarely 100% compatible between newer or older versions. Furthermore, for large libraries like these, there are often many, many different installation options for a given platform. It is highly unlikely that a given installation of VTK for example, is not only of a version compatibile with a given release of VisIt but also configured and installed on your system in exactly the way VisIt needs it. In addition, VisIt gets developed and thoroughly tested on a given version and configuration of a library. In some cases, VisIt developers have had to work-around a bug in a library or, worse, had to patch the actual library code itself to address an issue that might be specific to just one platform. Together, these issues result in a situation where VisIt must be compiled with precisely the libraries it is released on and rarely, if ever, can take advantage of an installation that came as part of the system VisIt is being built on. Lastly, it becomes almost impossible to duplicate buggy behavior and diagnose problems reported by users when users are running VisIt in a configuration that is substantially different from that which it is being developed upon.


A short overview of the variety of issues related to third party libraries

This section is far from complete. In its current form, it serves only to describe the wide variety of issues related to the configuration logic necessary to support various third party libraries.

Most but not all of VisIt's third party libraries define their own <libname>_INCLUDE and <libname>_LIB symbols (some use plural <libname>_INCLUDES and/or <libname>_LIBS; some use <libname>_FLAGS instead of <libname>_INCLUDE) for the include directives and linker directives. However, better names for these would be <libname>_CPPFLAGS and <libname>_LDFLAGS as they are really the flags necessary for the C pre-processor or the linker (ld).

The file extension used by libraries varies based on the type of library (static or dynamic) and on the platform. .dylib is the extension on a Mac while .so and .a on Linux variants. Though we don't deal with Windows platforms via our configure script, .dll is the extension on windows.

Some libraries, like zlib, are used by VisIt directly and are required to link various VisIt executables. In the case of zlib, it is used to support compression between engine and viewer and is therefore needed to link the engine and viewer. We have seen situations on AIX where we need to include direct references to a symbol in zlib in the viewer and engine main.C file in order for AIX to get correct linkage to zlib.

Ordinarily, zlib comes with the system and so is a system library. However, we have encountered systems where it is either not available or not configured correctly. So, zlib is also a builtin library. In addition, we allow users to configure VisIt by specifying an arbitrary intstallion of zlib. Because zlib is either a system or builtin library, there are no ZLIB_INCLUDE/ZLIB_LIB make variables defined for it. Instead, it is expected to be handled via the more conventional CPPFLAGS/LIBS variables. When VisIt is built using the system zlib, it need only add '-lz' to LIBS variable and need not touch CPPFLAGS because zlib.h is in one of the standard include directories (/usr/include) used by the compiler. When VisIt is built using zlib builtin, libz.a will get created in VisIt's lib directory and zlib.h will be placed in VisIt's include directory. Again, all that is required is to add '-lz' to the LIBS variable. When VisIt is built with an arbitrary zlib installation, the include dir for zlib.h needs to get added to CPPFLAGS and '-lz' needs to get added to LIBS variable.

A common practice when installing libraries is to install both the static (.a) and dynamic (.so) library file in the same directory. Linkers favor using the dynamic (.so) over the static (.a) file and it can often be difficult to force a linker to use the static (.a) file. However, if the static file is not used, then any executable resulting from a link to the .so needs to be able to find the .so file when the executable is loaded. This can wind up causing users to have to carefully manage the LD_LIBRARY_PATH environment variable. A number of things can go wrong.

The binary may not work correctly on binary compatible platforms connected via a cross-mounted filesystem. Both platforms may be able to see the executable but only one may be able to correctly find its dynamic libraries. The library can later be (re)moved either intentionally or accidentally causing all executables that depend on it to no longer operate. The executable itself cannot easily be packaged up and re-distributed in binary form to another compatible platform. A partial for many remedy for many of these issues is to use the rpath option of the dynamic linker which can be used to embed in the executable knowledge of where to go look to find the library. However, if the executable (or library) needs to be moved, this does not work.

To address these issues, VisIt has taken to linking to third party libraries by specifying the entire path to a specific .a file as a file to the linker command itself. This has the affect of forcing the linker to link to the library statically. This does not mean the resultant object from the link must be static. It means only that all of that object's references to the library are resolved and compiled into the resultant (possibly shared) object. The downside of this is that for a library like HDF5, which has grown dramatically in size in the past several years and which is used by a number of VisIt's database plugins, it is getting copied into each and every plugin resulting in much larger binary distributions for VisIt.

Most libraries are installed such that include, lib and bin are common subdirectories at the installation point. Thus, there is no reason and in fact it is more error prone to separately specify paths to the include and bin directories. On the other hand, such a naming scheme is not uniform. Exodus uses inc instead of include. Mili installations have had both the header and library files placed in the same directory.

An important library used by VisIt at LLNL, Silo, is often installed with a variety of names; silo, siloh5 and more recently siloxx. The intent of the different installed naming schemes for Silo was to keep users from having to modify their makefiles if they did not want to use newer features of the Silo library. For example, when an HDF5 enabled version of Silo was released, a new name, siloh5 was used for that installation. For better or worse, this naming scheme has proliferated. In retrospect, since it is much more common for LLNL Silo users to create their own installations of Silo, this is probably no longer that useful.

Some libaries are needed to completely link VisIt only because they are used indirectly by a library VisIt uses. szip is an example. Nowhere does VisIt reference or use szip directly. However, VisIt does use HDF5 directly and it is common for HDF5 to be installed with szip compression features. Prior to integration of zlib with VisIt, zlib was another example of this kind of situation.

In the world of I/O libraries, it is quite common to have multiple levels of interdependence between libraries. For example, Exodus requires NetCDF. NetCDF can be installed with dependence on HDF5. I think the newest NetCDF libraries will be built by default on top of HDF5. The Mili library has been implemented on top of Silo. Silo in turn uses HDF5 and PDB (Portable Database). CGNS can be built with HDF5, etc., etc. In the case of Exodus, more than likely it requires a specific, older, version of NetCDF and more than likely the version it requires is different from the NetCDF VisIt might use through the NetCDF database plugin. That is ok as long as no more than one is dynamically linked. Otherwise, with two or more database plugins dynamically linked with a libnetcdf.so, it will be impossible for VisIt to know which is needed where.

I often see libraries installed with funny numbers after their names like libhdf5.so.5.0.0 and libhdf5.so.5.0. I have no idea where these funny numbers come from, how they are generated or what they mean. Perhaps they can be used to resolve just the problem alluded to in the preceding paragraph; multiple different versions of a library needed by different plugins within the same executable.

Typically, a given library involves a single library file (.a/.so/.dylib). However, sometimes a given library is an ensemble of library files which may or may not all be in the same directory. So, in general, compiling with a given library involves an arbitrary number of -I<path> include directives and linking to a given library involves an arbitrary number of -L<path>/-l<name> linker directives.

How does information regarding libraries make its way through to VisIt's Makefiles?

VisIt uses CMake to generate a build system from the build logic described in VisIt's CMakeLists.txt and assorted helper files. In order for VisIt to find various libraries, it needs to be told where to find them during the configuration process. There are several ways in which CMake can be told about the existence and location of various libraries.

These are:

  • command-line options to CMake (e.g. -DVISIT_HDF5_DIR:PATH=/path/to/hdf5)
  • variables specified in a host.cmake file in the src/config-site directory