Updating 3rd Party Libraries on Windows

VisIt's svn repository contains prebuilt libraries for various versions of Microsoft Visual Studio to make building VisIt easier, as building 3rd party libraries for Windows is the most complex part of building VisIt. With this step done for you, you can concentrate on building VisIt itself. The svn repository usually contains prebuilt libraries for Visual Studio 2005, and 2008. Visual Studio 2010 support is underway but not generally available.

If you do not develop on Windows then you can ignore this page.

If you find yourself using a compiler for which there are no prebuilt 3rd party libraries, you are in a very difficult situation. Few of the 3rd party packages that VisIt uses provide easy to use build systems for Windows, if they provide a build system for Windows at all. This page will try to provide the steps needed to build various packages with a new Visual Studio compiler and show how to contribute those libraries back into VisIt's svn repository so the next developer using the new compiler won't be stuck too.

Building the 3rd party libraries will follow the order listed on this page.

Background

The VisIt svn repository is broken down into different top-level directories such as data, src, third_party, and windowsbuild. The compiled 3rd party libraries for Windows are stored in windowsbuild/ThirdParty. When we build the 3rd party libraries with a new compiler, we build them and stick them into subdirectories under windowsbuild/ThirdParty/<package>. If a package provides a CMake build system then we typically try to use that system to install the package into the ThirdParty directory. This means that the resulting installation will contain all headers, libraries, etc.

Example:

Thirdparty/
          /vtk/
              /6.0.0/
                    /MSVC90/
                           /bin
                           /lib
                           /include
                    /MSVC2010-x64/
                                 /bin
                                 /lib
                                 /include

For other packages where there isn't an installation target, we are a little looser with the directory organization. We often will have a single top level include directory and then have different compiler version subdirectories under the lib directory. This organization saves on files that we need to keep in the repository since there is just one copy of the header files.

Example:

ThirdParty/
          /silo_x64/
                   /include/
                   /lib/
                       /MSVC90/
                       /MSVC2010-x64/

Required

These packages are required to build VisIt.

Mesa

You can skip building this library. It is normally required on other platforms so read on.

Mesa is normally the first library that one would build since it is usually a prerequisite to building VTK. Mesa's build system is hopeless. The UNIX Makefile build cannot be used and neither can the SCons-based system. The SCons build system is purported to work on Windows but not in this developer's experience. While the build does start, it ultimately fails because of reliance on Linux tools such as flex/bison. It must not get tested much. I've tried to CMake-ify Mesa but it is complex and I gave up -- not because of CMake but because of all of the challenges presented by the Mesa source code itself.

Mesa would be well served to start over with a CMake build system and to:

  1. Not use Linux-isms or the newest compiler stuff (#include <stdbool.h> -- BAH!)
  2. Not rely on external tools like flex/bison
  3. Separate Mesa into a simple software renderer (where it started) and move the drivers and all that junk into some higher level library so users who simply want a library containing gl functions can build it easily.
  4. Provide Visual Studio C files with inline assembly instead of separate .S files that can't be used

The end result is that we're not using Mesa on Windows so save yourself the pain and don't bother trying to build it.

Qt

Qt is actually a fairly easy package to build because its developers value cross-platform builds. Once you unzip the source code, open a command line shell that has your build environment set up (e.g. Visual Studio x64 Win64 Command Line). Navigate to the directory where you unzipped the code and run the configure.exe program.

configure.exe -platform win32-msvc2010 -shared -no-qt3support -nomake docs \
              -nomake examples -nomake demos -opensource -confirm-license -no-libmng
nmake
  • Visual Studio 2012 can't build Qt 4.8.3's WebKit so add -no-webkit to the configure command line.

Once the Qt libraries build, you can create a new place to put them under trunk/windowsbuild/ThirdParty/Qt-4.7.4. For example, I'm building using Visual Studio 2010 and I'm building 64-bit so I'll create the trunk/windowsbuild/ThirdParty/Qt-4.7.4/MSVC2010-x64 directory. In that directory, I'll copy includes, binaries, and libraries. Have a look at some of the previous compiler directories to see what you need to copy and structure the new directory in the same way.

Python

Python comes with some Windows project files and a solution in its PCbuild directory. You can open up the solution and the projects will be converted to the latest version of Visual Studio. When you open the solution in Visual Studio, open the Configuration manager dialog and choose "Release" and "x64" for the platform. Build the solution. Not all projects will finish successfully but the ones we care about will complete.

Create a new trunk/windowsbuild/ThirdParty/python-2.7.2/MSVC2012-x64 directory. Copy the include and Libs directories from the Python sources into it. Make DLLs and libs directories and copy the dlls and import libraries into them from your Python build. Also copy the built Python interpreters and main DLL into the top level directory. Looking at installations for other compilers will make it apparent which files you'll need to copy as well as where you should put them.

VTK

Unzip the VTK source code. This will make a directory called VTK6.0.0. Next to that directory, create a VTK6.0.0-build directory in which to build VTK. You will also want to create a destination directory into which VTK can install. I created one called trunk/windowsbuild/ThirdParty/vtk/6.0.0/MSVC2012-x64 to contain a 64-bit build from Visual Studio 2012.

Run the cmake gui and point it to your VTK6.0.0 and VTK6.0.0-build directories for source and build, respectively. Click the configure button to make the VTK build process set up the rest of its variables. CMake will ask you which compiler you want to target. Your compiler will probably be in the list. You'll also want to disable some packages and point the build to your Python installation in trunk/windowsbuild/ThirdParty. It is important to get these settings right!

Settings:

  • BUILD_SHARED_LIBS:BOOL=true
  • BUILD_TESTING:BOOL=false
  • BUILD_DOCUMENTATION:BOOL=false
  • CMAKE_INSTALL_PREFIX:PATH=<path>/windowsbuild/ThirdParty/vtk/6.0.0/MSVC2012-x64
  • VTK_USE_64BIT_IDS:BOOL=true

Allow VisIt to override any of vtk's classes:

  • VTK_ALL_NEW_OBJECT_FACTORY:BOOL=true

Turn off module groups:

  • VTK_Group_Imaging:BOOL=false
  • VTK_Group_MPI:BOOL=false
  • VTK_Group_Qt:BOOL=false
  • VTK_Group_Rendering:BOOL=false
  • VTK_Group_StandAlone:BOOL=false
  • VTK_Group_Tk:BOOL=false
  • VTK_Group_Views:BOOL=false

Turn on individual modules. dependent modules are turned on automatically:

  • Module_vtkCommonCore:BOOL=true
  • Module_vtkFiltersFlowPaths:BOOL=true
  • Module_vtkFiltersHybrid:BOOL=true
  • Module_vtkFiltersModeling:BOOL=true
  • Module_vtkGeovisCore:BOOL=true
  • Module_vtkIOEnSight:BOOL=true
  • Module_vtkIOGeometry:BOOL=true
  • Module_vtkIOLegacy:BOOL=true
  • Module_vtkIOPLY:BOOL=true
  • Module_vtkIOXML:BOOL=true
  • Module_vtkInteractionStyle:BOOL=true
  • Module_vtkRenderingAnnotation:BOOL=true
  • Module_vtkRenderingFreeType:BOOL=true
  • Module_vtkRenderingFreeTypeOpenGL:BOOL=true
  • Module_vtkRenderingOpenGL:BOOL=true
  • Module_vtklibxml2:BOOL=true

Tell VTK where to locate qmake if we're building graphical support:

  • Module_vtkGUISupportQtOpenGL:BOOL=true
  • QT_QMAKE_EXECUTABLE:FILEPATH=${QT_BIN_DIR}/qmake

Add python wrapping:

  • VTK_WRAP_PYTHON:BOOL=true
  • PYTHON_EXECUTABLE:FILEPATH=<path>/windowsbuild/ThirdParty/python-2.7.3/python.exe
  • PYTHON_INCLUDE_DIR:PATH=<path>/windowsbuild/ThirdParty/python-2.7.3/include
  • PYTHON_LIBRARY:FILEPATH=<path>/windowsbuild/ThirdParty/python-2.7.3/MSVC2012-x64/lib/python27.lib

Once you've gone through a couple of rounds of clicking the Configure button, the generate button will be activated. Click it to generate the projects and solution file for VTK.

  1. Run Visual Studio
  2. Open the Project->Build Configuration dialog and choose "Release" and "x64" for the platform.
  3. Build the solution.
  4. Right click on the INSTALL project in the solution explorer and build it to make Visual Studio install VTK to your installation directory.
  5. Move all of the contents of the bin directory into the lib directory and remove the bin directory.

Recommended

These packages are recommended but they are not needed to complete a build of VisIt.

MPI

You can install MPI for Windows if you want to create a parallel build of VisIt. You can get the required packages from:

zlib

  1. Use zlib-1.2.8 available in the VisIt repo: zlib-1.2.8.
  2. Untar the zlib-1.2.8.tar.gz file to make the zlib-1.2.8 directory.
  3. Make a zlib-1.2.8-build directory.
  4. Make an installation directory in your windowsbuild/Thirdparty/zlib directory. Call it 1.2.8 directory. All zlib-related files will be installed here.
  5. Run CMake
  6. Set the INSTALL_BIN_DIR, INSTALL_INC_DIR, INSTALL_LIB_DIR, INSTALL_MAN_DIR, INSTALL_PKGCONFIG_DIR entries in CMake so the bin, include, lib directories will be installed under your new installation directory.
  7. Configure/Generate in CMake.
  8. Quit CMake
  9. Open Visual Studio
  10. Use the Configuration Manager to select Release/x64 build.
  11. Edit the properties for zlib so it will be called zlib1.dll, zlib1.lib.
  12. Build the ALL_BUILD target and then the INSTALL target.
  13. Quit Visual Studio
  14. Open Windows Explorer and navigate to your zlib installation directory.
  15. Delete the Release, share, and man directories.
  16. Move the contents of bin into lib and delete bin.

szip

Follow the instructions in the szip VisIt README] file.

HDF5

HDF5 1.8.7 has a CMake build so unzip the sources to make a hdf5-1.8.7 directory. Create a hdf5-1.8.7-build directory in which to build the sources. Run CMake and set it up to use the HDF5-related source and build directories. Click the configure button to start configuring CMake.

Settings:

  • BUILD_SHARED_LIBS:BOOL=ON
  • CMAKE_INSTALL_PREFIX:PATH=/path/to/trunk/windowsbuild/ThirdParty/hdf5-1.8.7/MSVC2010-x64
  • HDF5_BUILD_CPP_LIB:BOOL=ON
  • HDF5_BUILD_HL_LIB:BOOL=ON
  • HDF5_BUILD_TOOLS:BOOL=ON
  • HDF5_ENABLE_SZIP_SUPPORT:BOOL=ON
  • HDF5_ENABLE_Z_LIB_SUPPORT:BOOL=ON

Click the configure button.

Settings:

  • SZIP_INCLUDE_DIR:PATH=/path/to/trunk/windowsbuild/ThirdParty/szip/include
  • SZIP_LIBRARY_RELEASE:FILEPATH=/path/to/trunk/windowsbuild/ThirdParty/szip/lib/MSVC2010-x64/szlibdll.lib
  • ZLIB_INCLUDE_DIR:PATH=/path/to/trunk/windowsbuild/ThirdParty/zlib/include
  • ZLIB_LIBRARY:FILEPATH=/path/to/trunk/windowsbuild/ThirdParty/zlib/lib/MSVC2010-x64/zlib1.lib

Click the configure button. Click the generate button.

Open the hdf5-1.8.7-build/HDF5 solution file in Visual Studio. Open the Configuration manager dialog and select "Release" and "x64" for the platform. Build the solution. Then build the INSTALL target.

Silo

Silo sources are available at Silo's web site. Unzip the silo-4.8 sources to make a silo-4.8 directory.

In the SiloWindows subfolder, there are projects for Visual Studio 2005. Unfortunately, these files are incomplete, and having Visual Studio automatically convert to a new version is problematic. Project files for Visual Studio 2010 have been created and are available from VisIt's repo. Only configurations for shared libraries with HDF5 support are in these projects. Full VS 2010 support (including static libs, and non-hdf5 supported libs) will be available in the next release of Silo.

  1. Obtain silo-4.8-VS2010-projects.zip file.
  2. Extract the contents to silo's SiloWindows folder. It will want to overwrite copysilo.bat, runsilexmoc.bat and the README. Allow the overwrite.
  3. Follow the directions as specified in the new README.
  4. If you built a 64-bit version, make a new directory trunk/windowsbuild/ThirdParty/silo_x64/lib/MSVC2010-x64 and copy the silohdf5.dll and silohdf5.lib files into it.
  5. If you built a 32-bit version, make a new directory trunk/windowsbuild/ThirdParty/silo/lib/MSVC2010 and copy the silohdf5.dll and silohdf5.lib files into it.

Optional

These packages are optional for building VisIt. They are primarily I/O libraries.

jpeg

Here are some steps for building jpeg. You must follow all steps if you want to build jpeg as a DLL. The stock project files will produce a static library by default.

Instructions taken in part from original post.

  • Download the source from the IJG website, unzip it, and then follow the instructions in the install.txt file. The relevant portions for Visual Studio 2010 are:
  • Open the command prompt, change to the main directory and execute the command line
NMAKE /f makefile.vc  setup-v10
  • Open the solution file jpeg.sln
  • Open the jpeg project properties and change some settings:
    • Go to the Configuration Properties/General section.
      • Change the Configuration Type to Dynamic Library (.dll)
      • Change the TargetName from $(ProjectName) to lib$(ProjectName)
    • Go to the C/C++/Preprocessor section.
      • Add 'JPEG_CREATE_LIBRARY' preprocessor definition
  • Edit the jmorecfg.h header file around line 194 and change these lines:
/* a function referenced thru EXTERNs: */
#define GLOBAL(type)		type
/* a reference to a GLOBAL function: */
#define EXTERN(type)		extern type

to:

#ifdef JPEG_CREATE_LIBRARY
#define JPEGEXPORTVERB dllexport
#else
#define JPEGEXPORTVERB dllimport
#endif
/* a function referenced thru EXTERNs: */
#define GLOBAL(type)		__declspec(JPEGEXPORTVERB) type
/* a reference to a GLOBAL function: */
#define EXTERN(type)		extern __declspec(JPEGEXPORTVERB) type
  • Build the jpeg project.
  • Save the resulting libjpeg.lib and libjpeg.dll to VisIt's svn.
  • Copy jconfig.h, jerror.h, jmorecfg.h, jpeglib.h to the include directory you made for jpeg.

To build a 64-bit version, do the following extra steps:

  • Open the Configuration Manager
  • Click on the Active solution platform combo box and select New... from it
  • Add the x64 option and copy the settings from Win32.
  • Rebuild the project as 64-bit.

HDF 4

Obtain the source code. Extract the source files, a hdf-4.2.6 folder will be created. HDF 4.2.6 has a CMake build, so steps will be similar to HDF5 above.

Create a hdf-4.2.6-build folder in which to build the sources. Run CMake and set it up to use the HDF-related source and build directories.

HDF4 does not present all of its CMake options up front. If you want to skip multiple configure steps, then add all of the settings below using the 'Add Entry' button in CMake's gui before your first 'Configure'.

Click the configure button to start configuring CMake.

Settings:

  • BUILD_SHARED_LIBS:BOOL=ON
  • CMAKE_INSTALL_PREFIX:PATH=/path/to/trunk/windowsbuild/ThirdParty/hdf-4.2.6/MSVC2010-x64
  • JPEG_INCLUDE_DIR:PATH=/path/to/trunk/windowsbuild/ThirdParty/jpeg-8c/include
  • JPEG_LIBRARY:FILEPATH=/path/to/trunk/windowsbuild/ThirdParty/jpeg-8c/MSVC2010-x64/libjpeg.lib

Click the configure button.

Settings:

  • ZLIB_INCLUDE_DIR:PATH=/path/to/trunk/windowsbuild/ThirdParty/zlib/include
  • ZLIB_LIBRARY:FILEPATH=/path/to/trunk/windowsbuild/ThirdParty/zlib/MSVC2010-x64/zlib1.lib

Click the configure button.

Settings:

  • HDF4_BUILD_FORTRAN:BOOL=OFF
  • HDF4_ENABLE_SZIP_SUPPORT:BOOL=ON

Click the configure button.

Settings:

  • SZIP_INCLUDE_DIR:PATH=/path/to/trunk/windowsbuild/ThirdParty/szip/include
  • SZIP_LIBRARY_RELEASE:FILEPATH=/path/to/trunk/windowsbuild/ThirdParty/szip/MSVC2010-x64/szlibdll.lib


Click the configure button.

Click the configure button again.

Click the generate button.

Open the hdf-4.2.6-build/HDF4 solution file in Visual Studio. Open the Configuration manager dialog and select "Release" and "x64" for the platform. Build the solution. Then build the install target.

H5Part

Follow the instructions in the H5Part VisIt README file. The procedure for updating H5Part for the new compiler is essentially the same as for zlib, with the exception of needing to modify a property sheet file to point to correct paths for HDF5, SZIP and ZLIB.

NetCDF

There is a CMake-based build solution for shared libraries. Follow the directions in the Readme. Requires HDF5, SZIP and ZLIB.

VisIt

VisIt's build is largely influenced by the src/config-site/windows.cmake file since that file contains the hints that the build system uses when it needs to look for various libraries. You'll need to edit the windows.cmake file so it knows to look for your libraries. All along, you've been installing your libraries into new subdirectories with names like MSVC2010-x64. The top of the windows.cmake file contains a little scripting to map the version number of Visual Studio to a given directory name. Edit that code to add a new case for your new compiler version.