Build visit development

build_visit is a large bash script that automates the Visit build process, including the building of necessary 3rd party libraries.

For an overview of the script & its usage please see: build_visit overview

This page presents some helpful info when updating/developing build_visit.

Modifying build_visit

Output

To resolve the battle between presenting messages to a user and capturing output to the log file, a few helper functions were created to aid build_visit developers:

Function Details Purpose
info Presents user with a message and echoes it to the log.
  • In graphical mode displays the message in a gui box
  • In console mode echoes the message to the screen.
Use for general info.
warn Echoes message to the screen and log. Use for non fatal error.
error Echoes message to the screen and log and kills the build process. Use for fatal error.

When the build process begins output is redirected to the log file with the following magic:

exec 3>&1 >> ${LOG_FILE} 2>&1
REDIRECT_ACTIVE="yes"

This captures the output of various configure/make commands to the log, avoiding spamming the user. The functions mentioned above are built to work correctly both before and after the redirect: They still present info to the user, and not just the log. This is why it is important to use them instead of simple echo commands.

Helpers for preparing to build a library

To reduce redundancy in coding for preparing/checking build directories and obtaining tarballs three helper methods are provided:

check_if_installed

  • Checks if library is installed. Returns: 1 on failure, 0 for success.
  • Usage Examples:
check_if_installed "qt" $QT_VERSION
check_if_installed "boxlib"

See build block for qt near the bottom of build_visit for an embedded usage example.

ensure_built_or_ready

  • Checks for installed lib, or obtains the tarball. Returns 1 on failure, 0 on success.
  • Usage Example (from check_required_3rdparty):
ensure_built_or_ready "cmake"  $CMAKE_VERSION  $CMAKE_BUILD_DIR  $CMAKE_FILE

prepare_build_dir

  • Checks for build dir, untars source file if necessary. Returns:-1 on failure, 0 for success, 1 for success if source was extracted from tarball.
  • Usage Example (from build_mesa):
prepare_build_dir $MESA_BUILD_DIR $MESA_FILE

General Coding Guidelines

  • If you develop new functions make sure they return 0 on success - inline with expected bash practices.

Suggestions for future changes

  • Create helper for generation of dynamic libs on OSX

Testing build_visit

To help reduce platform bugs and maintenance headaches we created a simple automated testing system that tests building 3rd party libraries using build_visit on various systems and posts the results to a central location. Current testing only exercises the svn mode of build_visit, so it is limited to developers with svn access.

We collect global test results at:

davinci.nersc.gov:/project/projectdirs/visit/bvauto_results/

I plan on setting up a cron job on davinci to reindex the results once an hour. To view the results log onto davinci and run:

firefox /project/projectdirs/visit/bvauto_results/index.html

Test scripts

There are two main test scripts (located in the repo @ src/svn_bin/):

bvtest.py

This script provides a set of classes that test building 3rd party libraries with build_visit and log test results to xml.

bvauto.py

This script automates build_visit tests using the classes in bvtest.py in following manner:

  • Sets up a temporary run directory.
  • Obtains the trunk version of bvtest.py & build_visit
  • Tests build_visit over a set of libraries from an input file.
  • Creates/Manages the result xml log and build_visit output logs.
  • Optionally posts the results (Note: Posting the results to a remote location requires a passwordless ssh to the host.)
  • Optionally cleans up the run directory (Note: This currently only happens in conjunction with posting the results.)

How do I use these scripts and contribute test results?

Test prep

Make sure:

  • You are a member of visitdev group on NERSC systems.
  • You have svn access to the VisIt repo.
  • You have passwordless ssh setup on both portal-auth.nersc.gov and davinci.nersc.gov.
  • You have the environment variable SVN_NERSC_NAME properly set to your NERSC username (on the test system).

Running a test

On your test system:

  • Grab a copy of bvauto.py & bvauto.input from the VisIt repo (@ trunk/src/svn_bin)
  • Fire up a test:
./bvauto.py bvauto.input

If you want to post the build results include "--post [result_root] [result_group]" arguments, for our purposes you want:

./bvauto.py bvauto.input --post username@davinci.nersc.gov:/project/projectdirs/visit/www/bvauto/ visit

Note: These should be accessible via http://portal.nersc.gov/project/visit/bvauto/index.html

Testing System Development Tasks

  • build_visit -> gdal should return w/ error when building on AIX
  • bvauto.py -> show failures on index page
  • bvauto.input -> update with full list of libs
  • bvauto.py & bvtest.py -> support for branches (and rev numbers?)