Missing Data

This page describes how VisIt handles missing data values. Various I/O libraries are able to annotate arrays with a special value that indicates missing data. When the special missing data value appears in a data array, the client code that processes the data is supposed to ignore the missing data values. For the most part, VisIt removes missing data values from the visualization so cells or nodes containing missing data are absent. The missing data value is also not used in extents calculations.

Missing data are often used for:

  • Indicating that parts of the data have no real data and should be ignored
  • Writing a structured mesh representation and then tagging cells that don't matter so they can be removed
    • This is sort of a poor man's unstructured mesh

This feature will appear in VisIt 2.5.

Representing missing data

The missing data value has various names depending on the I/O library that was used. The NETCDF convention for missing data seems to bleed into some other libraries such as HDF4/5 and so we will examine the convention here. For more information, see NETCDF attribute conventions. To paraphrase, there are a few different attributes present on a variable that can be used to represent missing data.

  • _FillValue : The value that was written to the array where no other data values were written.
  • missing_value : The value that represents missing data.
  • valid_min : Every value in the array below valid_min's value is missing data.
  • valid_max : Every value in the array above valid_max's value is missing data.
  • valid_range : Every value outside of the specified range is missing data.

See Missing Data Implementation to see how missing data are exposed from database reader plugins.

Treatment of missing data

Missing data are removed from plots

VisIt usually will try to remove cells or nodes that have missing data values so they are not shown. Note that for plots such as the Mesh plot, there are no variables so the full mesh is returned for the plot. The cells removed for missing data depend on the variable being plotted since each variable will have different missing data criteria.

MissingDataRemoval.png

Spreadsheet plot

VisIt's Spreadsheet plot identifies missing data values but does not throw them away. This permits the plot to show missing data values using a special background color (currently light pink).

MissingDataSpreadsheet.png

Expressions

When VisIt detects that variables that are inputs into an expression use missing data values, it creates a mask for the mesh, taking into account the missing data values for each variable. The final values that are plotted only show the cells where the data were valid for all of the input variables. In this example, var1 has values that are valid on land and var2 has values that are valid on the sea surface. There is a little overlap between valid cells in the land/sea boundary so var1+var2 is defined only on the land/sea boundary where both input variables have valid values.

MissingDataExpressions.png