Filters To Consider Converting To VTKM

Next Gen Discussions

Review of Derived Field (expressions) on Many Core Archs using Python

Not like EAVL or DAX but a way to compose array-like expressions to make them more efficient for many core execution.

Did broad brush study of array-based expressions (derived variables) using a few different execution strategies; round-trip (aka naive), staged and fused. The differences in these strategies ultimately impact memory requirements of target device (CPU or GPU). Round-trip demonsrated best memory behavior but worst execution performance (due to all the data movement between CPU and target device). Staged demonstrated worst memory behavior and mid-level performance. Fused demonstrated best memory behavior and excellent performance. In fact, auto-fused was typically not any worse than manually fused kernels. Memory requirements of fused approach dependent on maximum fan-out in dataflow network representing kernel execution. Analysis of the dataflow network for expression execution enables automated JIT fusing of kernels for simple expressions. Approach taken not currently suitable/adaptable to support all of VisIt's expressions (function expressions are problematic here).

How might streaming impact this? Could impact all three approaches in positive ways. More likely to reduce memory constraints of staged approach?

Might it be possible to analyze dataflow network for task-level parallelism (much like parallel Make does for build dependencies), enabling a network to be decomposed for task-level parallelism?

Might it be possible to take a staged/fused hybrid approach by analyzing dataflow network and decoupling in stages to minimize highest fanout?

Why isn't this in VisIt yet? It is still pretty limited in scope (only 4 operators handled so far). It it is still being used more or less as a research tool.

Detailed list of all the filters

Two somewhat orthogoanl issues related to VTKm. One is a better data model. The other is Data parallel execution.

EAVL offers what we think is a better data model. VTKm may use EAVL's data model maybe with or maybe without the data parallel part of it.

EAVL's execution model is not like a VTK pipeline. You can specify a bunch of kernels and it will run them one after the other. But, you don't create long, complex pipelines and then do something like an update operation on them and then have a contract flow down and a bunch of execution occur.

Will we add OpenMP directly to VisIt or only via EAVL or both, depending on context?

How are FPEs handled in GPUs?

Deciding individual expressions difficulty in mapping to GPU is somewhat easy. However, managing long expressions involving many sub-terms in a way that doesn't result in a lot of data transfer between CPU and GPU is at least in the moderately difficult if not very difficult category.

Some expressions, like global-nodeid, don't really make sense to even do on a GPU. OTOH, if we don't make them available on a GPU and they appear in the middle of an expression, the lack of availablility on the GPU would wind up forcing CPU/GPU data transfer for no good reason.

Time integration expressions probably involve enough I/O that GPU is not relevant.

The user may want our typically Always/Auto/Never three-way choice for controlling whether VisIt uses GPUs. It may not make sense to use GPUs always when they are available. My guess is that dataset size will play a role in whether GPU will result in a boost in performance. Maybe part of the VisIt install process on a given platform can perfrom some GPU tests to determine minimum dataset size before GPUs should be used.

Regarding the mesh quality expressions, I suggested we investigate if there is ongoing development on the Verdict library and, in particular, if it might be being enhanced for GPU execution. First, I learned its very difficult to find a version of Verdict at Sandia's pages. It appears it might be available there embedded in other products such as Verde (which is available as a binary turnkey tool) and Mesquite. However, I learned that Verdict is also now available in VTK (http://www.vtk.org/Wiki/VTK/mesh_quality) as the vtkMeshQuality filter. In all cases, it was difficult to determine if Verdict is still under active development or supported.


This list of filters (derived from avtFilter) rates the difficulty in converting the filter to run in a data parallel manner using VTKm.

Filter Name Difficulty to Convert
avt/Expressions/Abstract/avtExpressionDataTreeIterator.C
avt/Expressions/General/avtConnComponentsExpression.C
avt/Expressions/General/avtFindExternalExpression.C
avt/Expressions/MeshQuality/avtNeighborExpression.C
avt/Filters/avtCondenseDatasetFilter.C NA
avt/Filters/avtContourFilter.C Hard
avt/Filters/avtCoordSystemConvert.C Easy
avt/Filters/avtDisplaceFilter.C Easy
avt/Filters/avtFacelistFilter.C Hard
avt/Filters/avtFeatureEdgesFilter.C
avt/Filters/avtGhostZoneFilter.C
avt/Filters/avtLineScanFilter.C
avt/Filters/avtLineToPolylineFilter.C
avt/Filters/avtLineoutFilter.C
avt/Filters/avtMeshLogFilter.C
avt/Filters/avtMissingDataFilter.C
avt/Filters/avtNamedSelectionFilter.C
avt/Filters/avtPointToGlyphFilter.C
avt/Filters/avtShiftCenteringFilter.C
avt/Filters/avtSmoothPolyDataFilter.C
avt/Filters/avtStaggeringFilter.C
avt/Filters/avtSurfaceFilter.C
avt/Filters/avtThresholdFilter.C
avt/Filters/avtTransform.C
avt/Filters/avtVertexNormalsFilter.C
avt/Filters/avtWarpFilter.C
avt/Filters/avtWorldSpaceToImageSpaceTransform.C
avt/Pipeline/AbstractFilters/avtDataTreeIterator.C
avt/Pipeline/AbstractFilters/avtSIMODataTreeIterator.C
avt/Pipeline/AbstractFilters/avtStructuredChunkDataTreeIterator.C
engine/main/CumulativeQueryNamedSelectionExtension.C
operators/AMRStitchCell/avtAMRStitchCellFilter.C
operators/BoundaryOp/avtBoundaryOpFilter.C
operators/Box/avtBoxFilter.C
operators/CartographicProjection/avtCartographicProjectionFilter.C
operators/Clip/avtClipFilter.C
operators/Cone/avtConeFilter.C
operators/ConnCompReduce/avtConnCompReduceFilter.C
operators/Context/avtContextFilter.C
operators/CoordSwap/avtCoordSwapFilter.C
operators/CracksClipper/avtCracksDensityFilter.C
operators/CracksClipper/avtRemoveCracksFilter.C
operators/CreateBonds/avtCreateBondsFilter.C
operators/Cylinder/avtCylinderFilter.C
operators/Decimate/avtDecimateFilter.C
operators/Delaunay/avtDelaunayFilter.C
operators/DualMesh/avtDualMeshFilter.C
operators/Edge/avtEdgeFilter.C
operators/EllipsoidSlice/avtEllipsoidSliceFilter.C
operators/ExtractPointFunction2D/avtExtractPointFunction2DFilter.C
operators/Extrude/avtExtrudeFilter.C
operators/FFT/avtFFTFilter.C
operators/FiveFoldTetSubdivision/avtFiveFoldTetSubdivisionFilter.C
operators/IndexSelect/avtIndexSelectFilter.C
operators/InverseGhostZone/avtInverseGhostZoneFilter.C
operators/Isovolume/avtIsovolumeFilter.C
operators/MetricThreshold/avtMetricThresholdFilter.C
operators/ModelFit/avtModelFitFilter.C
operators/MultiresControl/avtMultiresControlFilter.C
operators/OnionPeel/avtOnionPeelFilter.C
operators/PDF/avtPDFFilter.C
operators/Project/avtProjectFilter.C
operators/RectilinearProject2D/avtRectilinearProject2DFilter.C
operators/Reflect/avtReflectFilter.C
operators/RemoveCells/avtRemoveCellsFilter.C
operators/Replicate/avtReplicateFilter.C
operators/Revolve/avtRevolveFilter.C
operators/Slice/avtSliceFilter.C
operators/Smooth/avtSmoothFilter.C
operators/SphereSlice/avtSphereSliceFilter.C
operators/Stagger/avtStaggerFilter.C
operators/SubdivideQuads/avtSubdivideQuadsFilter.C
operators/ThreeSlice/avtThreeSliceFilter.C
operators/ToroidalPoloidalProjection/avtToroidalPoloidalProjectionFilter.C
operators/TriangulateRegularPoints/avtTriangulateRegularPointsFilter.C
operators/Tube/avtTubeFilter.C
operators/ZoneDump/avtZoneDumpFilter.C
plots/Boundary/avtBoundaryFilter.C
plots/Curve/avtPolarToCartesianFilter.C
plots/FilledBoundary/avtFilledBoundaryFilter.C
plots/Histogram/avtHistogramFilter.C
plots/Label/avtLabelFilter.C
plots/Label/avtLabelSubsetsFilter.C
plots/Mesh/avtMeshFilter.C
plots/Molecule/avtMoleculeFilter.C
plots/ParallelCoordinates/avtParallelCoordinatesFilter.C
plots/Pseudocolor/avtPseudocolorFilter.C
plots/Scatter/avtScatterFilter.C
plots/Subset/avtSubsetFilter.C
plots/Surface/avtWireframeFilter.C
plots/Tensor/avtTensorFilter.C
plots/Topology/avtTopologyFilter.C
plots/Truecolor/avtTruecolorFilter.C
plots/Vector/avtVectorFilter.C
plots/Volume/avtLowerResolutionVolumeFilter.C
plots/WellBore/avtWellBoreFilter.C

Detailed list of all the expressions

This list of expressions rates the difficulty in converting the expression to run in a data parallel manner using VTKm.

Mesh Quality

Expression Name Difficulty
aspect Med
aspect_gamma Med
condition Med
degree Med
diagonal_ratio Med
min_diagonal Med
max_diagonal Med
dimension Med
jacobian Med
max_edge_length Med
max_side_volume Med
maximum_angle Med
min_edge_length Med
min_side_volume Med
min_corner_area Med
min_sin_corner Med
min_sin_corner_cw Med
minimum_angle Med
neighbor Med
node_degree Med
oddy Med
relative_size Med
scaled_jacobian Med
shape Med
shape_and_size Med
shear Med
skew Med
stretch Med
taper Med
warpage Med
face_planarity Med
relative_face_planarity Med

Math

Expression Name Difficulty
abs Easy
ceil Easy
floor Easy
exp Easy
ln Easy
log10 Easy
max Easy
min Easy
mod Easy
round Easy
sqr Easy
sqrt Easy
random Easy
+ Easy
- Easy
* Easy
/ Easy
^ Easy
& Easy

Relational

Expression Name Difficulty
eq Easy
ge Easy
gt Easy
le Easy
lt Easy
ne Easy

Conditional

Expression Name Difficulty
if Easy

Logical

Expression Name Difficulty
and Easy
not Easy
or Easy

Trig

Expression Name Difficulty
acos Easy
asin Easy
atan Easy
cos Easy
cosh Easy
deg2rad Easy
rad2deg Easy
sin Easy
sinh Easy
tan Easy
tanh Easy

Vector

Expression Name Difficulty
color Easy
color4 Easy
colorlookup Easy
cross Easy
dot Easy
hsvcolor Easy
magnitude Easy
normalize Easy

Tensor

Expression Name Difficulty
contraction Easy
determinant Easy
effective_tensor Easy
eigenvalue Easy
eigenvector Easy
inverse Easy
principal_deviatoric_tensor Easy
principal_tensor Easy
tensor_maximum_shear Easy
trace Easy
transpose Easy
viscous_stress Easy

Array

Expression Name Difficulty
array_compose Easy/NA??
array_compose_with_bins Easy/NA??
array_decompose Easy/NA??

Materials

Expression Name Difficulty
dominant_mat Med
materror Med
matvf Med
nmats Med
specmf Med
value_for_material Med

Mesh

Expression Name Difficulty
area Med
coord Easy
cylindrical Easy
cylindrical_radius Easy
cylindrical_theta Easy
external_cell Hard
external_node Hard
global_nodeid Easy/NA??
global_zoneid Easy/NA??
max_coord ??
min_coord ??
nodeid Easy/NA??
polar Easy
polar_radius Easy
polar_theta Easy
polar_phi Easy
revolved_surface_area Med
revolved_volume Med
volume Med
zoneid Easy/NA??
zonetype Easy/NA??

Misc

Expression Name Difficulty
bin Med
cell_constant Easy
conn_components Hard
curl Easy
curve_domain Easy
curve_integrate Easy
curve_swapxy Easy
cycle Easy
divergence Easy
enumerate Easy
gauss_curvature Med
gradient Med
ij_gradient Med
ijk_gradient Med
isnan Easy
lambda2 Easy
Laplacian Easy
map ??
mean_curvature Med
nodal_constant Easy
point_constant Easy
procid Easy
q_criterion Easy
recenter Med
resample Hard
resrad Med
surface_normal Med
point_surface_normal Med
cell_surface_normal Med
edge_normal Med
point_edge_normal Med
cell_edge_normal Med
time Easy
timestep Easy
zonal_constant Easy

Image Processing

Expression Name Difficulty
abel_inversion ??
conservative_smoothing Med
mean_filter Med
median_filter Med

Comparison

Expression Name Difficulty
conn_cmfe Hard
curve_cmfe Hard
pos_cmfe Hard
symm_point Hard
symm_plane Hard
symm_transform Hard

Time Iteration

Expression Difficulty
average_over_time NA
cycle_at_minimum NA
cycle_at_maximum NA
first_cycle_when_condition_is_true NA
first_time_when_condition_is_true NA
first_time_index_when_condition_is_true NA
last_cycle_when_condition_is_true NA
last_time_when_condition_is_true NA
last_time_index_when_condition_is_true NA
min_over_time NA
max_over_time NA
sum_over_time NA
time_at_minimum NA
time_at_maximum NA
time_index_at_minimum NA
time_index_at_maximum NA
value_at_minimum NA
value_at_maximum NA
var_when_condition_is_first_true NA
var_when_condition_is_last_true NA