Adding a new data object type

A brief overview of data objects in AVT

Each data object type is a derived type of avtDataObject. avtDataObject is special because avtSink, avtSource, and avtFilter, the key base classes for how VisIt does data flow, use the avtDataObject type to for their inputs and outputs. Hence, any input or output for VisIt's data flow network library must inherit from this type.

VisIt uses a complex inheritance scheme for building up filters that deal with the varied data object types. This could probably be solved with templates, but the amount of headaches with porting templates to various platforms makes the alternative that we chose (i.e. a complex inheritance hierarchy) acceptable. The inheritance scheme also adds value because it allows for specialized code to be placed at exactly the right layer. (This could probably also be accomplished with a specialized template approach.)

The steps to create a new data object

In principle, adding a new data object type requires that new types of sources, sinks, and filters be added with it. However, a script located at /src/tools/avt_do_add/avt_do_add will generate code for these classes for you.

Here's how to add a new data object:

  1. Run avt_do_add for your new class.
  2. Follow the directions avt_do_add gives you about modifying Makefiles
  3. Run configure and make. There will be several failures during the compile. These failures are intentional and they force you to consider the issues that are specific to your type that could not be codegen'd.
  4. (Optional) If you want to play around with your new type, make some plugins to test the type (see section below)
  5. Follow the directions avt_do_add gives you about making the new files be Subversion objects

Testing your new type with plugins

The game plan for testing your new type is to create three new operators:

  1. The first operator will convert an avtDataset into your data object,
  2. The second operator will work directly on your data type, and
  3. The third operator will convert your data object back to an avtDataset.

You need the first and third operators because, initially, there will be no file format readers that produce your new data object and no plots that can render your new data object. If we only wrote the second operator, then that operator would have a type mismatch; it would receive an avtDataset, when it would be expecting your data type. Eventually, we will solve that problem by using "bridges", which will accept avtDatasets and do inline conversions, but the purpose of this step is to confirm that your new data object type "works".

Assumptions

In the example I'm giving, I'll implement a new avtDataObject that stores only vertices. Of course, vertices could be stored natively in an avtDataset_p, but I'll pretend that it can't. My data object will be called avtMyDataObject. It will not be able to store field data, only point positions.

Changes to avtMyDataObject

I will add two new methods:

   void                      SetPoints(int, const float *);
   void                      GetPoints(int &, const float *&) const;

I will also add two new data members:

   int                       nPoints;
   float                    *points;

Constructor

avtMyDataObject::avtMyDataObject(avtDataObjectSource *src)
   : avtDataObject(src)
{
   nPoints = 0;
   points = NULL;
}


Destructor

avtMyDataObject::~avtMyDataObject()
{
   if (points != NULL)
       delete [] points;
}


ReleaseData

void
avtMyDataObject::ReleaseData(void)
{
   if (points != NULL)
   {
       delete [] points;
       points = NULL;
   }
}

DerivedCopy

 void
 avtMyDataObject::DerivedCopy(avtDataObject *dob)
 {
   avtMyDataObject *typed_dob = (avtMyDataObject *) dob;
   int npts = 0;
   const float *pts = NULL;
   typed_dob->GetPoints(npts, pts);
   float *pts_copy = new float[3*npts];
   for (int i = 0 ; i < 3*npts ; i++)
       pts_copy[i] = pts[i];
   SetPoints(npts, pts_copy);
   delete [] pts_copy;
}

GetPoints (Custom method)

void
avtMyDataObject::GetPoints(int &npts_out, const float *&pts_out) const
{
   npts_out = nPoints;
   pts_out  = points;
}

SetPoints (Custom method)

 void
 avtMyDataObject::SetPoints(int npts_in, const float *pts_in)
 {
    if (points != NULL)
        delete [] points;
    points = new float[3*npts_in];
    for (int i = 0 ; i < 3*npts_in ; i++)
        points[i] = pts_in[i];
    nPoints = npts_in;
 }

Adding DSToMy Plugin

The purpose of this plugin is to convert from an avtDataset to an avtMyDataObject.

  1. Go to an empty directory. My directory is called DSToMy, but that is not required.
  2. Run "xmledit DSToMy.xml"
    • (For "Plugin" tab)
    • Name = DSToMy
    • Label = DSToMy
    • Version = 1.0
    • Plugin type = Operator
    • (For "Attribute" tab)
    • Name = DSToMyAttributes
    • Purpose = Attributes for DSToMy operator
    • (For "Fields" tab)
    • Click "New"
    • Name = "unused"
    • Label = "Plugin tools don't work unless there is at least one field"
    • Type = "bool"
    • Go to File->Save
  3. Run "xml2plugin DSToMy.xml"
    • This will codegen the operator.
  4. Modify avtDSToMyFilter.h
    • Add include for <avtDatasetToMyDataObjectFilter.h>
    • Change the inheritance to be "class avtDSToMyFilter : virtual public avtPluginFilter, virtual public avtDatasetToMyDataObjectFilter"
    • (The inheritance from avtPluginFilter will tell VisIt how to deal with this filter as a plugin.)
    • Remove the method "ExecuteData"
    • Add the method "virtual void Execute(void);"
  5. Modify avtDSToMyFilter.C
    • Remove the method "ExecuteData"
    • Implement the method "Execute" (as per below)
  6. Run "make"
 #include <vtkDataSet.h>
 
 void avtDSToMyFilter::Execute(void)
 {
     int i, j;
     avtDataTree_p tree = GetInputDataTree();
     int nLeaves = 0;
     vtkDataSet **leaves = tree->GetAllLeaves(nLeaves);
     int npts = 0;
     for (i = 0 ; i < nLeaves ; i++)
         npts += leaves[i]->GetNumberOfPoints();
     float *pts = new float[3*npts];
     int current = 0;
     for (i = 0 ; i < nLeaves ; i++)
     {
         int np = leaves[i]->GetNumberOfPoints();
         for (j = 0 ; j < np ; j++)
         {
             double pt[3];
             leaves[i]->GetPoint(j, pt);
             pts[3*current+0] = pt[0];
             pts[3*current+1] = pt[1];
             pts[3*current+2] = pt[2];
             current++;
         }
     }
     
     avtMyDataObject_p output = GetTypedOutput();
     output->SetPoints(npts, pts);
     delete [] pts;
 }

Adding MyDODisplace Plugin

The purpose of this plugin is to displace the points in a random direction.

  1. Go to an empty directory. My directory is called MyDODisplace, but that is not required.
  2. Run "xmledit MyDODisplace.xml"
    • (For "Plugin" tab)
    • Name = MyDODisplace
    • Label = MyDODisplace
    • Version = 1.0
    • Plugin type = Operator
    • (For "Attribute" tab)
    • Name = MyDODisplaceAttributes
    • Purpose = Attributes for MyDODisplace operator
    • (For "Fields" tab)
    • Click "New"
    • Name = "magnitude"
    • Label = "Magnitude of displacement"
    • Type = "double"
    • Initialization code = "1.0"
    • Go to File->Save
  3. Run "xml2plugin MyDODisplace.xml"
    • This will codegen the operator.
  4. Modify avtMyDODisplaceFilter.h
    • Add include for <avtMyDataObjectToMyDataObjectFilter.h>
    • Change the inheritance to be "class avtDSToMyFilter : virtual public avtPluginFilter, virtual public avtMyDataObjectToMyDataObjectFilter"
    • (The inheritance from avtPluginFilter will tell VisIt how to deal with this filter as a plugin.)
    • Remove the method "ExecuteData"
    • Add the method "virtual void Execute(void);"
  5. Modify avtMyDODisplaceFilter.C
    • Remove the method "ExecuteData"
    • Implement the method "Execute" (as per below)
  6. Run "make"
void avtMyDODisplaceFilter::Execute(void)
{
   int npts = 0;
   float *pts;
   avtMyDataObject_p in_do = GetTypedInput();
   in_do->GetPoints(npts, pts);
   const float *new_pts = new float[3*npts];
   double mag = atts.GetMagnitude();
   for (int i = 0 ; i < npts ; i++)
   {
      double z = (rand() % 10000) / 5000. - 1;
      double pi = ((rand() % 10000) / 10000.) * 2 * M_PI;
      double plane_dist = mag*sqrt(1-z*z);
      z *= mag;
      double x = plane_dist*sin(pi);
      double y = plane_dist*cos(pi);
      new_pts[3*i+0] = pts[3*i+0] + x;
      new_pts[3*i+1] = pts[3*i+1] + y;
      new_pts[3*i+2] = pts[3*i+2] + z;
   }
   
   avtMyDataObject_p out_do = GetTypedOutput();
   out_do->SetPoints(npts, new_pts);
   delete [] new_pts;
}

Adding MyToDS Plugin

The purpose of this plugin is to convert from an avtMyDataObject to an avtDataset.

  1. Go to an empty directory. My directory is called MyToDS, but that is not required.
  2. Run "xmledit MyToDS.xml"
    • (For "Plugin" tab)
    • Name = MyToDS
    • Label = MyToDS
    • Version = 1.0
    • Plugin type = Operator
    • (For "Attribute" tab)
    • Name = MyToDSAttributes
    • Purpose = Attributes for MyToDS operator
    • (For "Fields" tab)
    • Click "New"
    • Name = "unused"
    • Label = "Plugin tools don't work unless there is at least one field"
    • Type = "bool"
    • Go to File->Save
  3. Run "xml2plugin MyToDS.xml"
    • This will codegen the operator.
  4. Modify avtMyToDSFilter.h
    • Add include for <avtMyDataObjectToDatasetFilter.h>
    • Change the inheritance to be "class avtMyToDSFilter : virtual public avtPluginFilter, virtual public avtMyDataObjectToDatasetFilter"
    • (The inheritance from avtPluginFilter will tell VisIt how to deal with this filter as a plugin.)
    • Remove the method "ExecuteData"
    • Add the method "virtual void Execute(void);"
  5. Modify avtMyToDSFilter.C
    • Remove the method "ExecuteData"
    • Implement the method "Execute" (as per below)
  6. Run "make"
#include <vtkUnstructuredGrid.h>
#include <vtkCellType.h>

void avtMyToDSFilter::Execute(void)
{
    int i, j;
    avtMyDataObject_p input = GetTypedInput();
    int npts = 0;
    const float *pts_in = NULL;
    input->GetPoints(npts, pts_in);
    
    vtkUnstructuredGrid *ugrid = vtkUnstructuredGrid::New();
    vtkPoints *pts = vtkPoints::New();
    pts->SetNumberOfPoints(npts);
    for (i = 0 ; i < npts ; i++)
    {
        double pt[3];
        pt[0] = pts_in[3*i+0];
        pt[1] = pts_in[3*i+1];
        pt[2] = pts_in[3*i+2];
        pts->SetPoint(i, pt);
    }
    ugrid->SetPoints(pts);
    pts->Delete();
    
    ugrid->Allocate(2*npts);
    for (i = 0 ; i < npts ; i++)
    {
        vtkIdType id[1];
        id[0] = i;
        ugrid->InsertNextCell(VTK_VERTEX, 1, id);
    }
  
    avtDataTree_p tree = new avtDataTree(ugrid, 0, "merged_points");
    ugrid->Delete();
    SetOutputDataTree(tree);
    int nLeaves = 0;
}


Putting it into action

  • Open up the file noise.silo that comes with VisIt.
  • Make a Mesh plot of PointMesh.
  • Apply the operator DSToMy.
  • Apply the operator MyDODisplace.
  • Apply the operator MyToDS.
  • Draw
  • You will get a point mesh in the screen.

Summarizing the previous points from this page:

  • The file noise.silo produced an avtDataset
  • That output was connected to the filter avtDSToMyFilter, which created an avtMyDataObject output
  • The next filter is avtMyDODisplaceFilter, which expects an avtMyDataObject and produces an avtMyDataObject.
  • The final filter, avtMyToDSFilter creates an avtDataset. This avtDataset is then fed into the filters for the mesh plot, which expect an avtDataset.

A final wrinkle

If you change the variable of the Mesh plot to be "Mesh" instead of "PointMesh", it doesn't work. This is because each filter is expected to maintain meta-data and the filters we quickly implemented didn't do it. The mesh plot looks at the topological dimension of the input. Our filter DSToMy should have indicated that the topological dimension changed from 3 to 0. Because we didn't communicate that information, the Mesh plot got confused and gave up. We can fix it by doing the following:

  • Modify avtDSToMyFilter.h to declare the method "virtual void RefashionDataObjectInfo(void);"
  • Then define this method in avtDSToMyFilter.C:
void avtDSToMyFilter::RefashionDataObjectInfo(void)
{
  GetOutput()->GetInfo().GetAttributes().SetTopologicalDimension(0);
}

What's missing

A nice improvement would be to have VisIt automatically detect when a filter expects one data type, but that data type is not available. This would obviate the need for the DSToMy and MyToDS operators. Instead, bridges would exist between each of the data types and VisIt would do automatic conversion. We're not there yet. However, you (Christoph) can do development using the operators as (explicit) bridges and I'll take care of the infrastructure work later.