Exporting from a simulation

As in situ becomes increasingly popular, exporting reduced data from a running simulation will also increase in importance. VisIt's libsim interface currently relies on having an interactive connection to VisIt. This can be overcome with the use of the windowless CLI program that sets up the plots if you also include the exporting code within your simulation.

Enabling the simulation

The simulation should not export until the interactive VisIt connection has been made. This means that you will probably want to add a custom command to your simulation to turn on exporting once you have something that you want to export.

void simulate_one_timestep(simulation_data *sim)
{
    // STUFF TO CALCULATE ON THE SIMULATION (OMITTED)

    // Tell VisIt there is new data.
    VisItTimeStepChanged();
    VisItUpdatePlots();

    // NEW CODE: do the export
    //
    // If we have enabled exporting, then come up with a filename and tell VisIt to export.
    // This code uses Silo but you could do any of the supported formats such as XDB.
    if(sim->export)
    {
        char cmd[500];
        sprintf(cmd,  
        "dbAtts = ExportDBAttributes()\n"
        "dbAtts.db_type= \"Silo\"\n"
        "dbAtts.filename = \"updateplots_export%04d\"\n"
        "dbAtts.dirname = \".\"\n"
        "dbAtts.variables = (\"default\",)\n"
        "ExportDatabase(dbAtts)",
        sim->saveCounter);
        VisItExecuteCommand(cmd);
        sim->saveCounter++;
    }
}

void ControlCommandCallback(const char *cmd, const char *args, void *cbdata)
{
    simulation_data *sim = (simulation_data *)cbdata;

    if(strcmp(cmd, "halt") == 0)
        sim->runMode = SIM_STOPPED;
    else if(strcmp(cmd, "step") == 0)
        simulate_one_timestep(sim);
    else if(strcmp(cmd, "run") == 0)
        sim->runMode = SIM_RUNNING;
    else if(strcmp(cmd, "addplot") == 0)
    {
        VisItExecuteCommand("AddPlot(\"Pseudocolor\", \"zonal\")\n");
        VisItExecuteCommand("DrawPlots()\n");
    }

    // NEW CODE: enable exporting flag
    else if(strcmp(cmd, "export") == 0)
        sim->export = 1;
}

visit_handle
SimGetMetaData(void *cbdata)
{
    visit_handle md = VISIT_INVALID_HANDLE;
    simulation_data *sim = (simulation_data *)cbdata;

    /* Create metadata. */
    if(VisIt_SimulationMetaData_alloc(&md) == VISIT_OKAY)
    {
        // CODE OMITTED

            
        // NEW CODE: add an "export" command
        visit_handle cmd = VISIT_INVALID_HANDLE;
        if(VisIt_CommandMetaData_alloc(&cmd) == VISIT_OKAY)
        {
            VisIt_CommandMetaData_setName(cmd, "export");
            VisIt_SimulationMetaData_addGenericCommand(md, cmd);
        }
    }

    return md;
}

Driving the simulation

Once you have enabled your simulation so that it can export data, you will need a script to set up the plots that you want to create and then enable exporting in your simulation. Here is an example of such as script, though you will need to set up the plots that you would like and so on:

exportsim.py:

import os

# Routine to get the latest sim2 file in case you don't know the name of the sim2 file.
def GetSimFile(prefix):
    simdir=os.path.join(os.getenv("HOME"), ".visit", "simulations")
    files = [os.path.join(simdir, x) for x in os.listdir(simdir) if prefix in x]
    files.sort()
    return files[-1]

# Connect to the sim.
db = GetSimFile("updateplots")
OpenDatabase(db)

# Set up the plots that we want. We can use any VisIt plots and operators here.
AddPlot("Pseudocolor", "zonal")
DrawPlots()

# Tell the simulation to turn on exporting
SendSimulationCommand("localhost", db, "export")

# Tell the simulation to run again
SendSimulationCommand("localhost", db, "run")

You can run the script using an off-screen VisIt CLI like this:

visit -cli -nowin -s exportsim.py

When the CLI runs, it will connect to the running simulation, set up some plots and then turn on exporting before telling the simulation to run. The simulation is coded to call VisIt's export functions at each time step.

  • The VisItExecuteCommand function sends a command to execute to VisIt's CLI from the simulation. I forget whether the code returns immediately or goes into a blocking event loop that waits for the command to be executed before returning from VisItExecuteCommand. In other words, this export approach might have some synchronization issues with respect to the simulation.

Python driving both simulation and VisIt

Some simulations are composed of many cooperating programs unified with Python. In these systems, Python is used at the high level to start and coordinate the operation of many sub processes. The simulation and VisIt's Python-based CLI can be coordinated by a high level manager written in Python. Starting most programs using Python's subprocess module is straightforward. This holds true for many simulations but VisIt's CLI is itself an embedded Python interpreter and starting it under subprocess presents some challenges because the stdin and stdout file descriptors are not reliably captured by subprocess. This makes it challenging to have the overarching Python manager program send commands to the VisIt CLI over the pipes available through subprocess. One way around this is to use a named pipe to communicate with the VisIt CLI. The following code example shows how to use Python to both start a simulation and the VisIt CLI and communicate with the CLI using a named pipe and a small program to dispatch commands received over the pipe.

import os, subprocess, sys, select

class Program(object):
    def __init__(self):
        self.p = None
        self.connected = False

    def start(self, args):
        """
        Start up the program.
        """
        self.p = subprocess.Popen(args, 
                                  stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=True)
        if self.p != None:
            self.connected = True
        return self.p != None

    def end(self):
        """
        Terminate the program.
        """
        self.p.terminate()
        return 1

    def command(self, cmd):
        """
        Send a console command to the program."
        """
        if self.connected:
            self.p.stdin.write(cmd + "\n")
            self.p.stdin.flush()

class CLI(Program):
    def __init__(self, visit, debug=False):
        super(Program, self).__init__()
        self.visit = visit
        self.pipename = "to_cli"
        self.pipe = None
        self.debug = debug

    def start(self):
        """
        Start the CLI. We make a named pipe too so we can communicate with the CLI."
        """
        # Open the named pipe if we have not done that yet.
        if os.path.exists(self.pipename):
            os.unlink(self.pipename)
        os.mkfifo(self.pipename)
        # Write the script the CLI will use.
        cliscript = self._writescript()
        if self.debug:
            ret = Program.start(self, ["xterm", "-e", self.visit, "-cli", "-s", cliscript, "-pipename", self.pipename])
        else:
            ret = Program.start(self, [self.visit, "-cli", "-nowin", "-s", cliscript, "-pipename", self.pipename])
        self.pipe = os.open(self.pipename, os.O_WRONLY)
        return ret

    def command(self, cmd):
        """
        Send a command to the program. We override for the CLI because its 
        stdin,stdout don't work well when launched from subprocess.
        """
        if self.connected:
            os.write(self.pipe, cmd + "\n")

    def _writescript(self):
        """
        Write a script to read commands on the named pipe and execute them.
        """
        scriptname = "cliscript.py"
        f = open(scriptname, "wt")
        f.write("import os, sys\n")
        f.write("pipename = None\n")
        f.write("for i in range(len(sys.argv)):\n")
        f.write("    if sys.argv[i] == '-pipename':\n")
        f.write("        pipename = sys.argv[i+1]\n")
        f.write("if pipename == None:\n")
        f.write("    sys.exit(-1)\n")
        f.write("pipe = open(pipename, 'r')\n")
        f.write("while 1:\n")
        f.write("    line = pipe.readline()[:-1]\n")
        f.write("    print line\n")
        f.write("    eval(line)\n")
        f.close()
        return os.path.join(os.path.abspath(os.curdir), scriptname)

# Routine to get the latest sim2 file in case you don't know the name of the sim2 file.
def GetSimFile(prefix):
    simdir=os.path.join(os.getenv("HOME"), ".visit", "simulations")
    files = [os.path.join(simdir, x) for x in os.listdir(simdir) if prefix in x]
    files.sort()
    return files[-1]
        
def main():
    # The path to VisIt.
    visitdir = "/Users/bjw/Development/MAIN/trunk/src"
    # The simulation name
    simname = "updateplots"
    # The simulation executable.
    simulation = os.path.join(visitdir, "tools/DataManualExamples/Simulations/", simname)

    # The "visit" executable
    visit = os.path.join(visitdir, "bin", "visit")

    # Start up a simulation.
    sim = Program()
    if sim.start([simulation, "-dir", visitdir]):
        print "Started ", simulation
    else:
        print "Could not start ", simulation
        sys.exit(0)

    # Start up the CLI. We could pass debug=True to make the CLI run with a
    # window and have the console run in an xterm. The default is to run 
    # with no windows.
    cli = CLI(visit)
    if cli.start():
        print "Started cli."
    else:
        print "Could not start cli"
        sim.end()
        sys.exit(0)

    # Tell the cli to connect to the sim and set up some plots.
    db = GetSimFile(simname)
    cli.command('OpenDatabase("%s")' % db)
    cli.command('AddPlot("Pseudocolor", "zonal")')
    cli.command('DrawPlots()')

    # Tell the sim to run (via the CLI since sim.command() is unreliable)
    cli.command('SendSimulationCommand("localhost", "%s", "export")' % db)
    cli.command('SendSimulationCommand("localhost", "%s", "run")' % db)

    #This should work but doesn't in this example...
    #sim.command("export")
    #sim.command("run")

    # Read text output from the simulation.
    while 1:
        fds = select.select([sim.p.stdout, sim.p.stderr], (), ())
        for fd in fds:
            if len(fd) > 0:
                print fd[0].readline()

    cli.end()
    sim.end()

# Call main.
main()