VisIt in a web browser
VisIt provides rich data processing and plotting machinery. VisIt also provides an application front end that plots graphics and allows users to interact with plots. Some users would like to couple the power of the VisIt back-end with a web browser so data processed by VisIt could be viewed without having to install VisIt on a local system. There are many ways to achieve this and the purpose of this page is to explore some of the options.
Attached is an image which shows a prototype of VisIt Viewer forwarding a "RenderObject" to its clients. Currently, the GUI is the only one that is able to receive this information. The middle image is the VisIt Viewer rendering an image, the other render windows are VisIt clients (GUI) rendering a remote avtDataObject.
The api will be designed to remotely request VisIt for render data for any set of active windows. Currently, the plan is to support having VisIt forward , from least resource intensive to most, (screen capture images, FBO, Scalable Rendering, avtDataSet). Any client implementing the ViewerProxy object will get access to this information.
For example, browsers can request screen image data while desktops with high bandwidth could support rendering avtDataSets.
Eventually, this can be put into the portal as a native rendering through the Java/Liferay application. Actually, the VisIt Java client seems like the best place to start and merge the code as a Liferay applet.
I see several benefits to enabling this:
1. Multiple remote clients: can connect to the same Viewer and share render information. Example: User (A) can connect via a website in Japan and user (B) can connect using VisIt client from US and see the same dataset that lives on NERSC, RDAV, or any shared resource.
2. Tiled Displays/VR environment: Each client can show the same dataset with different Views or Angles effectively constructing a tiled look or it can be put in a CAVE.
3. Personal rendering libraries: Forwarding the RenderObject also allows the creation of custom plotting libraries for the destination languages. For instance, the Java client would be able to render the avtDataSet with its own libraries and not be associated directly with the VisIt Viewer. Same goes from WebGL or whatever anyone can think of.
4. Splitting Vis resources: Launching several VisIt Viewers that then connect a subset of clients to each Viewer and possibly a python script to manage swapping buffers. Imagine connect 10 clients to NERSC and 10 to RDAV machines and pooling resources from both. (crazy imagination :)
Probably a lot of other use cases....
Again, this is now at a prototype stage and has a more effort ahead of it. I thought I would share the progress so far with all people that may be interested in this. I need to modify the Viewer to be enabled remotely rather than being hardcoded to "localhost" as well as create a simplified render client in Java.
In deciding how to approach this, we should have some goals for the end product that gets embedded in the web browser.
- No VisIt components are required to exist on the client system.
- Bring up a standard web browser such as firefox and go to a URL that serves up plots created by VisIt.
- Communicate with some back-end that serves up web pages that let you browse the file system, create plots, control their attributes, etc.
- Depending on implementation, the back-end will send image files of plots or possibly the geometry which we might be able to display using WebGL or similar.
- I've had a request to be able to browse a live simulation instrumented with VisIt's libsim. Some of the challenges are the same.
VisIt in a browser can be achieved many ways. We're not actually putting VisIt in a browser. We're using VisIt's compute server within a web server to help generate content for dynamic web pages that let the user visualize data.
We will need to figure out how to integrate VisIt with a web server. This probably looks like writing a Python or PHP module that can drive either the CLI without a window or a module that can directly talk to the engine. Once the module can get either plot images or data from VisIt, the rest of the problem becomes how we serve up pages that let the user do various things. Of course, this part can take various forms, depending on the best tech for the job.
Browser talks to engine
In this scenario, we write a web server that includes the VisIt compute server. The web server is a PHP or Python module (whatever, really) and the compute server sits in it and waits for commands from the browser. The commands probably come in as page requests and the server dispatches that into commands to the compute server. We send back either images done with SR mode or we send back geometry that we'll come up with a way to render on the client.
- Note that if we display dynamic geometry in the browser then we'll have to write web-equivalent of VTK mappers and any specialized renderers that our plots have.
Browser talks to viewer
In this scenario, we write a web server in, say, Python and that server does "import visit" to get the VisIt module and the normal viewer (windowless), etc. The web server module accepts page requests and those are turned into calls that drive VisIt via the Python interface. Image files get written and are served back to the browser client as content along with whatever pages get created to permit visualization. We don't have a way to send back geometry with this method.
Notes (In progress)
These are notes from a meeting held between Sean Ahern, Brad Whitlock, and Hari Krishnan about determining a course of action to take for enabling VisIt to be used from within a browser. All modifications that were discussed would affect the VisIt's front end - Gui and Viewer. Our goal is to interactively use VisIt within a browser without external dependencies. The end results should also enable using VisIt to work on any system from modern PC's to mobile platforms.
Several front end interfaces already exist for VisIt. Notably, Python cli, Java, and Qt C++. A web version of the gui would require effort but should be trivial to create. The primary difficulty is communicating with the Viewer. This is where the majority of our efforts would lie.
The VisIt Viewer has two main tasks: state management and rendering results. Converting VisIt's state management to client side would require a significant investment in time and effort. For that reason the first iteration of our efforts will separate the viewer into two parts. We plan to move the renderer to the client side and leave the state management on the server side.
On the client side:
A new pure visualization renderer will need to be created. This has been proposed to be written in either WebGL or Java/JOGL. The communication can either occur through AJAX or sockets if it is within a Java applet.
On the remote side:
A new extension called "external_window" mode will be added to VisIt. This option will now allow the viewer to start in windowed mode (default), the "-nowin" option to do offscreen rendering, and "-extwin" option to pass through results to an external window. This new option will allow external viewers whether local or remote to communicate and interact with the visualization of the data.
Some glue code needs to be written to allow communication through a web api. This can be just forwarding instructions to the Visit viewer running on the remote side as well as maintaining session for unique users.
Once the viewer receives commands from a remote user then it can use the normal VisIt execution process. The results of this computation will then be forwarded back to the listening client. Our initial efforts will focus on successfully transmitting images to a external viewer. Once this stage is completed then we will look into more complicated models such as transmitting VTK data or OpenGL information.
Need Image Use Case: Web Gui -> WebGL Client -> (AJAX) Tomcat or Apache running new VisIt viewer -> (executes work) Visit compute engine (remote site/RDAV) -> (Images/Data) VisIt Viewer -> Pass through -> WebGL Client -> Visual Results
Other possible advantages
- Collaborative Visualization: Multiple remote users can connect to the same session and view the same results
- Complete VisIt viewer on client side: Extensive rework of VisIt is required
- Plugin version of VisIt: Requires download and will not work on mobile platforms