Immersive OSPRay Studio

We provide several extensions to OSPRay Studio v1.0.0 to enable immersive virtual reality and storytelling experiences.

1. Display a single, coherent virtual environment on tiled display walls
   • a branch in OSPRay Studio: an extension to support the display feature
   • Configuration Generator: Unity project for arranging display objects in a spatial layout and save the configuration in a JSON file
2. Use gesture-based interaction techniques
   • a plugin for OSPRay Studio: a plugin to support a flying gesture to move around in a 3D environement
   • Gesture Tracking Server: C++ TCP server that sends body tracking data from Kinect SDK to connected clients
3. Create annotated, animated stories
   • a plugin for OSPRay Sutdio: a plugin to support the story creation
   • Storyboard: Unity client for constructing a story
   • V-Mail Server: a server for managing created stories
4. (UPCOMING) Move around a 3D environment using spoken words

Note

See this branch in OSPRay Studio that merges all the extensions made to OSPRay Studio.

1. Display a single, coherent virtual environment on tiled display walls

We extended the application to simultaneously run multiple windows and move/scale these windows to create a large window. In the image, you see OSPRay Studio running on TACC's Rattler system, a tiled-display system driven by a cluster of 19 Linux PCs. Each node runs an MPI-process that shows a window on display, and all the displays form a hemisphere providing a surrounding view.

Note

See this branch for details and installation steps.

At the start, the application reads a JSON configuration file to set its cameras and arrange windows. We provide this Unity project to assist users in creating this JSON file.

2. Use gesture-based interaction techniques

We created a plugin and a server application to support a flying gesture to move in a 3D virtual environment. The server application gets body tracking data from Microsoft Kinect SDK and send these to connected clients. The plugin detects a flying gesture from the recieved data and moves the camera accordingly.

In the video, a user moves around a 3D virtual environment by lifting both hands - pretending to be a bird - and leaning the body to fly in that direction. When multiple users are presented in the area, the user closest to the sensor is considered the primary user.

Note

See the plugin for details and installation steps.

3. Create annotated, animated stories

We created a plugin to incorporate Visualization Mail (V-Mail) capabilities, which enables users to create annotated, animated stories directly from OSPRay Studio.

In the above image, you see three applications:

1. Top-left: OSPRay Studio with the plugin
2. Bottom-left: the server for managing created stories
3. Right: Unity client for constructing a story

Note

See the plugin for details and installation steps.

(UPCOMING) 4. Move around a 3D environment using spoken words

The goal is to create a plugin with voice-control capabilities so users can move around a 3D environment using spoken words.