-
Notifications
You must be signed in to change notification settings - Fork 0
Home
It contains the VRCanvas element which creates the three-dimensional virtual environment. The VRCanvas element contains a Plane element in order to represent the floor and to make the environment appear less spars and vertigo-inducing. The VR controllers are added to the virtual environment using the DefaultXRControllers element. The component HandMenu attaches a floating menu to the player's left controller. Next, there is the WorldspaceMenu which is a standalone menu present in the virtual environment which the player can interact with and the DataObject is the element which will visualise and control the interaction with the data cube streamed from the server.
A button component from the three-mesh-ui library for the VRCanvas GUI is used to trigger functions on the select interaction. It is used within Panel components to construct the user interface. It takes in a string of text to be displayed on the button and a function to be triggered on the select event as props. Saved as a JavaScript (.js) to circumnavigate an error encountered when attempting to use the component with TypeScript (Typescript does not recognise the "content" prop).
This is a three-mesh-ui panel component used to display a chart.js canvas as a texture on the panel surface. It receives the texture as a base64 image string and if the texture value changes the ChartObject is re-rendered and updated to display the new texture/image.
This component is also a three-mesh-ui panel but contains a ChartObject component. The chart.js instance is generated within this component in order for the component not to be affected by updates due to data changes. The image data of the chart.js canvas is sent to a nested panel component to be displayed, leaving the ChartPanel able to maintain its state. In order for the chart.js canvas to be interactive within the virtual environment, the ray (targetRay) emitted by the controller must be tracked in order to determine if and where the ray is on the chart.js canvas. This is done by detecting if the targetRay intersects the canvas and then determining where the ray points to on the local plane (x,y) coordinate it is converted for world space coordinates to the panel object's local space coordinates. The (x,y) are projected to the canvas as a mouse event and therefore trigger any mouse events present in the chart.js instance. The chart.js canvas is sampled every frame and is sent to the ChartObject component for it to be displayed as a texture. However, past the initial rendering of the chart on the ChartObject panel the texture does not update because the browser cannot process an offscreen canvas. The chart is not updating because the events that are projected to it are not processed. There is such a thing as an offscreencanvas however it only works in chromium-based web browsers, therefore it should be compatible with the Oculus Quest 2 Browser because it to is based off of Chromium.
The component where the data is rendered and visualised with a volume shader and colourmap, and manipulated using the VR controllers. The data is visualised as a cube the user manipulates instead of the user being enclosed by the data. The data cube object can be scaled by squeezing both controllers while pointing them at the cube and then bringing their hands closer together or pulling them further apart makes the cube larger or smaller within a range. The cube can be rotated by selecting the cube with both controllers and moving the controllers in the direction of the axis they would like to rotate the cube. Translation of the cube around the virtual space can be done by selecting the cube with one controller, the cube will mimic the translation of the controller.
(This is designed based on the assumption that the input data will be floating-point values between the values of 0 and 1) A Three.js Data3DTexture is used to hold the data, it is a three-dimensional texture and stores colour values in the form of unsigned integer RGBA values [84, 255, 12, 1.0] with is the default setup for the Data3DTexture. The data which we would like to display is single scalar floating-point values. The Data3DTexture must be set to RedFormat where it only takes in a single colour value (the R of RGBA) and set the texture type to float. this sets up the texture to store only floating-point scalars. However, by using RGBA format for the data points it is possible to send multiple floating-point scalar values to the shader and values can be isolated and visualised in different manners. This means multiple data cubes can be combined into a single visualisation, and can be layered or cycled through. The colourmap used to determine the colour of the individual data points is generated by the colourmap library and creates 100 shades as floating-point RGBA colour values. This colourmap is used as input for a Three.js DataTexture where the scalar values from the input data are mapped to the corresponding pixel/texel on the x-axis of the generated colourmap and therefore giving the data point its assigned colour. How the point appears on the screen and its opacity is determined by the vertex and fragment shader used for the cube's material. The material used to visualise the cube takes in uniforms which are values that are passed to the shaders, a vertex shader, a fragment shader, and settings which tell the renderer to render the backsides of the cube and that the material also uses transparency. The material is attached to the cube geometry and that creates the three-dimensional volume rendering of the Data3DTexture.
Registers the viewer with the backend. Responds with REGISTER_VIEWER_ACK
Acknowledgement response for REGISTER_VIEWER. Informs the frontend whether the session was correctly created.
VRDAVis 2022