Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rationalizing and clarifying XRFrameOfReference and XRCoordinateSystem #409

Merged
merged 12 commits into from
Oct 12, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
198 changes: 17 additions & 181 deletions explainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -123,9 +123,9 @@ function beginXRSession() {
Only one immersive session per XR hardware device is allowed at a time across the entire UA. Any non-immersive sessions are suspended when an immersive session is active. Non-immersive sessions are not required to be created within a user activation event unless paired with another option that explicitly does require it.

Once the session has started, some setup must be done to prepare for rendering.
- A `XRFrameOfReference` must be created to define the coordinate system in which the `XRDevicePose` objects will be defined. See the Advanced Functionality section for more details about frames of reference.
- The depth range of the session should be set to something appropriate for the application. This range will be used in the construction of the projection matrices provided by `XRFrame`.
- A `XRFrameOfReference` should be created to establish a coordinate system in which `XRDevicePose` data will be defined. See the [Spatial Tracking Explainer](spatial-tracking-explainer.md) for more information.
- A `XRLayer` must be created and assigned to the `XRSession`'s `baseLayer` attribute. (`baseLayer` because future versions of the spec will likely enable multiple layers, at which point this would act like the `firstChild` attribute of a DOM element.)
- Then `XRSession.requestAnimationFrame` must be called to start the render loop pumping.

```js
let xrSession = null;
Expand All @@ -135,18 +135,15 @@ function onSessionStarted(session) {
// Store the session for use later.
xrSession = session;

// The `XRFrameOfReference` provides the coordinate system in which
// `getViewMatrix()` and the `poseModelMatrix` are defined. For more
// information on this see the `Advanced functionality` section
xrSession.requestFrameOfReference("head-model")
.then((frameOfRef) => {
xrFrameOfRef = frameOfRef;
})
.then(setupWebGLLayer) // Create a compatible XRWebGLLayer.
.then(() => {
// Start the render loop
xrSession.requestAnimationFrame(onDrawFrame);
});
xrSession.requestFrameOfReference({ type:'stationary', subtype:'eye-level' })
.then((frameOfRef) => {
xrFrameOfRef = frameOfRef;
})
.then(setupWebGLLayer) // Create a compatible XRWebGLLayer
.then(() => {
// Start the render loop
xrSession.requestAnimationFrame(onDrawFrame);
});
}
```

Expand Down Expand Up @@ -198,7 +195,7 @@ The `XRWebGLLayer`s framebuffer is created by the UA and behaves similarly to a

Once drawn to, the XR device will continue displaying the contents of the `XRWebGLLayer` framebuffer, potentially reprojected to match head motion, regardless of whether or not the page continues processing new frames. Potentially future spec iterations could enable additional types of layers, such as video layers, that could automatically be synchronized to the device's refresh rate.

To get view matrices or the `poseModelMatrix` for each `XRFrame`, developers must call `getDevicePose()` and provide an `XRCoordinateSystem` to specify the coordinate system in which these matrices should be defined. Unless the "head-model" `XRFrameOfReference` is being used, this function is not guaranteed to return a value. For example, the most common frame of reference, "eye-level", will fail to return a `viewMatrix` or a `poseModelMatrix` under tracking loss conditions. In that case, the page will need to decide how to respond. It may wish to re-render the scene using an older pose, fade the scene out to prevent disorientation, fall back to a "head-model" `XRFrameOfReference`, or simply not update. For more information on this see the [`Advanced functionality`](#orientation-only-tracking) section.
To get view matrices or the `poseModelMatrix` for each `XRFrame`, developers must call `getDevicePose()` and provide an `XRFrameOfReference` in which these matrices should be returned. Due to the nature of XR tracking systems, this function is not guaranteed to return a value and developers will need to respond appropriately. For more information about what situations will cause `getDevicePose()` to fail and recommended practices for handling the situation, refer to the [Spatial Tracking Explainer](spatial-tracking-explainer.md).

```js
function onDrawFrame(timestamp, xrFrame) {
Expand Down Expand Up @@ -398,7 +395,7 @@ The properties of an XRInputSource object are immutable. If a device can be mani

### Input poses

Each input source can query a `XRInputPose` using the `getInputPose()` function of any `XRFrame`. Getting the pose requires passing in the `XRInputSource` you want the pose for, as well as the `XRCoordinateSystem` the pose values should be given in, just like `getDevicePose()`. `getInputPose()` may return `null` in cases where tracking has been lost (similar to `getDevicePose()`), or the given `XRInputSource` instance is no longer connected or available.
Each input source can query a `XRInputPose` using the `getInputPose()` function of any `XRFrame`. Getting the pose requires passing in the `XRInputSource` you want the pose for, as well as the `XRFrameOfReference` the pose values should be given in, just like `getDevicePose()`. `getInputPose()` may return `null` in cases where tracking has been lost (similar to `getDevicePose()`), or the given `XRInputSource` instance is no longer connected or available.

The `gripMatrix` is a transform into a space where if the user was holding a straight rod in their hand it would be aligned with the negative Z axis (forward) and the origin rests at their palm. This enables developers to properly render a virtual object held in the user's hand. For example, a sword would be positioned so that the blade points directly down the negative Z axis and the center of the handle is at the origin.

Expand Down Expand Up @@ -661,110 +658,6 @@ For each of these events the `XRInputSource`'s target ray must be updated to ori

Beyond the core APIs described above, the WebXR Device API also exposes several options for taking greater advantage of the XR hardware's capabilities.

### Orientation-only tracking

Some headsets (Daydream, Gear VR, Cardboard), are naturally limited to only tracking a user's head rotation. This is known as "3 Degree of Freedom" tracking, or "3DoF". For this type of device, a small amount of simulated translation is usually applied to the returned matrices by estimating the motion of the user's neck based on the device rotation. If the translation portions of the matrices provided by an `XRDevicePose` do not reflect a real position in space, that must be indicated by setting the `emulatedPosition` attribute to `true`.

For devices that are not naturally limited to orientation-only tracking, it can still be useful for app developers to explicitly specify that they don't want any positional tracking in the matrices they receive. This is primarily necessary when viewing 360 photos or videos, since the source material is intended to be viewed from a single point. (This may also provide power savings on some devices, since it may allow some sensors to be turned off.) That can be accomplished by requesting a "head-model" `XRFrameOfReference`.

```js
xrSession.requestFrameOfReference("head-model").then((frameOfRef) => {
xrFrameOfRef = frameOfRef;
});

// Use xrFrameOfRef as detailed above.
```

Any `XRDevicePose`s queried with a "head-model" `XRFrameOfReference` must have their `emulatedPosition` attributes set to `true`. Any `XRInputPose`s queried with a "head-model" `XRFrameOfReference` must provide positions relative to the head's emulated position, so that the input devices appear in the correct location from the user's point of view.

### Room-scale tracking and boundaries

Some XR devices have been configured with details about the area they are being used in, including things like where the floor is and what boundaries of the safe space is so that it can be communicated to the user in XR. It can be beneficial to render the virtual scene so that it lines up with the users physical space for added immersion, especially ensuring that the virtual floor and the physical floor align. This is frequently called "room scale" or "standing space". It helps the user feel grounded in the virtual space. WebXR refers to this type of bounded, floor-relative play space as a "stage". Applications can take advantage of that space by creating a stage `XRFrameOfReference`. This will report values relative to the floor, ideally at the center of the room. (In other words the users physical floor is at Y = 0.) Not all XR devices will support this mode, however. `requestFrameOfReference` will reject the promise in that case.

```js
// Try to get a frame of reference where the floor is at Y = 0
xrSession.requestFrameOfReference("stage").then((frameOfRef) => {
// Will always succeed due to stage emulation. See the section titled
// "Emulated stage frame of reference" for details.
xrFrameOfRef = frameOfRef;
});

// Use xrFrameOfRef as detailed above, but render the floor of the virtual space at Y = 0;
```

When using a stage `XRFrameOfReference` the device will frequently have a configured "safe area" that the user can move around in without fear of bumping into real world objects. WebXR can communicate the rough boundaries of this space via the `XRFrameOfReference.bounds` attribute. It provides a polygonal boundary given in the `geometry` point array, which represents a loop of points at the edges of the safe space. The points are given in a clockwise order as viewed from above, looking towards the negative end of the Y axis. The shape it describes is not guaranteed to be convex. The values reported are relative to the stage origin, but do not necessarily contain it. The `bounds` attribute is null if the bounds are unavailable for the current frame of reference.

Points in the `bounds.geometry` array must have a `y` value of `0` and a `w` value of `1`.

If the `bounds` are available the application should try to ensure that all content the user needs to interact with can be reached while staying inside the described bounds geometry.

```js
// Demonstrated here using a fictional 3D library to simplify the example code.
function onBoundsUpdate() {
if (xrFrameOfRef.bounds) {
// Visualize the bounds geometry as 2 meter high quads
boundsMesh.clear();
let pointCount = xrFrameOfRef.bounds.geometry.length;
for (let i = 0; i < pointCount - 1; ++i) {
let pointA = xrFrameOfRef.bounds.geometry[i];
let pointB = xrFrameOfRef.bounds.geometry[i+1];
boundsMesh.addQuad(
pointA.x, 0, pointA.z, // Quad Corner 1
pointB.x, 2.0, pointB.z) // Quad Corner 2
}
// Close the loop
let pointA = xrFrameOfRef.bounds.geometry[pointCount-1];
let pointB = xrFrameOfRef.bounds.geometry[0];
boundsMesh.addQuad(
pointA.x, 0, pointA.z, // Quad Corner 1
pointB.x, 2.0, pointB.z) // Quad Corner 2
} else {
// No bounds geometry to visualize
boundsMesh.clear();
}
}
```

Changes to the bounds while a session is active should be a relatively rare occurrence, but it can be monitored by listening for the frame of reference's `boundschange` event.

```js
xrFrameOfRef.addEventListener('boundschange', onBoundsUpdate);
```

### Emulated stage frame of reference

Often times content designed to be used with a `stage` `XRFrameOfReference` (that is, the physical floor is at Y=0) can still be used with headsets that don't have appropriate knowledge of the user's physical space. For example the headset may only support 3DoF tracking, or 6DoF tracking without floor detection. In these case as a matter of developer convenience an emulated `stage` frame of reference is provided by default as a fallback.

An emulated `stage` frame or reference is functionally identical to an `eye-level` frame of reference with an offset applied along the Y axis to place the user's head at an estimated height. The default estimated height is determined by the UA, and can be an aritrary or user configurable value. If the platform APIs provide a user configured height that should be taken into consideration when determining the emulated height. Note that the floor's location will almost certainly not match up with the user's physical floor when using `stage` emulation, the intent is just to get it "close enough" that the user doesn't overtly feel like they are stuck in the ground or floating. No bounds are reported for an emulated `stage`.

Using an emulated `stage` as a fallback prevents the need for additional state tracking and matrix transforms on the developer's part in order to render the same content on a wide range of devices. To detect if the stage is using an emulated height or not after creation developers can check the `XRFrameOfReference`'s `emulatedHeight` attribute. A non-zero value indicates that the `stage` is being emulated.

If the system is capable of providing native `stage` tracking it must do so instead of providing an emulated `stage` frame of reference. Some applications may require a non-emulated `stage`, however, so the application is allowed to opt-out of emulation by setting the `disableStageEmulation` dictionary option to `true` when calling `requestFrameOfReference()`.

```js
// Get a native stage frame of reference if one is available, fail otherwise.
xrSession.requestFrameOfReference("stage", { disableStageEmulation: true })
.then((frameOfRef) => {
xrFrameOfRef = frameOfRef;
}).catch(() => {
// No stage frame of reference available for this device. Fall back to
// using a different frame of reference type of provide an appropriate
// error to the user.
});
```

Some experiences that use a `stage` frame of reference may assume that the user will be sitting, kneeling, or assuming some pose other than standing for the majority of the experience, such as a racing game or a meditation application. To accommodate these types of non-standing `stage` experiences a preferred height can also be provided to the `requestFrameOfReference()` options dictionary via the `stageEmulationHeight` option, given in meters. When `stageEmulationHeight` is not `0` it must be used in favor of UA provided default values while emulating.

```js
// Get a stage frame of reference, emulating one defaulting to 1.2 meters high
// if necessary.
xrSession.requestFrameOfReference("stage", { stageEmulationHeight: 1.2 })
.then((frameOfRef) => {
// Will always succeed.
xrFrameOfRef = frameOfRef;
});
```

### Controlling rendering quality

While in immersive sessions, the UA is responsible for providing a framebuffer that is correctly optimized for presentation to the `XRSession` in each `XRFrame`. Developers can optionally request either the buffer size or viewport size be scaled, though the UA may not respect the request. Even when the UA honors the scaling requests, the result is not guaranteed to be the exact percentage requested.
Expand Down Expand Up @@ -810,20 +703,6 @@ function onDrawFrame() {
}
```

### Responding to a reset pose

Many XR systems have a mechanism for allowing the user to reset which direction is "forward." For security and comfort reasons the WebXR Device API has no mechanism to trigger a pose reset programmatically, but it can still be useful to know when it happens. Pages may want to take advantage of the visual discontinuity to reposition the user or other elements in the scene into a more natural position for the new orientation. Pages may also want to use the opportunity to clear or reset any additional transforms that have been applied if no longer needed.

A page can be notified when a pose reset happens by listening for the 'resetpose' event from the 'XRSession'.

```js
xrSession.addEventListener('resetpose', xrSessionEvent => {
// For an app that allows artificial Yaw rotation, this would be a perfect
// time to reset that.
resetYawTransform();
});
```

### Controlling depth precision

The projection matrices given by the `XRView`s take into account not only the field of view of presentation medium but also the depth range for the scene, defined as a near and far plane. WebGL fragments rendered closer than the near plane or further than the far plane are discarded. By default the near plane is 0.1 meters away from the user's viewpoint and the far plane is 1000 meters away.
Expand Down Expand Up @@ -930,16 +809,13 @@ dictionary XRSessionCreationOptions {

attribute EventHandler onblur;
attribute EventHandler onfocus;
attribute EventHandler onresetpose;
attribute EventHandler onend;

attribute EventHandler onselect;
attribute EventHandler onselectstart;
attribute EventHandler onselectend;
attribute EventHandler oninputsourceschange;

Promise<XRFrameOfReference> requestFrameOfReference(XRFrameOfReferenceType type, optional XRFrameOfReferenceOptions options);
NellWaliczek marked this conversation as resolved.
Show resolved Hide resolved

long requestAnimationFrame(XRFrameRequestCallback callback);
void cancelAnimationFrame(long handle);

Expand All @@ -966,8 +842,9 @@ enum XREnvironmentBlendMode {
readonly attribute XRSession session;
readonly attribute FrozenArray<XRView> views;

XRDevicePose? getDevicePose(XRCoordinateSystem coordinateSystem);
XRInputPose? getInputPose(XRInputSource inputSource, XRCoordinateSystem coordinateSystem);
// Also listed in the spatial-tracking-explainer.md
XRDevicePose? getDevicePose(optional XRFrameOfReference frameOfReference);
XRInputPose? getInputPose(XRInputSource inputSource, optional XRFrameOfReference frameOfReference);
};

enum XREye {
Expand All @@ -988,9 +865,8 @@ enum XREye {
};

[SecureContext, Exposed=Window] interface XRDevicePose {
readonly attribute boolean emulatedPosition;
readonly attribute Float32Array poseModelMatrix;

Float32Array getViewMatrix(XRView view);
};

Expand Down Expand Up @@ -1032,36 +908,6 @@ interface XRWebGLLayer : XRLayer {
static double getNativeFramebufferScaleFactor(XRSession session);
};

//
// Coordinate Systems
//

[SecureContext, Exposed=Window] interface XRCoordinateSystem : EventTarget {
Float32Array? getTransformTo(XRCoordinateSystem other);
};

enum XRFrameOfReferenceType {
"head-model",
"eye-level",
"stage",
};

dictionary XRFrameOfReferenceOptions {
boolean disableStageEmulation = false;
double stageEmulationHeight = 0.0;
};

[SecureContext, Exposed=Window] interface XRFrameOfReference : XRCoordinateSystem {
readonly attribute XRStageBounds? bounds;
readonly attribute double emulatedHeight;

attribute EventHandler onboundschange;
};

[SecureContext, Exposed=Window] interface XRStageBounds {
readonly attribute FrozenArray<DOMPointReadOnly> geometry;
};

//
// Input
//
Expand Down Expand Up @@ -1111,16 +957,6 @@ dictionary XRSessionEventInit : EventInit {
required XRSession session;
};

[SecureContext, Exposed=Window,
Constructor(DOMString type, XRCoordinateSystemEventInit eventInitDict)]
interface XRCoordinateSystemEvent : Event {
readonly attribute XRCoordinateSystem coordinateSystem;
};

dictionary XRCoordinateSystemEventInit : EventInit {
required XRCoordinateSystem coordinateSystem;
};

[SecureContext, Exposed=Window,
Constructor(DOMString type, XRInputSourceEventInit eventInitDict)]
interface XRInputSourceEvent : Event {
Expand Down
Binary file added frame-of-reference-flow-chart.JPG
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading