OMI_audio_emitter

[robertlong]

This is a work in progress draft of a glTF 2.0 extension for audio emitters. It is inspired by the Mozilla Hubs audio-emitter component of theMOZ_hubs_components extension and the MSFT_audio_emitter extension. The properties are closely related to the HTML <audio> element and Web Audio AudioNode and PannerNode APIs.

Some differences from the Hubs/Microsoft extensions:

• Multiple global audio emitters can be placed on the scene
• Audio clips are defined outside of the audio emitter objects for reuse in multiple emitters.
• Audio clips can reference multiple audio files for better content negotiation in different browsers/game engines.
• Emitters are not triggerable via animations for simplicity. This may be added in another extension.
• The autoPlay property can either play the audio clip on scene load or when set to false, it can be triggered externally.

Rendered

Out of Scope:

• Audio triggers
• Audio effects / graphs

Research On Other Engines:
#2

[fire]

@humbletim

We want to compare multiple implementations of audio features in Game Engines before the meeting.

• Web Audio
• Godot Engine
• Unity
• Unreal Engine
• Wwise
• OpenAL

[fire]

The audio emitter uses an attenuation inner cone is not possible in Godot Engine.

Trying to look for a fallback.

[humbletim]

Would it make sense to include support for descriptive alt tags / aria labels as part of audio emitters? (or maybe an accessibility extension on its own?)

[robertlong]

Would it make sense to include support for descriptive alt tags / aria labels as part of audio emitters? (or maybe an accessibility extension on its own?)

I think it would make more sense to have an accessibility extension that can be applied to any node.

[robertlong]

The audio emitter uses an attenuation inner cone is not possible in Godot Engine.

Trying to look for a fallback.

Yeah, looking at the AudioStreamPlayer3D class it may require some additional code. I'll look into this further next week. Let me know if you have any ideas. I think we'll run into similar issues with Unity.

[takahirox]

To clarify for understanding audioClips, this extension doesn't limit video file formats. So that mimeType is always required, unlike glTF 2.0 core image which requires mimeType only for bufferView, to let user agents detect the video file format is supported. Is my understanding correct?

[takahirox]

In glTF 2.0 core spec, all angles are in radian. Should we follow it?

[fire]

I second radians.

[robertlong]

I'm not sure if I represented this correctly. The maximum is 2PI radians then. Idk how to represent that in JSON schema.

[robertlong]

Default value should be rounded down at 6 decimal places and maximum value should be rounded up at 6 decimal places to deal with floating point precision.

[robertlong]

Let's also clarify with the Khronos Working Group.

[robertlong]

Based on this discussion about the yfov and the punctual spotlight spec. I think we should go with a maximum and minimum value of 6.283185307179586. We can also write in the readme/description that the maximum value is 2 * PI.

This will work just fine in Javascript btw:

Math.PI / 180 * 360 === 6.283185307179586

6.283185307179586 * 180 / Math.PI === 360

[robertlong]

I've made this change now. I think it should be fine, but could use a second opinion.

[robertlong]

Notes from meeting:

• Use Radians to conform with glTF Spec
• Still need to look into file format types, but leaning towards MPEG if we can get away with it in Three.js, Unity, Unreal, and Godot
• Inverse distance model doesn't support max distance so don't use it in our example
• Document inner/outer cone and cone gain
• Determine what volume is specified as and what unit it should be represented as and how to implement in each engine, for example Godot which uses Db.

[humbletim]

2021.08.05 my additional gltf-extensions meeting notes:

• last week @robertlong completed initial three-omi importer/exporter

  • see: https://github.com/omigroup/three-omi
  • enables OMI_audio_emitter glTFs to be used across three.js ecosystem
  • (pending) later paired with omi-gltf-transform this may also help to reuse Spoke content (independently from Hubs)

• robert planning to create omi-gltf-transform command-line tool

  • will enable Hubs => OMI glTFs conversion
  • (maybe) will enable reverse too (OMI => Hubs glTFs)

• robert made significant progress on omi-blender-gltf addon

  • see: https://github.com/omigroup/omi-blender-gltf
  • plan is to support adding OMI_audio_emitter elements + export to glTF from blender
  • plugs into existing khronos glTF add-on using its official python hooks
  • exported scenes anticipated to be compatible with three-omi
  • (later on) anticipated to be compatible with @fire's Godot importer
  • (help needed) robert is planning to circle back next week after omi-gltf-transform, but if anyone has time to sooner feel free to jump in (python) and help prototype eg: audio node properties panel

• @fire has prototyped a Godot MozHubs Importer

  • https://github.com/V-Sekai/godot-blue-steel-wolf
  • right now just a proof of concept specific to .glb's with Spoke/Hubs extensions
  • next steps are to adapt for supporting .glb's with OMI_audio_emitter extension
  • @RangerMauve and @KooIaIa potentially interested in collaborating on this

• reminder: next meeting scheduled for Thursday August 12th 2:30PM PDT <t:1628803800:f>

[robertlong]

We still need feedback on Unity and Unreal. How should we specify the falloff for the cone so that it can be implemented correctly in each engine? If we give people the algorithm for finding the volume given the listener's position and the audio emitter's transform, is that enough? Are there other features here that are incompatible with either engine? Are we missing properties that would make this hard to use in these engines?

We'd love your help if you have experience with audio in Unity or Unreal. Comment or leave a review if you're interested in helping out.

[mrmetaverse]

@robertlong (I have pinged you a lot today, sorry) Do you know what the status is of this? Are we waiting to merge for a vote or something? :D

[robertlong]

@robertlong (I have pinged you a lot today, sorry) Do you know what the status is of this? Are we waiting to merge for a vote or something? :D

After these final changes are in, we need to write a markdown explainer. Including explaining the various audio rolloff algorithims.

We also need two implementations of the spec before it can go to a vote. So far we have three-omi and we've been talking about Unity and Godot implementations.

In parallel we're working on Blender tooling to generate new glTF content with this extension.

Once these are done we can take it to a vote.

[npolys]

Worthy of study and harmonization:
There is also an example of a spatialized 3D Web Specification (ISO X3D) that is also compatible with the W3C WebAudio standard: https://web3d.org/documents/specifications/19775-1/V4.0/Part01/components/sound.html