Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add audio mixing into data model and TTML representation #105

Closed
nigelmegitt opened this issue Dec 22, 2022 · 4 comments · Fixed by #118
Closed

Add audio mixing into data model and TTML representation #105

nigelmegitt opened this issue Dec 22, 2022 · 4 comments · Fixed by #118
Assignees

Comments

@nigelmegitt
Copy link
Contributor

We say surprisingly little at the moment about how to define audio mixing and link in audio resources. It's technically there via the profile definition, but it needs to be added to the data model and the TTML representation so it's clear how to use it.

@nigelmegitt
Copy link
Contributor Author

nigelmegitt commented Mar 1, 2023

Planning for this:

  • Add Audio to data model
  • Add Mixing Instruction to data model
  • Define Audio as either an <audio> with a src attribute, as a child of a Text (<p>) or a TTS directive on a Text, being tta:speak or tta:pitch
  • clipBegin and clipEnd to be supported
  • xml:lang to be supported? If so, must match xml:lang of parent
  • Define Mixing Instruction as a static value of tta:pan or tta:gain or a list of <animate> elements that vary those attributes (to support fill="freeze") - can be added to Script Events, Texts and Audios.
  • Add examples
  • Add informative explanation of audio mixing semantics - maybe use something like https://w3c.github.io/adpt/#webaudio-section ?
  • Make sure that profile feature requirements are complete and adequate for audio

@nigelmegitt
Copy link
Contributor Author

While working on the specification for adding audio recordings I reminded myself of the various ways in which an audio recording can be embedded and referenced, of which there are at least 5 in total. Requirement R15 of DAPT is clear that both referenced and embedded options need to be available, but should we be syntactically restricting the options for each? Will raise as separate issues.

@KEEFER82
Copy link

KEEFER82 commented Oct 7, 2024

@nigelmegitt
Copy link
Contributor Author

@KEEFER82 right, R15 there says:

A document must be able to reference audio tracks either included as binary data within the document or separately.

Nevertheless, I am interested in implementation experience to guide the related open issues #113, #114 and #115.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants