-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
4cda768
commit 02f5753
Showing
1 changed file
with
205 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,205 @@ | ||
# Audio Components | ||
|
||
This project was begun as a way to learn Polymer. | ||
Polymer has evolved since, so this version is a major rewrite to align with polymer 3. | ||
The most significant change is the use of javascript modules rather than HTML imports. | ||
|
||
## Installation | ||
|
||
- download a .zip file (not necessary to clone unless you need to submit a pull request) | ||
- use `loader.js` to load everything except `resonance-audio.min.js` which must be loaded separately: | ||
+ `script src="loader.js"></script>` | ||
- alternatively, each element can be independantly loaded via a script tag: | ||
+ `<script src="audio-delay.js" type="module" crossorigin="anonymous"</script>` | ||
|
||
## Elements | ||
|
||
The package contains five distinct types of elements: | ||
|
||
- context (from which all others are inherrited, and which directly wraps the webaudio AudioContext node) | ||
- connectors | ||
- audio processing elements | ||
- UI elements | ||
- automation elements | ||
|
||
### The connectors build the connection graph. They currently include: | ||
|
||
- audio-context | ||
- audio-series | ||
- audio-parallel | ||
- audio-split | ||
- audio-feedback | ||
|
||
### The audio processing elements wrap the webaudio nodes which actually do the heavy lifting, as well as provide a UI by which to control the processing. They currently include: | ||
|
||
- audio-player | ||
- audio-destination | ||
- audio-gain | ||
- audio-filter | ||
- audio-delay | ||
- audio-panner | ||
- audio-oscillator | ||
- audio-convolver | ||
- audio-reverb | ||
- audio-compressor | ||
- audio-equalizer (10 band graphic equalizer) | ||
- audio-resonance (provides access to Google's resonance audio system which does room simulation) | ||
|
||
### The UI elements handle UI generation and currently include: | ||
|
||
- ui-number | ||
- ui-boolean | ||
- ui-list | ||
- ui-text | ||
|
||
### The currently implemented automation elements are "audio-control" and "audio-parameter". | ||
|
||
- Audio-control modifies specified parameters of it's parent element based on mathematical functions in the time domain. The free variable "t" is the current time stored in the webaudio context node which contains the graph. | ||
- Audio-parameter specifies the parameter to be automated and a javascript function which generates values for the parameter. It must be wrapped inside an audio-control element and must appear *after* the element whose parameters are being automated. See examples for more details. | ||
|
||
## declarative rather than imperative | ||
|
||
The graph is created declaratively based on the given html; no javascript is required. | ||
We wrap everything inside an audio-context element. This corresponds directly to web audio's context node. | ||
|
||
## Example 1 | ||
|
||
Here is the markup for a simple webaudio graph which takes an audio signal from a file and pipes it through a gain node such that volume can be adjusted, and out to the audio out (typically the computer's standard audio outs): | ||
|
||
``` | ||
<audio-context label="example1"><audio-series> | ||
<audio-player src="some-file.mp3"></audio-player> | ||
<audio-gain label="master volume"></audio-gain> | ||
<audio-destination></audio-destination> | ||
</audio-series></audio-context> | ||
``` | ||
|
||
|
||
The player contains three buttons: play/pause, forward, and back. | ||
Our web components automatically add UI for specifying audio source file, as well as for controling the gain. | ||
If the `label` attribute on the gain element were not present, it's UI would be hidden and it would act as a constant gain whose gain value could be set by a `gain` attribute. | ||
|
||
### Javascript example | ||
|
||
Here is the same graph, built via javascript using webaudio directly. | ||
|
||
``` | ||
var audio = new AudioContext (); | ||
var audioElement = document.querySelector("#myAudio"); | ||
var player = audio.createMediaElementSource (audioElement); | ||
var gain = audio.createGain(); | ||
player.connect (gain) | ||
.connect (audio.destination); | ||
``` | ||
|
||
The above builds the graph and connects all the nodes, but no UI is present. The author is responsible for adding UI elements to control the gain, and the player (or use built-in browser controls for the player). | ||
## Limitations | ||
|
||
- webaudio can deal with more than 2 channels, but this project assumes all elements either 1 or 2 channels | ||
|
||
## Basic Architecture | ||
|
||
All elements are implemented as web components, and create new HTML elements (known as custom elements in webcomponents parlents). They consist of an html template which generates the UI, and an associated JS class definition. | ||
|
||
In the javascript domain, all `audio-` elements inherrit from `_AudioContext_`, which itself inherrits from `PolymerElement`. all `ui-` elements inherrit from `UI` which lives in `ui.js`. | ||
|
||
There are a small set of extra libraries which comprise a simple component model and provide lower level implementations of many of the processing elements. | ||
|
||
- `audio-component.js` is the module which implements the component model and exports class `AudioComponent` | ||
- `room.js` is an additional component which works with `resonance-audio` and also inherrits from `AudioComponent` | ||
|
||
In the HTML domain, the entire HTML graph needs to be wrapped in a `<audio-context> ... </audio-context>` element. | ||
|
||
If a label attribute is supplied on an element, it display a UI; without a label, an element is hidden, but still takes part in the graph using parameters supplied in the markup. | ||
|
||
If a `hide` attribute is present on an element, the value of that attribute is used as a comma-separated list of field names to hide. A field name is the label shown in the UI when the element is used. | ||
|
||
Most parameters displayed in the UI can be set in the HTML via attributes. For instance, including the boolean attribute `bypass` on an element will boot it up in bypass mode; as long as the UI for the bypass control is not hidden, the user can change it's value at any time via a checkbox in the element's UI. | ||
|
||
### Accessibility | ||
|
||
Each component's UI lives in its own fieldset and is labeled via `legend` which wraps an `h2` element containing the label provided via the element's `label` attribute. If `label` is not provided, the entire UI for that element is hidden. If audio-parameter elements are present, they wrap their labels in `h3` elements. | ||
|
||
All elements are controlable via the keyboard. | ||
Shortcut keys can be specified in markup. The user can also add a shortcut to an element via the UI. | ||
|
||
Most controls have native keyboard control as defined via HTML5. However, we have added extra features. | ||
|
||
#### Numeric Parameters | ||
|
||
They come in two flavors: input of type `number` and inputs of type `range`. | ||
|
||
- number can be edited directly via standard editing keys; range is a simple slider | ||
- both types support movement via up/down arrows and page up/down, with page up/down moving by larger amounts | ||
- both types use control+home to set to largest allowed value, and control+end to set smallest value | ||
- both support saving current value via control+enter and swapping with current value via control+space | ||
- slider can be set to zero via the "0" key, and one via the "1" key | ||
|
||
## UI Elements | ||
|
||
The `ui-` elements generate single controls. The label attribute specifies a label for the control. | ||
The `label` or `name` attribute, whichever is present, is used as the name of the parameter to be manipulated. The element which uses the control must insure that the parameter it exposed via polymer's properties or observers mechanism. | ||
The value attribute pipes the result back to the current element's property being manipulated. | ||
|
||
Numeric controls can also specify min, max, and step, which are passed directly to the underlying html input element. | ||
|
||
|
||
## Control Elements | ||
|
||
The audio-control element allows one to specify a parameter to be automated, and a function to generate values based on the current time stored in the underlying webaudio context. The `audio-control` element must wrap the element to which automation is to be applied as it's first child. | ||
|
||
Any additional children of `audio-control` must be `audio-parameter` elements specifying at least the name of the parameter being automated. If a `function` attribute is specified in markup, then it is taken to be the default function used to generate parameter values; this can be manipulated in the UI as long as the `function` attribute is not present in the audio-parameter's hide list. | ||
|
||
|
||
The automator function is a simple javascript expression which is evaluated in the context of the controled element (i.e. the "this" keyword references the currently controled element). | ||
It references the "Math" object using the javascript "with" statement which allows one to reference Math functions via just their name (i.e. sin(t) instead of Math.sin(t)). The variable `t` is the free variable and it contains the current time, in seconds, of the currently running audio context. | ||
|
||
|
||
## Connectors | ||
|
||
### Series connector | ||
|
||
- each child is connected in series to its siblings, in source order | ||
- _in of first child connects to _in of the series | ||
- _out of last child connects to _out of the series | ||
|
||
### Parallel connector | ||
|
||
- _in of the parallel connector connects to _in of all children | ||
- _out of each child connects to _out of the parallel connector | ||
|
||
### Feedback | ||
|
||
The audio-feedback element sends its input to the input of its single child, and sends the output of it's child element to the output of the audio-feedback element, as well as sending it back to it's input, through a delay node. A delay node *must* be included in a closed loop in the graph, thus we provide a delay node by default here so the feedback will always work. | ||
|
||
|
||
|
||
### Split | ||
|
||
- split must have either 1 or 2 children | ||
+if 2 children then first child is connected to left channel of input and second to right channel, and outputs are merged back into a stereo signal at the output of split | ||
+if 1 child and no attributes, left of input goes to left of output | ||
- the boolean swap-outputs attribute reverses channels on the output | ||
+ if 2 children then reverses stereo channels at output | ||
+ if 1 child then left channel of input goes to right channel of output | ||
- the boolean swap-inputs attribute causes channel reversal at the input | ||
+ with 2 children, same as swap-outputs | ||
+ with 1 child, causes right of input to go to left of output | ||
- both swap-inputs and swap-outputs on a stereo source with 2 children have no effect | ||
+ both attributes on stereo source with 1 child pipes right of input to right of output | ||
|
||
#### Example 2 | ||
|
||
This sends the output of an oscillator to the right speaker only: | ||
|
||
The gain elements inside split hide their UI since they have no label. Their gain defaults to unity, so in a sense they are simply placeholders. The split is only being used here to swap channels. | ||
|
||
``` | ||
<audio-context><audio-series> | ||
<audio-oscillator label="oscillator" frequency="250"></audio-oscillator> | ||
<audio-split swapOutputs> | ||
<audio-gain></audio-gain> | ||
<audio-gain></audio-gain> | ||
</audio-split> | ||
``` |