Skip to content

Commit

Permalink
Stereo sound (#102)
Browse files Browse the repository at this point in the history
From @cxrodgers 👍 

This is a PR for issue #94

This implements multi-channel (e.g., stereo) sound. 

Some major changes:
* The sound classes in autopilot.stim.sound accept a `channel` argument. (Right now, only Noise does this.) If this is `None` (the default), a mono (1-dimensional) sound is produced, exactly as before. If this is 0, a Noise burst only in the first channel is produced. If 1, a Noise burst only in the second channel is produced.
* Multi-channel sounds are properly padded to the correct chunk length in Jack_Sound.chunk (implemented here: https://github.com/cxrodgers/autopilot/blob/4dfe9e77687a8a9083ee7903136f99cc7e63671f/autopilot/stim/sound/sounds.py#L223)
* The preference OUTCHANNELS is now interpreted differently. If empty string, then plays in mono mode (same sound to all speakers). If a list, connect channels to physical speakers in that order. If an int, treat as a list of length one. (implemented here: https://github.com/cxrodgers/autopilot/blob/4dfe9e77687a8a9083ee7903136f99cc7e63671f/autopilot/stim/sound/jackclient.py#L160)
* I removed padding code in the process() function because 1) this should already be done by Jack_Sound.chunk; 2) process probably doesn't have time to do this kind of thing; 3) the padding code was causing errors on multi-channel sounds, would need to be reimplemented as in Jack_Sound.chunk, but see reason number 1.
* OUTCHANNELS default in prefs.py changes to empty string (mono mode, no changes to Task code needed). The old default of [1] suggested 1-based indexing and in any case produced essentially mono results.
* NCHANNELS deprecated in prefs.py


Error are raised if:
* The length of OUTCHANNELS is longer than the number of available ports
* OUTCHANNELS indicates mono mode, but multi-channel audio is provided
* OUTCHANNELS is length N, but M-channel audio is provided, where N != M. However, 1-dimensional (mono) audio can be provided, and this will play to all N speakers.

jonny had suggested using None as the OUTCHANNELS flag for "connect no speakers", but [] works for this more naturally. Also, None cannot easily be put into the json.

Known issues:
* Presently only Noise can produce multi-channel output. Others can only produce mono output, as before. Should be an easy improvement.
* Only works for Jack, not pyo! 

Not tested!
* pyo is not tested!
* AUDIOSERVER = 'docs' is not tested!
* continuous mode is not tested!

Tested on:
* jackd from APT and (less thoroughly) jackd built from source

I know this is a big change, please lmk if changes are requested/needed!

* first pass at stereo

* doc; remove extra whitespace; reorganize order of imports

* doc Noise

* doc and reorganize the way Noise is generated

* fix typo

* intify channel if possible

* remove duplicate outports code

* typo

* handle stereo or mono output

* new way of init outports

* remove empty lines and comments

* doc

* move soudn writing code to its own method

* cherry pick a commit to init stereo_output, which required a merge on init_logger

* fix chunking and padding for multi-channel sound

* removing debug statements

* reimplement mono sound

* fix typo; error messages; use empty string instead of None as mono flag

* remove comment

* change default of OUTCHANNELS to empty string (mono)

* deprecate NCHANNELS

* adding test_sound.py

* catch ModuleNotFoundError

* server_type is not accessible via self

* trace out FS; finish test_init_noise

* put various durations into test function

* flesh out mono and stereo tests

* comments

* change calling functions to pytest parameterizations

* deprecation for prefs

* replace SOUND_LIST with autopilot.get, make placeholder Stim metaclass to make it work when audio server is undefined

* fix registry with plugin present

* warn if jackd server sampling rate doesn't match prefs

Co-authored-by: sneakers-the-rat <JLSaunders987@gmail.com>
  • Loading branch information
cxrodgers and sneakers-the-rat authored Jul 27, 2021
1 parent ef38398 commit 4dfdfa9
Show file tree
Hide file tree
Showing 11 changed files with 838 additions and 197 deletions.
6 changes: 3 additions & 3 deletions autopilot/core/gui.py
Original file line number Diff line number Diff line change
Expand Up @@ -1569,7 +1569,7 @@ class Add_Sound_Dialog(QtWidgets.QDialog):
Presents a dialog to define a new sound.
Makes a selection box to choose the sound type from
:py:data:`.sounds.SOUND_LIST` , and then populates edit boxes
``autopilot.get_names('sound')``, and then populates edit boxes
so we can fill in its `PARAMS` .
Attributes:
Expand All @@ -1584,7 +1584,7 @@ def __init__(self):
# Sound type dropdown
type_label = QtWidgets.QLabel("Sound Type:")
self.type_selection = QtWidgets.QComboBox()
self.type_selection.insertItems(0, sounds.SOUND_LIST.keys())
self.type_selection.insertItems(0, autopilot.get_names('sound'))
self.type_selection.currentIndexChanged.connect(self.populate_params)

# Param form
Expand Down Expand Up @@ -1621,7 +1621,7 @@ def populate_params(self):
self.type = self.type_selection.currentText()
self.param_dict['type'] = self.type

for k in sounds.SOUND_LIST[self.type].PARAMS:
for k in autopilot.get('sound', self.type).PARAMS:
edit_box = QtWidgets.QLineEdit()
edit_box.setObjectName(k)
edit_box.editingFinished.connect(self.store_param)
Expand Down
17 changes: 13 additions & 4 deletions autopilot/prefs.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,10 @@
This iteration of prefs with respect to work done on the `People's Ventilator Project <https://www.peoplesvent.org/en/latest/pvp.common.prefs.html>`_
If a pref has a string for a ``'deprecation'`` field in :data:`.prefs._DEFAULTS` , a ``FutureWarning``
will be raised with the string given as the message
"""

# this is strictly a placeholder module to
Expand All @@ -62,7 +66,6 @@
# Prefs is a top-level module! It shouldn't depend on anything else in Autopilot,
# and if it does, it should carefully import it where it is needed!
# (prefs needs to be possible to import everywhere, including eg. in setup_autopilot)

import json
import subprocess
import multiprocessing as mp
Expand Down Expand Up @@ -358,15 +361,16 @@ class Scopes(Enum):
},
'NCHANNELS': {
'type': 'int',
'text': "Number of Audio channels",
'text': "Number of Audio channels (deprecated; used OUTCHANNELS)",
'default': 1,
'depends': 'AUDIOSERVER',
"scope": Scopes.AUDIO
"scope": Scopes.AUDIO,
'deprecation': "Deprecated and will be removed, use OUTCHANNELS instead"
},
'OUTCHANNELS': {
'type': 'list',
'text': 'List of Audio channel indexes to connect to',
'default': '[1]',
'default': '',
'depends': 'AUDIOSERVER',
"scope": Scopes.AUDIO
},
Expand Down Expand Up @@ -425,6 +429,11 @@ def get(key: typing.Union[str, None] = None):
return globals()['_PREFS']._getvalue()

else:
# check for deprecation
dep_notice = globals()['_DEFAULTS'].get(key, {}).get('deprecation', None)
if dep_notice is not None:
warnings.warn(dep_notice, FutureWarning)

# try to get the value from the prefs manager
try:
return globals()['_PREFS'][key]
Expand Down
9 changes: 9 additions & 0 deletions autopilot/stim/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

from autopilot.stim.managers import Stim_Manager, Proportional, init_manager


class Stim(object):
"""
Placeholder stimulus meta-object until full implementation
"""


if prefs.get('AGENT') == "pilot":
if 'AUDIO' in prefs.get('CONFIG'):
from autopilot.stim.sound import sounds


16 changes: 8 additions & 8 deletions autopilot/stim/managers.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@
import pdb
from collections import deque
import numpy as np
import autopilot
from autopilot import prefs
if prefs.get('AGENT') and prefs.get('AGENT').upper() == 'PILOT':
if 'AUDIO' in prefs.get('CONFIG') or prefs.get('AUDIOSERVER') is not None:
from autopilot.stim.sound import sounds
# TODO be loud about trying to init sounds when not in config

def init_manager(stim):
if 'manager' in stim.keys():
Expand Down Expand Up @@ -115,7 +115,7 @@ def do_bias(self, **kwargs):
def init_sounds(self, sound_dict):
"""
Instantiate sound objects, using the 'type' value to choose an object from
:data:`.sounds.SOUND_LIST` .
``autopilot.get('sound')`` .
Args:
sound_dict (dict): a dictionary like::
Expand All @@ -133,10 +133,10 @@ def init_sounds(self, sound_dict):
for sound in v:
# We send the dict 'sound' to the function specified by 'type' and '
# ' as kwargs
self.stimuli[k].append(sounds.SOUND_LIST[sound['type']](**sound))
self.stimuli[k].append(autopilot.get('sound', sound['type'])(**sound))
# If not a list, a single sound
else:
self.stimuli[k] = [sounds.SOUND_LIST[v['type']](**v)]
self.stimuli[k] = [autopilot.get('sound', v['type'])(**v)]

def set_triggers(self, trig_fn):
"""
Expand Down Expand Up @@ -376,10 +376,10 @@ def init_sounds_grouped(self, sound_stim):
for sound in v:
# We send the dict 'sound' to the function specified by 'type' and '
# ' as kwargs
self.stimuli[group_name][k].append(sounds.SOUND_LIST[sound['type']](**sound))
self.stimuli[group_name][k].append(autopilot.get('sound', sound['type'])(**sound))
# If not a list, a single sound
else:
self.stimuli[group_name][k] = [sounds.SOUND_LIST[v['type']](**v)]
self.stimuli[group_name][k] = [autopilot.get('sound', v['type'])(**v)]


def init_sounds_individual(self, sound_stim):
Expand All @@ -403,10 +403,10 @@ def init_sounds_individual(self, sound_stim):
self.stim_freqs[side] = []
if isinstance(sound_params, list):
for sound in sound_params:
self.stimuli[side].append(sounds.SOUND_LIST[sound['type']](**sound))
self.stimuli[side].append(autopilot.get('sound', sound['type'])(**sound))
self.stim_freqs[side].append(float(sound['management']['frequency']))
else:
self.stimuli[side].append(sounds.SOUND_LIST[sound_params['type']](**sound_params))
self.stimuli[side].append(autopilot.get('sound', sound_params['type'])(**sound_params))
self.stim_freqs[side].append(float(sound_params['management']['frequency']))

# normalize frequencies within sides to sum to 1
Expand Down
12 changes: 11 additions & 1 deletion autopilot/stim/sound/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1,11 @@
from autopilot.stim.sound.sounds import SOUND_LIST
"""Module for generating and playing sounds.
This module contains the following files:
sounds.py : Defines classes for generating sounds
jackclient.py : Define the interface to the jack client
pyoserver.py : Defines the interface to the pyo server
The use of pyoserver is discouraged in favor of jackclient. This is
controlled by the pref `AUDIOSERVER`.
"""
Loading

0 comments on commit 4dfdfa9

Please sign in to comment.