Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PCM audio capture to browser audio input library #95

Merged
merged 5 commits into from
Dec 10, 2024

Conversation

mnemitz
Copy link
Contributor

@mnemitz mnemitz commented Dec 10, 2024

Problem

When recording audio in the browser, it's not very straightforward to capture PCM audio data, since the MediaRecorder API only provides encoded audio, and the scriptProcessor approach is both deprecated and flaky in terms of performance.

Solution

  1. Update the existing @speechmatics/browser-audio-input package to expose
    • a PcmRecorder class
    • A minified PCM audio processor script, which can be run inside an AudioWorklet thread. See the MDN example here.
    • Updated README to explain how to consume the above script.
  2. Update @speechmatics/browser-audio-input-react to include:
    • new hook usePcmAudioRecorder, to control a context-backed instance of the PcmRecorder class
    • new hook usePcmAudioListener, to subscribe to incoming PCM audio
    • Context provider PcmAudioRecorderProvider to provide the React context needed for the previous two
  3. Update the NextJS example to use the above React bindings instead of the deprecated createScriptProcessor approach.

@mnemitz mnemitz changed the title Draft: Add PCM audio capture to browser audio input library Add PCM audio capture to browser audio input library Dec 10, 2024
Copy link

@anjz anjz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Very nice explanation in the README's

@mnemitz mnemitz merged commit 8dc06bd into main Dec 10, 2024
3 checks passed
@BellaT BellaT deleted the DEL-23076-PCM-library branch December 10, 2024 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants