Nice work!

[shyndman]

Hi there,

Stumbled across your library while I was trying to debug a pipeline problem I was having, and it was a huge help.

Just wanted to say thanks :)

[shyndman]

Quick update: I've started extending your project. I started with an envelope follower, and I'm moving on to a vocoder.

This is my first experience with React, believe it or not...it's been fun, and I like how you have thing set up.

[michalochman]

@shyndman cool! Feel free to open pull requests here with new modules, I am open to collaboration on this project.

[shyndman]

Oh, great.

Do you have a lot of experience in this area? I'm new to it, but eager to learn more. I'm basically reading everything I can get my hands on.

[michalochman]

I have a lot of experience with React. However, when it comes to Web Audio API and Digital Signal Processing this project was my way of learning them.

[shyndman]

Well, I like your React. Quick demo video of the vocoder. It's not perfect yet, but getting there.

Ignore the big visualization. It's there to help me debug the thing (and is actually likely to get a lot more complex).

vocoder.mov

[michalochman]

Nice! 🔥

I would not ignore the visualisation as my primary need for this project was to visualise the signal chain and understand what effect various components have on the signal – that's why I added Analyser and Meter.

[shyndman]

Thanks! I've kept going all day on it. I've had the same desire already. I was thinking fancy tooltips on the connection ports. Frequency spectrums for signal ports, scalars for a-rates....and maybe a scrolling line graph for the k-rates? Just thinking about loud here. I wonder if we could even animate the patch cables in some interesting way... Man...that standardized-audio-context library really threw me off. I was seeing way more worklet processors being created than what I expected. I had no idea what it was — debugged the worklet-loader, and the related framework code, without success. So I started digging into that library, and after a lot of reading, I found that it creates a "backup" audio context in a situation that I was regularly encountering, and my processors were being kicked off in two places at roughly the same time. Soooo, basically, it wasn't a problem at all...but was a lot of work to figure it out. :) Also, do you find the build times kind of slow? My laptop is terrible, but I thought you might have tips in any case. Scott

…

On Fri, Nov 19, 2021 at 1:36 PM Michał Ochman ***@***.***> wrote: Nice! 🔥 I would not ignore the visualisation as my primary need for this project was to visualise the signal chain and understand what effect various components have on the signal – that's why I added Analyser and Meter. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAKKVWK3XWU5RCGYUZUMJTUM2KLTANCNFSM5HQMNU5Q> .

[michalochman]

I was thinking fancy tooltips on the connection ports. Frequency spectrums for signal ports, scalars for a-rates....and maybe a scrolling line graph for the k-rates? Just thinking about loud here. I wonder if we could even animate the patch cables in some interesting way...

I'm up for building more tools.

Man...that standardized-audio-context library really threw me off. I
was seeing way more worklet processors being created than what I expected.
I had no idea what it was — debugged the worklet-loader, and the related
framework code, without success.

So I started digging into that library, and after a lot of reading, I found
that it creates a "backup" audio context in a situation that I was
regularly encountering, and my processors were being kicked off in two
places at roughly the same time. Soooo, basically, it wasn't a problem at
all...but was a lot of work to figure it out. :)

Good to know you figured it out. I have added standardized-audio-context, so the app can work on browsers other than Chrome. It was not easy from TypeScript perspective, but I think was worth doing.

Also, do you find the build times kind of slow? My laptop is terrible, but
I thought you might have tips in any case.

Build times are not bad for me, the app is usually available within 10-15 seconds of starting. However, my laptop is trying to fly away if I have a lot of analysers and meters in the graph.

[shyndman]

Build times are not bad for me, the app is usually available within 10-15 seconds of starting. However, my laptop is trying to fly away if I have a lot of analysers and meters in the graph.

So I discovered something that explains that a bit. Every time an AudioWorkletNode is created, you'd expect to see a single AudioWorkletProcessor, but we're actually getting 2. And they don't get cleaned up properly. It impacts every processor.

I have confirmed that hot module reloading is to blame. I got a lot of headroom for adding more nodes after disabling it (I react-scripts ejected).

Release builds were never affected, because no HMR.

[michalochman]

Something firing twice in development mode sounds a lot like StrictMode component. Can you see if you can reproduce this behaviour even with this component removed from main index.tsx file?

…

On Sat, 20 Nov 2021 at 20:32, Scott Hyndman ***@***.***> wrote: Build times are not bad for me, the app is usually available within 10-15 seconds of starting. However, my laptop is trying to fly away if I have a lot of analysers and meters in the graph. So I discovered something that explains that a bit. Every time an AudioWorkletNode is created, you'd expect to see a single AudioWorkletProcessor, but we're actually getting 2. And they don't get cleaned up properly. It impacts every processor. I have confirmed that hot module reloading is to blame. I got a lot of headroom for adding more nodes. Release builds were never affected, because no HMR. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAMNSIUGEFC7BXSYHDQVPNLUM7ZV7ANCNFSM5HQMNU5Q> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[shyndman]

It isn't firing twice. The WorkletAudioNode is only being constructed once on the react side. Two worklets appear.

[shyndman]

Side thought: I'd like to see tweakable gain on every input and output. I expect the operation is basically free, especially in the gain 0 and 1 cases.

[michalochman]

I would suggest creating new issues for feature requests – it will be easier to keep track of them. For instance, the idea of black-boxed nodes (#3) is to be able to define "complex" nodes made out of "primitive" nodes, e. g. an oscillator node connected to gain input with the gain output being the only output of the complex node. In my opinion, it would cover the scenario you are describing. If you look at the DelayEffect component, it's how it works already. However, I would like this to be defined during runtime rather than in code.

I am not particularly attached to the "black-boxed" name, it mostly comes from black-box testing, where we describe the system by its inputs and outputs, not its internal components. However, I think the name would serve well.

[shyndman]

The vocoder is 87 nodes internally. 😮 That number will grow.

I hear you on defining in the UI, but it's still too clunky...I have some ideas on that too.

OK, I will add to your feature requests.

Also, want a reproduction on the bug I'm talking about? You can see for yourself.

[michalochman]

Also, want a reproduction on the bug I'm talking about? You can see for yourself.

Sure, why not.