Live coding suggestions

[totalgee]

Thanks again for this project! Last night I used Twigl to to perform visuals, live coding from-scratch to accompany music in eight different mini-sessions (nine minutes each)... I usually use my own live coding environment, but it was a pleasure to work within the minimalism of Twigl. The only thing I really missed was some audio reactivity.

I think it's good if Twigl stays quite minimal (though of course, this is up to you). But I can think of a few things that might be interesting, to use it in a live coding (visuals) context:

• support live audio reactivity. Doesn't need to be full set of frequency bins or a texture, or even three (high/mid/low) frequencies...even just a single volume uniform would be nice. With optional smoothing/filtering would be nice. I assume one can get the mic input from the browser?
• (alternative solution) allow a way to send arbitrary values to Twigl, to be available as uniforms. Then, an external program (e.g. SuperCollider) could send audio info, or MIDI slider info, perhaps via OSC or something.
• optionally allow the clock/time to continue even after recompilation of the shader, so it doesn't always reset back to zero. (With a button to reset it when desired).
• optionally a scaling factor on the clock time, so you can adjust it to match musical tempo (approximately). The clock should continue forward as you adjust tempo (not just implemented in the shader as t * speedScale), so you can stop or slow down time (again, an added bonus would be to be able to set this time scale from an external place, e.g. using OSC).
• a few more key shortcuts, e.g. one to toggle on and off the code display (a shortcut for the existing button).

I'm not saying these should be implemented, but I thought I'd mention these as things that would make it more useful for live coding visuals (for performance). But as I say, a big part of Twigl's attraction to me is its minimalism, so maybe adding too much would ruin that, and make things unnecessarily complicated. Sometimes simpler tools are better. And having constraints is good for creativity... (-;

[doxas]

@totalgee

Thanks for the great suggestions!
How much to support is a difficult question.

At the very least, twigl currently has a simple function to select any mp3 from the local environment to play, and accept its volume as a uniform variable.

It's not listed in readme.md because it's only optional, but you can take it as sound in classic mode, or s in GEEKEST mode, etc. This is a single floating point value.

The feature you're looking for seems like a more flexible and powerful feature, but may be a bit excessive for twigl.
Nevertheless, I would like to give it some consideration.

[totalgee]

I noticed the sound file playback feature, nice! If it's "simple" it would be nice to have a variation that instead took the sound from the mic input (after giving permission to the browser to access it), and expose that volume as the sound (or s) uniform. This is similar to what things like Hydra (https://hydra.ojack.xyz/) do.

[digitalsignalperson]

Cool to see the sound feature. Similarily would be really cool to be able to either provide an mp4 stream URL or upload a local video file to access the frames as textures

Oh, also ability to provide a URL to an mp3 stream to make it fully sharable.