Advanced audio effects in Ebiten

[mortenson]

👋 Hi there - I was looking into https://github.com/faiface/beep recently, and noticed two features that would be nice to replicate in Ebiten:

• Audio resampling, specifically for changing pitch
• Panning, to support directional sound (see https://github.com/faiface/beep/tree/master/examples/doppler-stereo-room)

Would these be the kinds of features we could support in Ebiten? I thought of moving to use beep but it would require completely moving away from Ebiten audio, I think.

[hajimehoshi]

Hi, thank you for the suggestion!

The suggestion seems good, and we need more discussion.

Audio resampling, specifically for changing pitch

This should be worth considering. Do you mean changing pitch and speed at the same time? If not, i.e., if you want to change only pitch, this is not a trivial issue.

There is an internal resampling implementation: https://github.com/hajimehoshi/ebiten/blob/master/audio/internal/convert/resampling.go but this implementation might be inefficient. There is a tradeoff between quality and efficiency.

Panning, to support directional sound

There is an example to do that: https://github.com/hajimehoshi/ebiten/blob/master/examples/audiopanning/main.go

There are multiple ways to do that, so we would need further discussion (See also #1224)

I thought of moving to use beep but it would require completely moving away from Ebiten audio, I think.

I think that's a valid option. Fortunately the audio package is independent from Ebiten core.

[mortenson]

Thanks! How did I miss the panning example...

For pitch, I meant just changing the pitch and keeping the speed the same. For example, to support musical instruments in game from one original audio file/sample.

[hajimehoshi]

For pitch, I meant just changing the pitch and keeping the speed the same. For example, to support musical instruments in game from one original audio file/sample.

I see. Then, this is not a trivial issue. Fourier transforming would solve this but I don't know if this is efficient and fast enough for real-time audio.

[hajimehoshi]

Note: RPG Maker MZ has a feature to change pitch but this changes the speed at the same time.

[mortenson]

If it's much easier to just change pitch, then I think that'd be fine too. For a short sample or a small pitch range the speed wouldn't be extremely noticeable.

[hajimehoshi]

If it's much easier to just change pitch

I meant that changing only one of pitch or speed is pretty hard. Changing both at the same time is easy.