expose endTime and endTick

[jacopomaroli]

can you please consider exposing
this._sequencer.endTime as at.player.playbackEndTime
and
this._sequencer.endTick as at.player.playbackEndTick?

Use case is: I need to keep the tab in sync with an audio player. I know endTime is available in the window.at.playerPositionChanged event.
When I play the audio player and sync the time with the tab, I check the audio player current time did not overshoot the tab end time before playing the tab but now I lack that info at the first iteration

I'll open a PR as well since it's an easy fix

edit: PR is here #521

[Danielku15]

I do not yet get why the playerPositionChanged is not sufficient for your usecase. So let's step first back from your solution proposal and check first what your actual issue is.

According to your use case description you are trying to sync alphaTab with an audio player. From this I would derive that you actually need a mechanism to set the cursor of the tab to a given time (second) position which is coming from the audio player. Am I right with this assumption? If you have a separate audio player I would guess the synthesizer does not really have a relevance for you. Of course if the user might switch between synth and let's say mp3, you would need to send the current position once to the player.

All this goes with the assumption that the audio played in the source is already 100% synchronized with the synthesized audio and we can safely assume that there are no differences.

[jacopomaroli]

yes your assumptions are correct. Although, I had a corner case where the audio player goes beyond endTime. the tab track stops (window.at.player.state === 0) and then the user seek the audio player back. the tab here is supposed to resume playing so the code in the sync logic looks something like

function sync() {
 //...
  if(window.at.timePosition < window.atEndTime && window.at.player.state === 0){
    window.at.play()
  }
}

it would have been very convenient for me to reuse that code to trigger window.at.play() when I hit play in the audio player the first time instead of triggering explicitly window.at.play() .
it's basically this

function onPlay() {
//...
sync()
//...
}

vs

function onPlay() {
  // ...
  if (window.atEndTime) {
    sync()
  } else {
    window.at.play()
  }
  // ...
}

In that way I can just spam sync() all over my code without thinking too much about when to play alphatab.
It's not a big deal I can keep the workaround. I thought it was a small change worth throwing a PR but seems to be less trivial than I expected

sorry I dragged you into this mess of a project I'm trying to put up :D

[Danielku15]

In general adding support to synchronize the alphaTab playback (cursor highlighting) with an external audio/video source is on the roadmap. The approach there will work a bit different than you're doing it. I will try to explain the idea below, maybe you can use this idea and build this system already outside alphaTab (happy to accept contributions).

Your approach
I think the main issue on your side is rather the assumption to enforce the same start/end time in the audio and in the synth. If we stick to your approach (neglecting time shifts between audio/synth) my proposal would be the following:

1. You set the synth volume to 0 (or you do not load a soundfont) to avoid hearing the synth.
2. I add new parameters to the midiLoaded event which will tell you the endTick/endTime once the midi has been imported (as requested here). But you could also just use the initial playerPositionChanged trigger to do this. IIRC it should be anyhow fired once when the midi is loaded but I need to double check this.
3. You take the real audio end time and the synth end time to compute the right playbackSpeed alphaTab has to use. This way the playback on alphaTab and the external audio should be in sync and end at the same time.
4. When seeking and potentially in some regular intervals during playback you use the real audio time position, combine it with the ratio to set the timePosition in alphaTab. This might avoid drifts.

Delays between the synth start and real audio start could be handled also by triggering the synth-play only once the external audio notifies about the start position (on seeking you would have to consider this offset too).

The only thing I just don't get still is why you cannot use the data already provided by the playerPositionChanged event.

Future approach of alphaTab
What alphaTab today does is just taking the tick position, searching for the related beat in a lookup created during midi generation and then place the cursor there and animate it to the next one with a given duration. Where this tick position comes from (external source or the synth) does not really matter. So we just need to build a "bit of code" which can translate the time position of an external audio source, to the right midi tick position.

The base idea for this is to use a series of "stop points" marking the audio position and the related tick position:

By default there will be just 2 stop points:

• Audio Time 0 -> Midi Tick 0.
• Audio Time (end) -> Midi Tick (end).

And alphaTab will distribute all bars across this timeline (respecting beats and tempo changes) and build some internal lookup which can translate a given input time position to the internal midi tick. When the audio/video plays, the current audio time will be sent to alphaTab and it will use the same cursor system as today show the currently played beat and how long it will play timewise.

A developer could then provide additional stop points to "correct" any delays or time differences. e.g.

• Audio Time 5s -> Midi Tick 0
• Audio Time 30s -> Midi Tick 20000
• Audio Time 2min -> Midi Tick 80000
• Audio Time (end) -> Midi Tick (end).

The plan is to even create a visual editor for that where you can drag the markers along a timeline and see the cursor/video/spectrogram for testing.

[Danielku15]

I played around with the proposed intermediate solution and made some extensions that make it more convenient to use. The usage of playerPositionChanged was a bit tricky so I really added new parameters to the midiLoaded event.

On the seeking I noticed that Chrome currently has some issues with MP3 files. Setting the currentTime does not really precisely set the time to this position. It tells you afterwards it plays at this position, but actually some other audio a few seconds delayed is played. With OGG files it works fine though:

https://demo.alphatab.net/crm/sync/index.html (a normal speed OGG exported from Guitar Pro)
https://demo.alphatab.net/crm/sync/fast.html (a tempo 104 instead of 69 OGG exported from Guitar Pro)

Actually it works better than expected already including seeking 😉 Let me know if this is what you need, you can take the sample above as reference to update your codebase.

Known issues currently:

• When loading the midi file, you need to ensure that the playbackSpeed is set to 1, otherwise the ratio will be calculated wrongly.
• MP3 does not seem to work well with Chrome on Windows.

Update: also Firefox does not seek properly on MP3s.

[jacopomaroli]

This actually solves it
you could also just use the initial playerPositionChanged trigger to do this. IIRC it should be anyhow fired once when the midi is loaded
Now I can just spam sync() and it'll take care of starting alphatab when necessary.

Basically I was not setting my global atEndTime unless readyForPlayback was fired.
That event indeed triggers a couple of times when loading the tab before readyForPlayback.
I assumed no playerPositionChanged could be dispatched before readyForPlayback so I just moved that assignement after a bunch of early returns.

Few more details about my implementation:

My current sync implementation is basically setting at.timePosition to the player timePosition+offset on playerPositionChanged only if 250ms passed from the previous sync (but only if drifting more than 70ms otherwise it'll correct all the time).
Basing the syncing on the audio-player rather than the playerPositionChanged would produce too many chops as the sync operation would not be synced with alphatab synth events.
Anyway the chopping is not much of a problem; as you were saying I mute the synth and just show the notes/scales. Playing both audio should be happening only when transcribing the tab to ensure they're in sync as much as possible. Also unless the CPU is overwhelmed is not that bad

Also I don't really rely on the end time of the track being the same of the video cause when I transcribe a tab, the endTime is actually less than the video. I made a small editor which allows you to edit the tab with raw alphatex in relatime. (yeah interactive tab editing would be cool but that's a start)

I like the "stop point" approach. I remember seeing the part of the code where you're building the event map from last time when I was trying to get metronome events. Unluckily it's not yet exposed and IIRC it's buried in the worker. I Might try to build something on top of what you mentioned once events are exposed!

Your examples are really cool, the logic is very similar to what I'm doing. I like the one where you stretch the time. That'll definitively come handy to implement something like loop-up-to-speed.
I'm also doing the other way around (sync video to tab) when the user interact with the tab.
You made also a good point by using api.currentTime. I'm currently using window.performance.now() which is a bit wasted...

I'll DM a link to my POC in case you wanna have a sneak peak ;)

Thanks for the support!