Skip to content
This repository has been archived by the owner on Nov 10, 2021. It is now read-only.

Not precise enough #6

Closed
marioortizmanero opened this issue Dec 31, 2019 · 1 comment
Closed

Not precise enough #6

marioortizmanero opened this issue Dec 31, 2019 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@marioortizmanero
Copy link
Member

marioortizmanero commented Dec 31, 2019

The problem

This module works quite well when the recording starts as soon as a new song starts. But in a real situation, the recording starts after the following happens in Vidify:

  1. The API detects a new song: some APIs can have a delay of ~1000ms to notice this and may be straight up unusable. This is partly why Linux is only supported for now, since MPRIS can asynchronously recieve the changes and is more precise.
  2. The API notifies the GUI to start a new video with a Qt signal.
  3. The GUI receives the signal and runs the function to play the video.
    4. The URL of the audio to be downloaded is obtained with youtube-dl.
  4. The get_lag function is finally called.
  5. The lag is set in on_audiosync_done
    6. The player ALSO has lag. It's usually about 2000ms. And it depends on whether you're using VLC or Mpv (the latter has less lag).

VLC doesn't work well because it uses a large GoP size by default and the seeking isn't precise enough for the synchronization.

Ideas:

  • Record always in the background, and cut the track data once the video starts (hard to implement and may not even work well).
  • Have a continuous buffer with the songs that the players can reproduce. That way, the step number 6 can be avoided after the first song.
  • [DONE IN c5c8610df4924d7dd0d5385d9cc0391e2d004d45] Have a new option in the config with a manual offset for the audiosync feature. This delay depends on the hardware the app is ran on, so it would make sense that it'd need a bit of tweaking. But it could be annoying for the users.
  • [DONE IN youtube-dl integration & more intervals #8] The thread to download the audio should also get the URL with youtube-dl by its own. This way, the step number 4 (1) can be avoided, which is often the one that takes the longest.
  • [DONE IN d36cf9ee9940794bef67a8d006c90f26765a5c89] Shift with zeroes rather than rotating:

Original

img

Shifted

img

In this picture, the green shifted part at the beginning is the ending of the original track data. If at least it was empty, the confidence matching would be better. You can tell the shifting was actually quite precise, but the coefficient (0.060655 in this case) is affected by the initial part.

  • [DONE IN 783797a10fe8a3d1cb8c162d6ee1c193bc11f9bf] Rather than filling it with zeroes, just cut that part of the array. Filling with zeroes works, but there's a catch: most of the times, the delayed track is from YouTube. Similarly to the idea above, the YouTube track is filled with zeroes in one part, while the original track has audio. While it's more accurate than the previous version, it takes more segments to match (too many). Cutting the track can reduce precision, but it will be faster:

Original

Original

Shifted

Shifted

To avoid having too little precision, MIN_CONFIDENCE is raised to something like 0.95 (previously 0.75).

@marioortizmanero marioortizmanero self-assigned this Dec 31, 2019
@marioortizmanero marioortizmanero added the bug Something isn't working label Dec 31, 2019
@marioortizmanero marioortizmanero changed the title Not working well when recording starts late Not precise enough Jan 3, 2020
@marioortizmanero
Copy link
Member Author

Precision without noise has improved enough since this issue was created. The problem now is precision when there's noise. I'll open a new issue for that.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant