Video loading hangs when trying to load

[maplepy]

Behavior of the program
Using a VPN with one of the two following protocol makes the videos freeze, not load properly or just hang when loading
Note that closing Freetube and reopening it will "reset" the issue and load the video for a minute before hanging again

Expected behavior
The video is playing and loading properly

To Reproduce
Please add all steps to reproduce the behavior:

1. Connect to a VPN with Wireguard or OpenVPN
2. Open a video
3. Play it at 1080p
4. Enjoy the video loading after 30 seconds (time may not be exact)

Environment Information (please complete the following information):

• OS: EndeavourOS Linux x86_64
• OS Version: 5.14.9-arch2-1
• FreeTube version: v0.14.0 Beta
• Installation Method (If applicable): .AppImage and AUR
• Primary API Used: Local

Additional context
Note that it isn't a VPN speed issue since the video will load properly on YouTube with 2 Twitch streams opened in the background.

[maplepy]

This issue is the same as the #1797 but properly formatted

[petaded]

I have the same issue on Freetube: v0.14.0 beta using manjaro Kernel: 5.10.68-1-MANJARO - primary api: local - vpn provider mullvad
Here is an example video: https://youtu.be/Y6DFPYji2h4
Plays without buffering at 1440p on youtube at 2x speed, in freetube will only play at 360p without buffering.

Note: enabling "Proxy videos through Individious" seems to work around the issue for me - as soon as i enable this I have no buffering issues, disabling the setting brings back the buffering problem.

[7echLife]

I come from #1797 and also have issues with buffering while using VPN. I normally watch at 480p because of internet and low end hardware so 360p or higher I have issues. Issue happens after 5 seconds and it's constant.

[Publius1788]

This affects me as well. Speedtest with VPN connected is better than 50mbps up and down, and Youtube streams just fine at 1080p at 2x speed with VPN connected, but Freetube constantly buffers.

I've also noticed that closing Freetube and re-opening seems to improve the situation for a few minutes.

[peepo5]

this is a known issue we are trying to fix.

[efb4f5ff-1298-471a-8973-3d47447115dc]

This should be fixed try it out #1773 (comment)

[petaded]

Hi,

This is not fixed with your linked updates - at least for me.

Here is a screenshot of the networking using ./FreeTube-0.15.0-nightly-1277.AppImage from the development branch (look at the various mb loaded and time taken):

Here is the same video using https://www.youtube-nocookie.com/embed/CFIiwit8x7Q in my browser ( nothing else changed)

As you can see there is a considerable speed difference.

Also if i steal the videoplayback url from freetube under networking and open this in my browser, i experience the same poor performance.

I then noticed this issue (i know you are not using this lib but its a yt performance issue all the same) ytdl-org/youtube-dl#29326
So I decided to give yt-dlp a go (as its a suggested fix) instead of your node-ytdl-core. I updated freetube starting from the current development branch swapping node-ytdl-core for yt-dlp. Couldn't get DASH to work since it doesnt provide the indexRange or initRange and my lack of knowledege on whatever language you are using, dash, minimal time investment into this update etc etc. However legacy format worked fine.
Note; yt-dlp had some links that were marked as "maybe throttled" and some that were not marked like this, the maybe throttled did have poor performance on my computer compared to the others.

Here are my comparison numbers of yt-dlp in freetube vs node-ytdl-core using LEGACY format 720p 2x speed on the same computer starting both copies of the same video at the same time, numbers taken from networking tab dividing mb by time taken only used the first 2 mins for each video (2mins of real life time).

number in speed mb/s

Video ID	yt-dlp in freetube	normal freetube using node-ytdl-core	additional comment
CFIiwit8x7Q	0.59	0.075
ckbbY-fLLkI	0.399	0.074	yt-dlp hit full buffer hence stopped downloading, node-ytdl-core was 3 video mins behind due to stopping repeatedly
BbSjKyJopFw	0.095	0.071	yt-dlp was at full buffer the whole time, not a very strenuous video it seems
jaWD31gCM9g	0.132	0.138	This video seemed to escape the buffering, no problems on either
N89WrgJXep4	0.393	0.276	again no issues on either
L1v3ZmfV_hw	0.375	0.326
pegD78IpbhY	0.263	0.148
NXfEuZGf6Ls	0.720	0.248
qBWAMb_HlFw	0.378	0.043	biggest noticeable difference, normal freetube barely played anything

Overall this seems to be a youtube throttling issue where some of the links we are getting from note-ytdl-core come under youtubes throttling logic and yt-dlp links manage to somewhat avoid this - I don't know how they manage it.
The current freetube is consistently slower at loading videos than my update, so ill stick using this for now until you find a fix :)

Hope this information is helpful

My code change (not sure i can push branches) - this is just a proof of concept please dont hate my code too much ;P - src/renderer/store/modules/ytdl.js

const { exec} = require('child_process');

~~ SNIP to line 346 ~~

      exec("yt-dlp -J https://www.youtube-nocookie.com/embed/" + videoId, (error, stdout, stderr) => {
        if (error) {
            console.log(`error: ${error.message}`);
            return;
        }
        if (stderr) {
            console.log(`stderr: ${stderr}`);
            return;
        }
        var jsonParsed = JSON.parse(stdout);
        console.log(jsonParsed)

        ytdl.getInfo(videoId, {
          lang: 'en-US',
          requestOptions: { agent }
        }).then((result) => {
          result.player_response.streamingData.formats = []
          result.player_response.streamingData.adaptiveFormats = []
          for (var ytdlp_format of jsonParsed.formats){
            // setup the ytdlp format outputs to have similar values to the yt-dl formats

            if ( ! ytdlp_format.format.toString().includes("audio only")){
              ytdlp_format.mimeType = "video/" + ytdlp_format.video_ext + '; codecs=\"' + ytdlp_format.vcodec + '\"'
              if (ytdlp_format.vcodec != "none"){
                ytdlp_format.hasVideo = true
              }else{
                ytdlp_format.hasAudio = false
              }
              if (ytdlp_format.acodec != "none"){
                ytdlp_format.hasAudio = true
              }else{
                ytdlp_format.hasAudio = false
              }
              ytdlp_format.qualityLabel = ytdlp_format.format_note
              ytdlp_format.bitrate = ytdlp_format.filesize/5
              ytdlp_format.width = ytdlp_format.format.split(" ")[2].split("x")[0]
              ytdlp_format.itag = ytdlp_format.format_id
            }
            if (ytdlp_format.acodec != "none" && ytdlp_format.vcodec != "none" ){
              // only include legacy formats - since we cant get dash initRange anyway
              result.player_response.streamingData.formats.push(ytdlp_format)
            }
          }
          resolve(result)
        }).catch((err) => {
          reject(err)
        })
    });

[PrestonN]

We're well aware of the buffering issue at this point and as it's been mentioned the issue has to do specifically with how the local API currently behaves.

YouTube changed something about how videos need to be deciphered in order to load properly. Not using this new method is forcing the videos to have some extreme buffering. This behavior seems to happen more often when you're connected via a VPN, however there are reports of users experiencing it without a VPN and I imagine these reports will become more frequent as time goes on without a fix.

As was also mentioned, the module we use is an upstream one not maintained by us so we'll have to wait for a fix from them. I'll send them a message to see if we can get an update for it.

During this time, the easiest workaround is to switch over to the Invidious API or to simply enable Invidious video proxies if you don't want to use the full API. This issue is not present on videos returned by Invidious and should work fine while we wait for a fix to the local API.

Having said that, you'll likely notice an issue when attempting the above fix on v0.14.0. The video player does not parse the file returned by Invidious properly and forces the player to only play legacy formats. This means that videos are limited to a max of 720p. Assuming you're using the Invidious API and not the proxy, this will still work fine however I understand that it's less than ideal.

As of v0.15.0 (Along with the currently nightlies / RC builds) I've made a patch that fixes the issue where our video player handles Invidious files and FreeTube can properly play them at 1080p again. This is what I mentioned in the discussion thread linked above. It is not a patch for the local API. This is to help the Invidious API logic run smoother compared to how it works in v0.14.0.

For now, your options is to either download a nightly / RC or wait for v0.15.0 (ETA is this Friday). Once installed, update your settings to either use the Invidious API entirely, or enable video proxies. Once done, you shouldn't experience any throttling. Once a fix has been made for the local API, we will push the fix and create a new release.

[neurodiverseEsoteric]

Hey I was about to raise a similar issue, but I wasn't thinking it would be caused by a VPN, because the playback over the same invidious instance, but when opened in a browser, runs smoothly even while using a VPN. Whereas even if using an external player (mpv OR vlc), the hanging/buffering issue still occurs. So I thought it had an issue with caching and/or file access and/or system-level restrictions affecting FreeTube...

[pbwn]

i thought it was supposed to be fixed with the new update? im still having the same problem, if anything it's worse than before ie can't really use the app at all: ubuntu 18.04 + open vpn

[efb4f5ff-1298-471a-8973-3d47447115dc]

@p-m-brown #1835 (comment)

[arrowgent]

freetube 0.15.0

this is a youtube-dl issue
youtube is throttling, but thats not what is causing the issue.

fent/node-ytdl-core#964

https://github.com/ytdl-org/youtube-dl/issues?q=throttle

solution in PR waiting merge... (youtube-dl is unmaintained)
ytdl-org/youtube-dl#30184

preferable, maybe switch to yt-dlp. since it is currently active and doesnt have speed/throttle/bandwidth issue
https://github.com/yt-dlp/yt-dlp

although, there is no equivalent ytdlp js version?

as mentioned, "workaround" is to use invidious or proxy instances.

[peepo5]

@PrestonN You may want to check out - youtube-dl's activity, they have not updated since july

[arrowgent]

yes but this falls back to node-ytdl-core js, separate from youtube-dl's python2 version
similar issue however.

[efb4f5ff-1298-471a-8973-3d47447115dc]

We probably need to fork it like @PrestonN suggested in IRC.

[PrestonN]

Just to reiterate. We do not use youtube-dl or any fork of such. We use node-ytdl-core which mimics the functionality of youtube-dl except it's written in JavaScript and it's focused solely on YouTube. It has been an excellent module and attempting to replace it with youtube-dl or any of it's forks would not be beneficial / worth the effort.

This module that we use is what is required to be patched in order to fix the buffering issue. There is already a pull request available for the module that fixes the issue which you can find here. They are an independent team not associated with FreeTube therefore we must be patient with their efforts to review the pull request and allow them the time they need to solve any issues that they may have. Like FreeTube, the group that works on this module does so in their free time.

As hinted by @efb4f5ff-1298-471a-8973-3d47447115dc, we will likely use a fork of node-ytdl-core temporarily that includes this fix while we wait for the upstream project to merge over the pull request. We are also waiting for a patch from Electron that fixes a couple of issues. Once that is published, we will release a patch release with the Electron fixes as well as the buffering fixes.

Please be patient while we all wait for these to get released. We already have a workaround that you can do in the current release and we ask that you continue to do that until the patches are released. We're well aware that everyone is getting anxious waiting for the fix and asking us for updates is not going to make it happen any faster. We are keeping a close eye on these issues and are ready to push as update as soon as it's available. The only thing complaining does is take up our time and prompt responses such as this one. There's nothing that we can do in the meantime.

[peepo5]

woopie

[arrowgent]

i dont really see any "complaining here" we are also trying to figure out the cause and solution to slow download speeds.

some of us do know this change on youtube is effecting multiple other applications and people are looking for a fix

also maybe change the title to reflect that its not a VPN issue, that its related to node-ytdl-core and how it handles extraction

title:
"[ytdl] slow download & buffering caused by "n-transform" in youtube changes"

[efb4f5ff-1298-471a-8973-3d47447115dc]

Fixed in #1885

[neurodiverseEsoteric]

I fear this is back...The NewPipe team claims Google has recently changed stuff which broke their app as well...

[efb4f5ff-1298-471a-8973-3d47447115dc]

Yes they indeed have changed something. Fix will be available soon.

[maplepy]

Nice, I thought it was me having issues once again but glad I’m not the only one. By the way this happens even without the VPN

[maplepy]

Hey I'm not sure if it's only me but it seems that it's back (it's been a few days/weeks but my internet was acting up too). I can confirm that going on Youtube or on any invidious instance will properly load the video while it hangs on Freetube

[pbwn]

i recently gave up on freetube and uninstalled it because of this issue. Also, my whole computer would freeze occasionally while using freetube, and that's more than i can bear. I watch in the browser with invidious as you say

[maplepy]

i recently gave up on freetube and uninstalled it because of this issue. Also, my whole computer would freeze occasionally while using freetube, and that's more than i can bear. I watch in the browser with invidious as you say

Totally understand, I almost did too but I don't want to give google even more of my data and my privacy so I'm stuck with freetube. Don't get me wrong, when the app works it's incredible but these video loading problems are very annoying and almost make me wanna find an alternative

Do you have some way of keeping your subscriptions or do you manually check or maybe use an RSS reader?

[pbwn]

i created an account with invidious and moved my subscriptions there manually: yewtu.be , and i chose proxy videos in settings. I'm not sure how this compares with privacy offered by freetube. The other solution is to use eg yt-dlp to download and then watch the video in VLC. yt-dlp is very quick downloading, and if something is worth watching then it's worth downloading and watching in vlc i feel

[maplepy]

Sadly I have way too many subcriptions to import those manually and I'm used to watch videos at fast speeds (2× by default and often go up to 3.5×)

[maplepy]

Hey @efb4f5ff-1298-471a-8973-3d47447115dc not sure how to reopen the issue but wanted to let you know that the bug and some other bugs (that I reported) are back

[efb4f5ff-1298-471a-8973-3d47447115dc]

Hi @maplepy, we are aware of this and we are monitoring this closely internally because it is very inconsistent. if it becomes more consistent i'll reopen it!

[maplepy]

Okay great! Because I know some friends tried Freetube after I sold them the merits of this app but they removed it after a few days because of this issue. They are back to Youtube web app as of now unfortunately :/

I totally understand why it's driving them out of this app though, this is very annoying and the only way is to open an invidious or youtube instance by clicking the share button to actually play the video, I hope you guys find the culprit soon :) If you need help or some debug info, feel free to reach me here

[maplepy]

Is there an update on this? It is becoming very frustrating to have this bug coupled with the one who crashes the video when trying to change the quality making freetube embarrassing to use when trying to convert my friends to FOSS

[efb4f5ff-1298-471a-8973-3d47447115dc]

Most of the bugs are resolved in the latest nightly build including this one

[maplepy]

So I should get the nightly build now or is it introducing app breaking bugs?

[efb4f5ff-1298-471a-8973-3d47447115dc]

So I should get the nightly build now or is it introducing app breaking bugs?

I have been running the latest build and i havent seen any bugs.

[maplepy]

Neat! I'll use it then, thank you :)

[maplepy]

Most of the bugs are resolved in the latest nightly build including this one

Well doesn't seem like it's been resolved. Still buffering as much as it used to

Note: I downloaded this version v0.16.0-nightly-1608 Beta and it's not a problem with my connection I can watch the video fine on YouTube

[Draluy]

Having the same exact problem using v0.18.0-nightly-3011 Beta

[efb4f5ff-1298-471a-8973-3d47447115dc]

@Draluy see #3457 (comment)