-
-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
No playback of certain videos, infinite loading for HLS/DASH streams #2415
Comments
Why are there both "&sig=" and "&lsig="? I think the second one makes no sense, but I could be wrong. I'll investigate |
Which version are u using? |
0.16.2 |
It doesn't work for me either, but only on the WebM format. Here is the url I got from the extractor (I replaced my IP with X):
Decoded:
|
Appears jdf76/plugin.video.youtube#624 fixes this by adding I'd recommend changing DECYRYPTION_SIGNATURE_FUNCTION_REGEX to what Invidious uses ( |
I can't convert that regex to the java dialect... this is the regex I tried: "/^([^=]+)=function\\(a\\)\\{a=a\\.split\\(\"\"\\)/m" I removed the |
Try removing the leading and trailing slashes ( |
cannot resolve with "/^([^=]+)=function(a){a=a.split("")/m and |
@TobiGr this calls for an emergency release shall i take care of release once it it fixed or will you? |
By using only the first part of the regex Aq=function(a){a=a.split("&");for(var b={},c=0,d=a.length;c<d;c++){var e=a[c].split("=");if(1==e.length&&e[0]||2==e.length)try{var f=Xc(e[0]||""),k=Xc(e[1]||"");f in b?g.Ka(b[f])?ib(b[f],k):b[f]=[b[f],k]:b[f]=k}catch(m){var l=Error("Error decoding URL component");l.params="key: "+e[0]+", value: "+e[1];"q"==e[0]?g.Vp(l):g.L(l)}}return b}; Dq=function(a){a=a.split(",");return a=a.map(function(b){return g.Cq(b)})}; But both of them have something between the " ", so it can't be matched by `^([^=]+)=function(a){a=a.split(""). Using the old regex truncated before the " " I get the same result. If I, instead, use this part of the regex jaa=function(a,b){for(var c=g.Da(a)?a.split(""):a,d=a.length-1;0<=d;--d)d in c&&b.call(void 0,c[d],d,a)}; g.Wa=function(a,b,c){for(var d=a.length,e=g.Da(a)?a.split(""):a,f=0;f<d;f++)if(f in e&&b.call(c,e[f],f,a))return f;return-1}; g.Ya=function(a,b,c){for(var d=g.Da(a)?a.split(""):a,e=a.length-1;0<=e;e--)if(e in d&&b.call(c,d[e],e,a))return e;return-1}; xi=function(a,b,c,d,e){if(null==a)return"";b=b||"&";c=c||",$";"string"==typeof c&&(c=c.split(""));if(a instanceof Array){if(d=d||0,d<c.length){for(var f=[],k=0;k<a.length;k++)f.push(xi(a[k],b,c,d+1,e));return f.join(c[d])}}else if("object"==typeof a)return e=e||0,2>e?encodeURIComponent(yi(a,b,c,d,e+1)):"...";return encodeURIComponent(String(a))}; I also tried this regex but it only works if I use the part before " ", as above. |
@theScrabi That's a good question. I currently have little time due to upcoming exams. So it would be better if you take care of it. Nevertheless, we need a fix before we release something :D |
I'll take care of it. @Stypox has a fix already been send to the extractor? |
No, I'm sorry, but I wasn't able to get the regex to work (see comment above). Maybe @omarroth can help? :-) |
Would you mind testing with omarroth/NewPipeExtractor@098b835 to see if it works as expected? Currently it appears NewPipe has no trouble with the existing regex, so please do provide the saved |
omarroth/NewPipeExtractor@098b835 seems to have the same problems (Error 404)... |
I just tried downloading on invidious the same video and the only video format that can be downloaded is 360pMPEG4 (the same as NewPipe) so I don't think that new regex will fix this problem |
Taking another look, the video in question is likely a livestream that was uploaded later as a normal video. Playback appears to work as expected, download I expect doesn't work since the file itself is split up into ~90 segments (from looking at the DASH manifest). |
I am also having this problem, almost any music-related video stays forever in the spinner for me... (Note: background play doesn't work too.) Example of video that plays: https://www.youtube.com/watch?v=kpk2tdsPh0A |
I've been dealing with this problem for quite some time now. Nearly all music-related videos stay spinning, but no non-music videos do. Any updates? |
youtube-dl has this issue, too.
|
🤔 The 8 urls that do not work in the browser are DASH-videos, and they download fine with youtube-dl. By looking through the dash manifests I found an url that works in the browser. Is the NewPipeExtractor downloading dash manifests and parsing them? |
After some hours of reading code I understand what's wrong. The extractor correctly extracts the urls, but those links lead to DASH playbacks. I will call those DASH urls "base url"s. By adding
There is some junk (probably UTF7-base64 is not the correct encoding), but some useful DASH information can be extracted anyway: The base url works even after replacing all params separators (i.e. '&', '?', '=') with '/'. In this case the initialization url would be <?xml version="1.0" encoding="UTF-8"?>
<MPD xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="urn:mpeg:DASH:schema:MPD:2011" xmlns:yt="http://youtube.com/yt/2012/10/10" xsi:schemaLocation="urn:mpeg:DASH:schema:MPD:2011 DASH-MPD.xsd" minBufferTime="PT1.500S" profiles="urn:mpeg:dash:profile:isoff-main:2011" type="static" mediaPresentationDuration="PT291.131S">
<Period>
<AdaptationSet id="2" mimeType="video/mp4" subsegmentAlignment="true">
<Representation id="135" codecs="avc1.4d4014" width="854" height="480" startWithSAP="1" maxPlayoutRate="1" bandwidth="1155000" frameRate="30">
<BaseURL>DECODED_AND_ESCAPED_BASE_URL</BaseURL>
<SegmentList>
<Initialization sourceURL="sq/0"/>
<SegmentURL media="sq/1"/>
<SegmentURL media="sq/2"/>
<SegmentURL media="sq/3"/>
<SegmentURL media="sq/4"/>
<SegmentURL media="sq/5"/>
<SegmentURL media="sq/6"/>
<SegmentURL media="sq/7"/>
<SegmentURL media="sq/8"/>
<SegmentURL media="sq/9"/>
<SegmentURL media="sq/10"/>
<SegmentURL media="sq/11"/>
<SegmentURL media="sq/12"/>
<SegmentURL media="sq/13"/>
<SegmentURL media="sq/14"/>
<SegmentURL media="sq/15"/>
<SegmentURL media="sq/16"/>
<SegmentURL media="sq/17"/>
<SegmentURL media="sq/18"/>
<SegmentURL media="sq/19"/>
<SegmentURL media="sq/20"/>
<SegmentURL media="sq/21"/>
<SegmentURL media="sq/22"/>
<SegmentURL media="sq/23"/>
<SegmentURL media="sq/24"/>
<SegmentURL media="sq/25"/>
<SegmentURL media="sq/26"/>
<SegmentURL media="sq/27"/>
<SegmentURL media="sq/28"/>
<SegmentURL media="sq/29"/>
<SegmentURL media="sq/30"/>
<SegmentURL media="sq/31"/>
<SegmentURL media="sq/32"/>
<SegmentURL media="sq/33"/>
<SegmentURL media="sq/34"/>
<SegmentURL media="sq/35"/>
<SegmentURL media="sq/36"/>
<SegmentURL media="sq/37"/>
<SegmentURL media="sq/38"/>
<SegmentURL media="sq/39"/>
<SegmentURL media="sq/40"/>
<SegmentURL media="sq/41"/>
<SegmentURL media="sq/42"/>
<SegmentURL media="sq/43"/>
<SegmentURL media="sq/44"/>
<SegmentURL media="sq/45"/>
<SegmentURL media="sq/46"/>
<SegmentURL media="sq/47"/>
<SegmentURL media="sq/48"/>
<SegmentURL media="sq/49"/>
<SegmentURL media="sq/50"/>
<SegmentURL media="sq/51"/>
<SegmentURL media="sq/52"/>
<SegmentURL media="sq/53"/>
<SegmentURL media="sq/54"/>
<SegmentURL media="sq/55"/>
</SegmentList>
</Representation>
</AdaptationSet>
</Period>
</MPD> DECODED_AND_ESCAPED_BASE_URL is created by doing this (in pseudocode): baseUrl.decodeUrl().replace('&', '/').replace('?', '/').replace('=', '/').append('/') Note: youtube-dl does something different with DASH files. It downloads the MPD file provided in ytPlayerConfig["args"]["player_response"]["streamingData"]["dashManifestUrl"], that contains all of the supported formats (even non-DASH ones, it seems like). That file is ready to be played by a DASH player: vlc opened it just fine. Then ytdl extracts the formats along with their segment data: every extracted DASH stream has:
When ytdl wants to download a DASH file it just downloads all the fragments by concatenating the base url with the url of the segment (it just ignores initialization). |
@TobiGr @theScrabi @omarroth how should I proceed? But I don't think this can be implemented with the current NewPipeExtractor infrastructure, since a Stream can only provide an url. An option would be to add the function Another option would be to treat DASH files as a completely different stream type and create a new Another option still would be to ignore DASH streams. Very few videos have this problem with DASH video streams, and as far as I know there is always an alternative to that (e.g. the video from @obfripper has the 360p format that works) |
I wish Newpipe would just use yt-dl and let us update it ourselves instead of having to maintain their own extractor |
I appreciate the team using their own extractor, it also gives room to improve on it. |
@Pentaphon I agree, but I'm not entirely sure if there's a specific reason why they choose not to do that, because it seems like it'd be the easiest. |
Or everybody with extraction skills can just focus on improving youtube-dl while people with other skills work on Newpipe as a yt-dl frontend only. Every time Google breaks Newpipe, yt-dl fixes it first and then we have to wait for Newpipe to basically adapt the fix to the extractor. It makes way more sense to just download the latest yt-dl to Newpipe instead of rolling out a new version of Newpipe every time something breaks. I've raised the topic here: #4202 Hopefully, enough people can agree that we are just creating more work and the project can turn to that direction. |
mpv-android is adding youtube-dl support. You can use that as an external player now. |
@opusforlife2 It would be nice however not needing to use an external player. |
Wouldn't it be hard to implement yt-dl for Android since they are written in python? |
Correction: it's actually been available since 2017 or so. The apk is available in the PR. It's just that the NanoDroid version of mpv-android was updated to include youtube-dl, which is where I learned of it. |
In v0.20.0, we still won't be able to play those streams, but the non-playing streams won't be shown in the app. |
Which means you won't have to get irritated, then try the 1080p stream, see if it works or not, then try 360p. |
I haven't experienced this issue in quite a while eventhough I regularly watch smaller channels (views in the tens or hundreds) and this used to happen very often. The videos in this thread that are still available and some other ones I remember not working play just fine now. Is anyone still able to reproduce? I've noticed that the streams are now the 1080p h264 versions eventhough vp9 ones are available according to ytdl but, while not optimal, it's a rather minor issue compared to not playing at all. |
@Atemu this issue was fixed in 0.20.0, but only as a workaround, i.e. detecting the failing streams and hiding them. A real fix, allowing to play dash streams, needs to be implemented. |
I keep saying to close this and open a new issue for implementing DASH playback. <(`^´)> |
Or just edit the title to say "Implement DASH playback" |
Just a small update that TeamNewPipe/NewPipeExtractor#604 will make further qualities available in the app:
(Try by yourself using the APK linked in this PR (note that there are some bugs that prevent it to be merged as it is).) |
Can you please test the APK in #6537 and check if you have all the qualities available on YouTube in MP4 and WEBM formats, even those that are not currently available in the current version of NewPipe? Thank you in advance! |
@TiA4f8R It works! The WebM and MP4 streams both load fine. The only downside is that they take much more time to load than progressive streams. Also, all video items in lists are marked LIVE for some reason. |
@TiA4f8R Works good!
@opusforlife2 That's my fault. See TeamNewPipe/NewPipeExtractor#658 |
I am having issues playing certain videos, they all exhibit the same problem by sitting with the spinning wheel and nothing happening. If i select an external player, there is a pause followed by an error notification, which I have screencapped and says 'the location.......full screen url....cannot be played'. The fault occurs only on some resolutions, and it has been observed that in most cases the qualities 720p or 360p always work (e.g. https://youtu.be/zCGouRiflIw).
For example videos;
https://www.youtube.com/watch?v=2NT-MRkYeSo
All the recent videos on his channel exhibit the same issue.
https://www.youtube.com/watch?v=ZHeqOwV5bow
Some of the video on this channel play, some do not.
The text was updated successfully, but these errors were encountered: