Postprocessing for downloads and implement missing features #1759
Conversation
|
Aaaaa I need help reviewing code O.o |
|
@theScrabi requires lastest NewPipeExtractor and TeamNewPipe/NewPipeExtractor#118 |
There was a problem hiding this comment.
This is huge code. I see the advantage of it, but it looks like a massive amount of maintenance.
The thing is we wanted to get rid of gigaget, however if we create more and more functionality around it it will become almost impossible to do that.
From a functional point however this is a cool change :)
|
most of the code moved to TeamNewPipe/NewPipeExtractor#119 |
kapodamy
changed the title
Post-processing infrastructure * remove interfaces with one implementation * fix download resources with unknow length * marquee style for ProgressDrawable * "view details" option in mission context menu * notification for finished downloads * postprocessing infrastructure: sub-missions, circular file, layers for layers of abstractions for Java IO streams * Mp4 muxing (only DASH brand) * WebM muxing * Captions downloading * alert dialog for overwrite existing downloads finished or not. Misc changes * delete SQLiteDownloadDataSource.java * delete DownloadMissionSQLiteHelper.java * implement Localization from #114 Misc fixes (this branch) * restore old mission listeners variables. Prevents registered listeners get de-referenced on low-end devices * DownloadManagerService.checkForRunningMission() now return false if the mission has a error. * use Intent.FLAG_ACTIVITY_NEW_TASK when launching an activity from gigaget threads (apparently it is required in old versions of android) More changes * proper error handling "infrastructure" * queue instead of multiple downloads * move serialized pending downloads (.giga files) to app data * stop downloads when swicthing to mobile network (never works, see 2nd point) * save the thread count for next downloads * a lot of incoherences fixed * delete DownloadManagerTest.java (too many changes to keep this file updated)
Also this include: * Mp4 DASH reader/writter * WebM reader/writter * a subtitle converter for Timed Text Markup Language v1 and TranScript (v1, v2 and v3) * SharpStream to wrap IntputStream and OutputStream in one interface * custom implementation of DataInputStream
|
Please make the pullrequest compile. I tried compiling NewPipe against this commit, however there are still some other build issues. Pleas fix those before I can continue my review. |
|
@theScrabi |
* use getPreferredLocalization() instead of getLocalization() * use lastest commit in build.gradle * fix missing cast in MissionAdapter.java
|
It is great to see this PR compiling again :) I did a small test and I have to admit that I love your changes. Let's see what we can improve to make this PR even better. When a download is finished the notification informs the user about this. I think to make clear which app set up this notification, we should add NewPipe's logo to it. Additionally, we could enable line breaks whenever the title is too long. Using the light theme, the titles in the downloads list are hard to read. After queuing a video, it shows an estimated size of 2MB which is clearly incorrect. Is it possible to pass the file size we fetched when showing the download dialog? EDIT: Clicking on |
|
Finally, the PR so many of us have been waiting for! |
|
I started 2 downloads and tested the queuing behaviour (started and stopped the downloads twice). Everything was fine and the downloads continued. After a while the screen went off and I did not pay attention to the downloads anymore. After two or three minutes I realized the downloads stopped with following errors:
Unfortunately, I was not able to restart the download. EDIT: For some reason, NewPipe cleared the download history after closing the app and does not show these failed downloads anymore. Unfortunately, the download files still exist and cannot be deleted from within the app. The downloads view crashes when rotating the screen. Exception
Crash log
PS: please note my edit in the last comment :) |
|
|
@TobiGr about the errors |
* use bold style in status (mission_item_linear.xml) * fix download attemps not begin updated * dont stop the queue if a download fails * implement partial wake-lock & wifi-lock * show notifications for failed downloads * ¿proper bitmap dispose? (DownloadManagerService.java) * improve buffer filling (CircularFile.java) * [Mp4Dash] increment reserved space from 2MiB to 15MiB. This is expensive but useful for devices with low ram * [WebM] use 2MiB of reserved space * fix debug warning if one thread is used * fix wrong download speed when the activity is suspended * Fix "Queue" menu item that appears in post-processing errors * fix mission length dont being updated (missing commit)
|
@kapodamy It good to see that you were able to fix the crashes. Unfortunately, I produced some more :)
Exception
Crash log
Download bugsThere are issues with large downloads on Android 5. I experienced strange behaviours with streams >= 3 GB. As you can see, the download state is not displayed correctly and the current progress is not displayed either. Additionally, the download does not finish and at some point the download size (on the bottom left) grows every second. When opening the downloads view, for half a second or so, the current file site is displayed at the position of the current download speed in GB/s. I guess all these bugs are related to the incorrect state. I tested this with Android 5 and bad WiFi. Both are not optimal testing conditions, so in case you cannot reproduce this, I can try to create some more logs for you.
I'll take a look at your latest changes, which I didn't test, next week. |
|
* fix content length reading * use float overflow. Expensive, double is used instead * fix invalid cast after click the mission body * use a list for maximum attemps (downloads) * minor clean up (DownloadManager.java) * dont pass SharedPreferences instace to DownloadManager * use a switch instead of checkbox for cross_network_downloads * notify media scanner after deleting a finished download
* misc code clean-up * fix weird download speed, before switching the list view * fix CircularFile.java getting stuck on post-processing huge files >2GiB * keep crashed post-processing downloads visible to the user
* fast download pausing * fix UI thread blocking when calling pause() * check running threads before start the download * fix null pointer exception in onDestroy in the download service, without calling onCreate method (android 8)
|
@theScrabi |
|
|
|
The only issue is that the icons in the download manager do not adapt bright design. Pleas fix this in an additional commit.
|
|
is it able to download audio in mp3 ? |
|
@FaizKhan5 if you mean convert m4a to mp3 no. |
|
@kapodamy yes, i mean that. Is there any chance to implement that in near future ? |
|
Thanks for the update |
This PR include:
.gigafiles) to app dataAlso this include:
More details:
Notes:
DonwloadDialog, video-only streams the character~will be shown in front of the size, in the future it should be shown to add the size of the audio stream