The program crashes in half an hour or hour after starting a series of downloads images and video from search pages on Tumblr

[Taranchuk]

Hello Johannes, I hope you can solve this problem.
Since version 1.0.8.24 the program crashes in about half an hour or an hour usually if I add all the page searches to the queue and start the download. Version 1.0.8.23 is stable and crash does not happen, so I use this version mostly. I use the program only for downloading images and videos from the searches pages on Tumblr, I do not download regular blogs already, so I do not know if the crash is observed if download them. Also it seems to me that the note to release 1.0.8.24 can be related to this behavior of the program.

Somewhat "fixes" the timeout. Thus, if you have a wonky connection that frequently gets interrupted, TumblThree shouldn't stall anymore. The timeout value now counts for the whole connection time regardless of it's state. E.g. if you won't finish downloading a large file (video) within 120 seconds (default) increase the value or the file is truncated. If the release has any side effects since I've had to modify the core webrequest/downloader/crawler code for this, please try the v1.0.8.22 (#116).

Below here is the archive of my Index folder, so you can check if there is the crash on other machines (do not worry, the program should not download a lot of files, the entire date has already been downloaded and all file names are written in the index files).
Index.zip
Steps to reproduce the crash

1. run the program with my index folder
2. sort the "Url" column in alphabetical order, select all pages and add them to the queue, perhaps need to repeat several times
3. start the download
4. minimize the program, wait from half an hour to an hour approximately
5. crash occurs.

If this is not the case, and the crash is not observed, then this is likely my technical problems, then sorry for this. I just wrote this, because I want to update the program to version 1.0.8.31, but I can not because of the crash.

[Taranchuk]

I noticed that the speed of downloading from the GitHub is very slow. I uploaded to Mega for better speed https://mega.nz/#!bksC3IIQ!Fd47Tb6vhixKBqnKNJ1QIqJq3_NToR4w6GLfVYMk2Zk

[johanneszab]

Might be related to #164.

[johanneszab]

I've changed the default behavior of unobserved task to not terminate the application. In that commit you've quoted, I've wrapped all the connections into a Task in order to be able to terminate them if there is no response after the defined timeout. Since you might remember that the default timeout mechanism in the C# webrequest API doesn't work for async methods as I've found out in #116.

That means the application shouldn't crash anymore but all the timeouts get silently dropped. It requires some more code to notify the user in what method what exactly dropped. E.g. if during a download of a file, or crawling the blog/search or checking if a raw file exists the connection was terminated because it's over the timeout. Also there might be better options to handle such cases than just skipping over that connection.

[Taranchuk]

With the new release, the crash disappeared. Everything works fine. Thanks for the fix!