READY: Implemented endpoint to start a download from infohash #2338

devos50 · 2016-06-25T14:23:39Z

No description provided.

devos50 · 2016-06-26T10:00:51Z

(test seems to hang?)

devos50 · 2016-06-26T10:00:55Z

retest this please

whirm · 2016-06-27T12:50:06Z

retest this please

devos50 · 2016-06-28T08:49:08Z

@lfdversluis ready for your review

lfdversluis · 2016-06-28T09:31:04Z

Looks good to me.

devos50 · 2016-06-28T09:36:04Z

@lfdversluis I'm thinking about adding some parameters for anonymous downloading/seeding + destination on disk so I will work on that first (was a little to fast to ask you for review ;)

devos50 · 2016-06-29T17:19:30Z

retest this please

devos50 · 2016-06-29T17:30:39Z

See #2363

devos50 · 2016-06-29T17:30:42Z

retest this please

devos50 · 2016-06-29T17:47:41Z

Tests on mac timed out

devos50 · 2016-06-29T17:47:44Z

retest this please

synctext · 2016-06-30T06:16:09Z

retest this please

devos50 · 2016-06-30T13:51:53Z

retest this please

devos50 · 2016-07-01T07:23:45Z

The error encountered here is probably fixed by #2392

devos50 · 2016-07-03T16:23:25Z

Tribler/Core/Modules/restapi/downloads_endpoint.py

+ download_config = DownloadStartupConfig()
+
+ has_anon_download = 'anon_download' in parameters and len(parameters['anon_download']) > 0
+ has_safe_seeding = 'safe_seeding' in parameters and len(parameters['safe_seeding']) > 0


(Laurens): If now someone specifies (anon_download = false), then your code will think it's true. This is more a flag based approach than an argument based approach.

@whirm do you know the best way to make it a flag?

@devos50 I'll come to your office later to talk about this

devos50 · 2016-07-07T11:50:50Z

retest this please

devos50 · 2016-07-07T12:01:40Z

Job got aborted halfway?

devos50 · 2016-07-07T12:01:44Z

retest this please

devos50 · 2016-07-07T13:24:18Z

Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/tribler/Documents/workspace/workspace/GH_Tribler_PR_tests_mac/tribler/Tribler/Core/Utilities/twisted_thread.py", line 55, in _reactor_runner
    reactor.run(installSignalHandlers=False)
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/base.py", line 1194, in run
    self.mainLoop()
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/base.py", line 1203, in mainLoop
    self.runUntilCurrent()
--- <exception caught here> ---
  File "/usr/local/lib/python2.7/site-packages/twisted/internet/base.py", line 825, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "/Users/tribler/Documents/workspace/workspace/GH_Tribler_PR_tests_mac/tribler/Tribler/Core/Libtorrent/__init__.py", line 27, in <lambda>
    lambda_f = lambda a = args, kwa = kwargs: invoke_func(*a, **kwa)
  File "/Users/tribler/Documents/workspace/workspace/GH_Tribler_PR_tests_mac/tribler/Tribler/Core/Libtorrent/__init__.py", line 28, in invoke_func
    download.session.lm.threadpool.add_task(lambda_f, 1)
exceptions.AttributeError: 'NoneType' object has no attribute 'add_task'

devos50 · 2016-07-07T13:24:23Z

retest this please

brussee · 2016-07-07T14:56:17Z

retest this please

devos50 · 2016-07-08T07:34:51Z

test_ffsearch (Tribler.Test.test_remote_search.TestMixedRemoteSearch) ... Unhandled Error
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/Core/Utilities/twisted_thread.py", line 55, in _reactor_runner
    reactor.run(installSignalHandlers=False)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1194, in run
    self.mainLoop()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 1203, in mainLoop
    self.runUntilCurrent()
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/base.py", line 825, in runUntilCurrent
    call.func(*call.args, **call.kw)
  File "<string>", line 2, in _do_request

  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/Core/RemoteTorrentHandler.py", line 32, in pass_when_stopped
    return f(self, *argv, **kwargs)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/Core/RemoteTorrentHandler.py", line 455, in _do_request
    trackers = self._torrent_db_handler.getTrackerListByInfohash(infohash)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/Core/CacheDB/SqliteCacheDBHandler.py", line 696, in getTrackerListByInfohash
    torrent_id = self.getTorrentID(infohash)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/Core/CacheDB/SqliteCacheDBHandler.py", line 242, in getTorrentID
    return self.getTorrentIDS([infohash, ]).get(infohash)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/Core/CacheDB/SqliteCacheDBHandler.py", line 262, in getTorrentIDS
    torrents = self._db.fetchall(sql_stmt, to_select)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/dispersy/util.py", line 46, in helper
    return blockingCallFromThread(reactor, func, *args, **kargs)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/dispersy/util.py", line 225, in blockingCallFromThread
    return f(*args, **kwargs)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/Core/CacheDB/sqlitecachedb.py", line 401, in fetchall
    res = self.execute_read(sql, args)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/Core/CacheDB/sqlitecachedb.py", line 310, in execute_read
    return self.execute(sql, args)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/dispersy/util.py", line 46, in helper
    return blockingCallFromThread(reactor, func, *args, **kargs)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/dispersy/util.py", line 225, in blockingCallFromThread
    return f(*args, **kwargs)
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/Core/CacheDB/sqlitecachedb.py", line 263, in execute
    cur = self.get_cursor()
  File "/home/jenkins/workspace/GH_Tribler_PR_tests_linux@2/tribler/Tribler/Core/CacheDB/sqlitecachedb.py", line 196, in get_cursor
    self._cursor_table[thread_name] = self._connection.cursor()
exceptions.AttributeError: 'NoneType' object has no attribute 'cursor'

devos50 · 2016-07-08T07:34:55Z

retest this please

devos50 · 2016-07-08T08:25:16Z

This one is too unstable too be merged at the moment due to the thread pool stuff when checkpointing (this is probably since I'm starting downloads in the tests I've added).

whirm · 2016-07-08T09:12:39Z

I'm working on removing the threadpool altogether, but I'm finding so many bugs while doing it that I spend most of the time fixing them so I can go on.

brussee · 2016-07-08T15:15:34Z

retest this please

brussee · 2016-07-08T15:25:37Z

@whirm @devos50 Smooth sailing now! Ready to merge?

devos50 · 2016-07-10T10:25:35Z

@brussee I understand you need this endpoint but I'm afraid that if we merge this endpoint, the tests will become too unstable. Also, I first want a review from @whirm ;)

devos50 · 2016-07-10T10:26:02Z

@whirm can you review?

whirm · 2016-07-11T10:02:55Z

Tribler/Core/Modules/restapi/downloads_endpoint.py

+ .. sourcecode:: none
+
+ curl -X PUT http://localhost:8085/download/4344503b7e797ebf31582327a5baae35b11bda01
+ --data "anon_download=true&safe_seeding=true&destination=/my/dest/on/disk/"


We talked about using the hop number instead (0 hops being plain download)

@whirm ah I forgot to update the doc string here. Is that your only comment regarding this PR? :)

devos50 · 2016-07-12T13:35:11Z

retest this please

devos50 · 2016-07-12T14:48:54Z

success :)

devos50 · 2016-07-12T14:48:58Z

retest this please

devos50 · 2016-07-12T14:50:27Z

@whirm can you review this already? (see my last two commits)

whirm · 2016-07-12T14:58:04Z

Looks fine, let's see if the tests pass

whirm · 2016-07-12T14:59:09Z

Also, please send the fixes to next

Now, only the starting of the DHT is executed on the thread pool. The check itself is executed on the reactor thread.

devos50 · 2016-07-12T15:21:33Z

Still the same error on win64...

brussee · 2016-07-13T14:04:36Z

retest this please
edit: now the same error on win32

brussee · 2016-07-13T18:07:15Z

This PR is superseded by #2464

whirm · 2016-07-14T13:33:13Z

Shouldn't it be closed then?

brussee · 2016-07-14T15:04:46Z

@devos50 You can close this one since your awesome commits are merged with #2464. Nice teamwork!

devos50 force-pushed the start_download_infohash branch 2 times, most recently from 52cbd00 to f7130dd Compare June 26, 2016 10:43

devos50 force-pushed the start_download_infohash branch from f7130dd to e84803f Compare June 28, 2016 08:39

devos50 force-pushed the start_download_infohash branch 2 times, most recently from 88deea5 to a83c596 Compare June 29, 2016 17:06

devos50 force-pushed the start_download_infohash branch from a83c596 to b9488de Compare June 29, 2016 20:12

devos50 reviewed Jul 3, 2016
View reviewed changes

devos50 force-pushed the start_download_infohash branch from bffa981 to 0190c68 Compare July 7, 2016 10:48

devos50 changed the title ~~WIP: Implemented endpoint to start a download from infohash~~ READY: Implemented endpoint to start a download from infohash Jul 10, 2016

whirm reviewed Jul 11, 2016
View reviewed changes

devos50 force-pushed the start_download_infohash branch from 0190c68 to 3bcea68 Compare July 12, 2016 14:07

devos50 added 2 commits July 12, 2016 16:36

Implemented endpoint to start a download from infohash

5eea4d6

Added tests for starting download from infohash

58d74f3

devos50 force-pushed the start_download_infohash branch from 3bcea68 to bb5c165 Compare July 12, 2016 14:36

devos50 added 2 commits July 12, 2016 17:07

Refactored DHT check in LibtorrentMgr

050b26c

Now, only the starting of the DHT is executed on the thread pool. The check itself is executed on the reactor thread.

Fixed set_share_mode call

da834ca

devos50 force-pushed the start_download_infohash branch from bb5c165 to da834ca Compare July 12, 2016 15:07

devos50 closed this Jul 14, 2016

devos50 deleted the start_download_infohash branch July 14, 2016 15:52

READY: Implemented endpoint to start a download from infohash #2338

READY: Implemented endpoint to start a download from infohash #2338

Conversation

devos50 commented Jun 25, 2016

devos50 commented Jun 26, 2016

devos50 commented Jun 26, 2016

whirm commented Jun 27, 2016

devos50 commented Jun 28, 2016

lfdversluis commented Jun 28, 2016

devos50 commented Jun 28, 2016 • edited Loading

devos50 commented Jun 29, 2016

devos50 commented Jun 29, 2016

devos50 commented Jun 29, 2016

devos50 commented Jun 29, 2016

devos50 commented Jun 29, 2016

synctext commented Jun 30, 2016

devos50 commented Jun 30, 2016

devos50 commented Jul 1, 2016

devos50 Jul 3, 2016

Choose a reason for hiding this comment

devos50 Jul 3, 2016

Choose a reason for hiding this comment

whirm Jul 4, 2016

Choose a reason for hiding this comment

devos50 commented Jul 7, 2016

devos50 commented Jul 7, 2016

devos50 commented Jul 7, 2016

devos50 commented Jul 7, 2016

devos50 commented Jul 7, 2016

brussee commented Jul 7, 2016

devos50 commented Jul 8, 2016

devos50 commented Jul 8, 2016

devos50 commented Jul 8, 2016

whirm commented Jul 8, 2016

brussee commented Jul 8, 2016

brussee commented Jul 8, 2016

devos50 commented Jul 10, 2016

devos50 commented Jul 10, 2016

whirm Jul 11, 2016

Choose a reason for hiding this comment

devos50 Jul 11, 2016

Choose a reason for hiding this comment

devos50 commented Jul 12, 2016

devos50 commented Jul 12, 2016

devos50 commented Jul 12, 2016

devos50 commented Jul 12, 2016

whirm commented Jul 12, 2016

whirm commented Jul 12, 2016

devos50 commented Jul 12, 2016

brussee commented Jul 13, 2016 • edited Loading

brussee commented Jul 13, 2016

whirm commented Jul 14, 2016

brussee commented Jul 14, 2016

devos50 commented Jun 28, 2016 •

edited

Loading

brussee commented Jul 13, 2016 •

edited

Loading