Add torrent downloading to CLI #28
Conversation
zod/cli/torrent_download.py
Outdated
| ), | ||
| ): | ||
| # we have to make sure aria2c is installed | ||
| check_aria_install() |
There was a problem hiding this comment.
would be nice if this check considered the dry_run flag. So that I could see what command would run even without having aria installed :)
There was a problem hiding this comment.
Good suggestion. Added!
zod/cli/torrent_download.py
Outdated
|
|
||
|
|
||
| @app.command() | ||
| def download( |
There was a problem hiding this comment.
almost everything here is shared with the dropbox download script. Can we have a single, shared, entrypoint to simplify maintenance?
something like:
zod download --torrent (with interactive question if not specified)
or hierarchical like
zod download torrent vs zod download dropbox (but maybe messy to share arguments then?)
There was a problem hiding this comment.
I made it to be an argument in the download script, acc. your first suggestion. Now we have an entrypoint that chooses what downloading script to run based on user choice.
| subprocess.check_call(["aria2c", "--version"]) | ||
| except FileNotFoundError: | ||
| print( | ||
| "aria2c is not installed. Please install aria2c to download the dataset. Using: `apt install aria2`" # noqa |
There was a problem hiding this comment.
would be nice to provide the user with a browser link to academic torrents, and a message stating that you don't have to install aria2 and can instead download and extract the dataset yourself with your chosen torrent client.
Bonus points if we tell the user which files to download depending on the provided flags :)
There was a problem hiding this comment.
Fixed. Dont think adding the file indices is useful.
There was a problem hiding this comment.
we have some hardcoded things like torrent paths and file sizes here, that could potentially become outdated in the future. How about we just print a generic message if something goes wrong, along the lines of:
'this and this went wrong. please make sure to update to the latest cli version. if this doesn't fix it, please submit a github issue. ciao bye'
There was a problem hiding this comment.
Added this where it applies.
|
|
||
| from zod.cli.dropbox_download import DownloadSettings, FilterSettings | ||
| from zod.cli.dropbox_download import download_dataset as drobbox_download |
There was a problem hiding this comment.
| from zod.cli.dropbox_download import download_dataset as drobbox_download | |
| from zod.cli.dropbox_download import download_dataset as dropbox_download |
| _download_dataset(download_settings, filter_settings, subset.folder) | ||
|
|
||
| if source == DownloadSource.DROPBOX: | ||
| drobbox_download(download_settings, filter_settings, subset.folder) |
There was a problem hiding this comment.
| drobbox_download(download_settings, filter_settings, subset.folder) | |
| dropbox_download(download_settings, filter_settings, subset.folder) |
No description provided.