Add docs regaring --to-remote option for add/import-url #2091
Conversation
This comment has been minimized.
This comment has been minimized.
isidentical
force-pushed
the
straight-to-remote
branch
from
5cabdfc to
49f2f07
Compare
| @@ -1,8 +1,9 @@ | |||
| # import-url | |||
|
|
|||
| Download a file or directory from a supported URL (for example `s3://`, | |||
| `ssh://`, and other protocols) into the <abbr>workspace</abbr>, and track it (an | |||
| import `.dvc` file is created). | |||
| `ssh://`, and other protocols) into the <abbr>workspace</abbr> (or to the | |||
There was a problem hiding this comment.
I think we need to work on this a bit more ... probably get rid of Download?
brackets look too complicated for the very beginning of the doc
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
There was a problem hiding this comment.
But OK, if we completely rewrite this anyway, we can address the parenthesis situation along the way.
- Maybe something like
Track a file or directory found in an external location (`s3://`, `/local/path`, etc.), and download it to the local project, or make a copy in [remote storage](/doc/command-reference/remote).
There was a problem hiding this comment.
Please notice similar suggestion (for add --to-remote) in iterative/dvc#5198 (review) @isidentical:
Track an external target, but don't move it into the workspace, nor cache it. Transfer it to remote storage instead.
There was a problem hiding this comment.
Left a detailed review on add for now (some can be applied to import-url too, probably). Getting there, but this change is tricky and may result larger than it seemed at first.
I left specific, mergeable suggestions to make it easier but at some point we can take this over (after merged in core and most discussions are resolved, hopefully). Thanks @isidentical
| usage: dvc get-url [-h] [-q | -v] [-j <number>] url [out] | ||
| usage: dvc get-url [-h] [-q | -v] url [out] |
There was a problem hiding this comment.
Oops @isidentical I'm seeing lots of -j related changes here. Maybe this got contaminated from another one of your docs branches?
There was a problem hiding this comment.
This file (content/docs/command-reference/get-url.md), content/docs/command-reference/get.md, and content/docs/command-reference/import.md to be precise.
There was a problem hiding this comment.
Thanks @isidentical, I've finished dogin copy edits to the import-url example. One last thing pending (for consistency), suggested by Ivan earlier I believe:
| When you don't want to store the target data in your local system, you can still | ||
| create an import `.dvc` file while transferring a file or directory directly to | ||
| remote storage, by using the `--to-remote` option. See the | ||
| [Import straight to remote](#example-transfer-to-remote-storage) example for | ||
| more details. |
There was a problem hiding this comment.
Can we put something like this in the add reference Description (instead of the Transferring data directly to remote storage section) ?
| ## Example: Transfer to remote storage | ||
|
|
||
| When you have a large dataset in an external location, you may want to import it | ||
| to you project without downloading it to the local file system (for using it | ||
| later/elsewhere). The `--to-remote` option lets you skip the download, while |
There was a problem hiding this comment.
And copy over the Example as well (will need some some adapting). Thanks
|
@jorgeorpinel let's merge it and create a ticket to address the issue left. Probably simplify description, use an example instead, move info organically into Description - e.g. as a hint. |
|
Sounds good. Thanks again @isidentical |
|
Oops wait. We merged the |
|
BTW extracted the pending #2091 (review) to #2161. |
Bound to iterative/dvc#5198