-
Notifications
You must be signed in to change notification settings - Fork 405
misc. regular updates #1801
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
misc. regular updates #1801
Changes from all commits
73e9109
6fc5e8b
bd5e462
1c82664
c75a683
ef42cf8
960b98f
e4c2c87
79d1ae4
e635905
9b4ea34
1e75862
f150ccc
0dc8d22
f23fc5c
d16cd33
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -247,7 +247,6 @@ M model.pkl | |
| M data/features/ | ||
|
|
||
| $ dvc status | ||
|
|
||
| Data and pipelines are up to date. | ||
| ``` | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -146,27 +146,39 @@ $ dvc run -n download_file \ | |
|
|
||
| </details> | ||
|
|
||
| ## Example: DVC remote aliases | ||
| ## Example: Using DVC remote aliases | ||
|
|
||
| If instead of a URL you'd like to use an alias that can be managed | ||
| independently, or if the external dependency location requires access | ||
| credentials, you may use `dvc remote add` to define this location as a DVC | ||
| Remote, and then use a special URL with format `remote://{remote_name}/{path}` | ||
| to define an external dependency. | ||
| You may want to encapsulate external locations as configurable entities that can | ||
| be managed independently. This is useful if multiple dependencies (or stages) | ||
| reuse the same location, or if its likely to change in the future. And if the | ||
| location requires authentication, you need a way to configure it in order to | ||
| connect. | ||
|
|
||
| For example, for an HTTPs remote/dependency: | ||
| [DVC remotes](/doc/command-reference/remote) can do just this. You may use | ||
| `dvc remote add` to define them, and then use a special URL with format | ||
| `remote://{remote_name}/{path}` (remote alias) to define the external | ||
| dependency. | ||
|
|
||
| Let's see an example using SSH. First, register and configure the remote: | ||
|
|
||
| ```dvc | ||
| $ dvc remote add myssh ssh://myserver.com | ||
| $ dvc remote modify --local myssh user myuser | ||
| $ dvc remote modify --local myssh password mypassword | ||
| ``` | ||
|
|
||
| > Please refer to `dvc remote add` for more details like setting up access | ||
| > credentials for the different remote types. | ||
|
|
||
| Now, use an alias to this remote when defining the stage: | ||
|
|
||
| ```dvc | ||
|
Comment on lines
+173
to
175
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I also realized this doc only uses But then again you can't easily write an external dependency in dvc.yaml ... Hmmm 🤔 |
||
| $ dvc remote add example https://example.com | ||
| $ dvc run -n download_file \ | ||
| -d remote://example/data.txt \ | ||
| -d remote://myssh/path/to/data.txt \ | ||
| -o data.txt \ | ||
| wget https://example.com/data.txt -O data.txt | ||
| ``` | ||
|
|
||
| Please refer to `dvc remote add` for more details like setting up access | ||
| credentials for the different remotes. | ||
|
|
||
| ## Example: `import-url` command | ||
|
|
||
| In the previous examples, special downloading tools were used: `scp`, | ||
|
|
@@ -205,11 +217,11 @@ determine whether the source has changed and we need to download the file again. | |
|
|
||
| </details> | ||
|
|
||
| ## Example: Using import | ||
| ## Example: Imports | ||
|
|
||
| `dvc import` can download a <abbr>data artifact</abbr> from any <abbr>DVC | ||
| project</abbr> or Git repository. It also creates an external dependency in its | ||
| import `.dvc` file. | ||
| project</abbr>, or any file from a Git repository. It also creates an external | ||
| dependency in its import `.dvc` file. | ||
|
|
||
| ```dvc | ||
| $ dvc import git@github.com:iterative/example-get-started model.pkl | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This makes me realize our SSH examples for
remote add/modifymay be misleading (in other docs) because we sometimes usessh://user@example.com...when adding, and then mention about modifying the credentials or even show an example ofremote modify example user ...but I don't know if that will work correctly with DVC. Need to check...There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried testing this but can't because of treeverse/dvc#4712.