Skip to content

Commit

Permalink
Merge pull request #522 from jorgeorpinel/master
Browse files Browse the repository at this point in the history
docs: regular update (early August)
  • Loading branch information
shcheklein authored Aug 10, 2019
2 parents 6147596 + 6d1f6c1 commit f9b9e33
Show file tree
Hide file tree
Showing 38 changed files with 422 additions and 446 deletions.
66 changes: 36 additions & 30 deletions src/Documentation/glossary.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,54 +5,60 @@ export default {
'relevant DVC concepts and terminologies which are frequently used.',
contents: [
{
name: 'Workspace directory',
name: 'Workspace',
match: ['workspace'],
desc:
'The **workspace** contains all of your DVC **project** files and ' +
"directories. It's typically also a Git **repository**. See also " +
'[`dvc init`](/doc/commands-reference/init).'
desc: `
By "workspace" we refer to the directory containing all your project files. For
example raw datasets, source code, ML models, etc. A workspace becomes a DVC
project when [\`dvc init\`](/doc/commands-reference/init) is run, and
[DVC-files](/doc/user-guide/dvc-file-format) are created in it. It\s typically
also a Git repository.
`
},
{
name: 'DVC cache',
name: 'DVC Cache',
match: ['DVC cache', 'cache', 'cache directory'],
desc:
'The DVC cache is a hidden storage (by default located in the ' +
'`.dvc/cache` directory) for files that are under DVC control, and ' +
'their different versions. For more details, please refer to this ' +
'[document](/doc/user-guide/dvc-files-and-directories' +
'#structure-of-cache-directory).'
desc: `
The DVC cache is a hidden storage (by default located in the \`.dvc/cache\`
directory) for files that are under DVC control, and their different versions.
For more details, please refer to this
[document](/doc/user-guide/dvc-files-and-directories#structure-of-cache-directory).
`
},
{
name: 'Data artifact',
name: 'Data Artifact',
match: ['data artifact', 'data artifacts'],
desc:
'Any **data** file or directory, as well as intermediate or final ' +
'result (such as extracted features or a ML model file) that is ' +
'under DVC control. Refer to [Data and Model Files Versioning]' +
'(/doc/use-cases/data-and-model-files-versioning) for more details.'
desc: `
Any data file or directory, as well as intermediate or final result (such as
extracted features or a ML model file) that is under DVC control. Refer to
[Data and Model Files Versioning](/doc/use-cases/data-and-model-files-versioning)
for more details.
`
},
{
name: 'Import stage',
name: 'Import Stage',
match: ['import stage', 'import stages'],
desc:
'Stage (DVC-file) created with the `dvc import` or `dvc import-url` ' +
'commands. They represent files or directories from external sources.'
desc: `
Stage (DVC-file) created with the \`dvc import\` or \`dvc import-url\`
commands. They represent files or directories from external sources.
`
},
{
name: 'Output',
match: ['output', 'outputs'],
desc:
'A file or a directory that is under DVC control. See `dvc add`, ' +
'`dvc run`, `dvc import`, `dvc import-url` commands.'
desc: `
A file or a directory that is under DVC control. See \`dvc add\` \`dvc run\`,
\`dvc import\`, \`dvc import-url\` commands.
`
},
{
name: 'External Dependency',
match: ['external dependency', 'external dependencies'],
desc:
'A DVC-file dependency with origin in an external source, for ' +
'example HTTP, SSH, Amazon S3, Google Cloud Storage remote ' +
'locations, or even other DVC repositories. See ' +
'[External Dependencies](/doc/user-guide/external-dependencies).'
desc: `
A DVC-file dependency with origin in an external source, for example HTTP, SSH,
Amazon S3, Google Cloud Storage remote locations, or even other DVC repositories.
See [External Dependencies](/doc/user-guide/external-dependencies).
`
}
]
}
4 changes: 2 additions & 2 deletions static/docs/commands-reference/add.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ models). This way you bring data provenance and make your project reproducible.
location, by including a path in the provided value (e.g.
`-f stages/stage.dvc`).

## Examples: Single file
## Example: Single file

Take a file under DVC control:

Expand Down Expand Up @@ -158,7 +158,7 @@ $ file .dvc/cache/d8/acabbfd4ee51c95da5d7628c7ef74b
.dvc/cache/d8/acabbfd4ee51c95da5d7628c7ef74b: ASCII text
```

## Examples: Directory
## Example: Directory

What if you have not one dog picture, but hundreds of pictures of dogs and cats?
Your goal might be to build an algorithm to identify dogs and cats in pictures,
Expand Down
13 changes: 9 additions & 4 deletions static/docs/commands-reference/cache/dir.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# cache dir

Set/unset the cache directory location intuitively (compared to using
`dvc config cache`).
Set/unset the <abbr>cache directory</abbr> location intuitively (compared to
using `dvc config cache`).

## Synopsis

Expand Down Expand Up @@ -40,7 +40,12 @@ file location**. They are required in the latter form for the config file.

- `-h`, `--help` - prints the usage/help message, and exit.

## Examples: Using relative path
- `-q`, `--quiet` - do not write anything to standard output. Exit with 0 if no
problems arise, otherwise 1.

- `-v`, `--verbose` - displays detailed tracing information.

## Example: Using relative path

```dvc
$ dvc cache dir ../dir
Expand All @@ -54,7 +59,7 @@ $ cat .dvc/config
`../dir` has been resolved relative to the `.dvc/` dir, resulting in
`../../dir`.

## Examples: Using absolute path
## Example: Using absolute path

```dvc
$ dvc cache dir /path/to/dir
Expand Down
14 changes: 1 addition & 13 deletions static/docs/commands-reference/cache/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# cache

Contains a helper command to set the cache directory location:
Contains a helper command to set the <abbr>cache directory</abbr> location:
[dir](/doc/commands-reference/cache/dir).

## Synopsis
Expand Down Expand Up @@ -34,15 +34,3 @@ config option, for more information on file links on different platforms.)
problems arise, otherwise 1.

- `-v`, `--verbose` - displays detailed tracing information.

## Example

The main use of this command is to apply the `-v` and `-q` options to
`dvc cache dir` which doesn't have them:

```dvc
$ dvc cache --verbose dir mycache
DEBUG: Writing '/Users/user/myproject/.dvc/config'.
$ dvc config cache.dir
../mycache
```
28 changes: 6 additions & 22 deletions static/docs/commands-reference/checkout.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,36 +103,21 @@ be pulled from a remote cache using `dvc pull`.

## Examples

To explore `dvc checkout` let's consider a simple workspace with several
[stages](/doc/commands-reference/run), and a few Git tags. Then with
`git checkout` and `dvc checkout` we can see what happens as we shift from tag
to tag.
Let's employ a simple workspace with some data, code, ML models, pipeline
stages, as well as a few Git tags, such as our
[get started example repo](https://github.com/iterative/example-get-started).
Then we can see what happens with `git checkout` and `dvc checkout` as we switch
from tag to tag.

<details>

### Click and expand to setup the project

This step is optional, and you can run it only if you want to run this examples
in your environment. First, you need to download the project:
Start by cloning our sample repo if you don't already have it:

```dvc
$ git clone https://github.com/iterative/example-get-started
```

Second, let's install the requirements. But before we do that, we **strongly**
recommend creating a virtual environment with
[virtualenv](https://virtualenv.pypa.io/en/stable/) or a similar tool:

```dvc
$ cd example-get-started
$ virtualenv -p python3 .env
$ source .env/bin/activate
```

Now, we can install requirements for the project:

```dvc
$ pip install -r requirements.txt
```

</details>
Expand All @@ -157,7 +142,6 @@ solving the problem:

```dvc
$ git tag
baseline <- first simple version of the model
bigram <- use bigrams to improve the model
```
Expand Down
26 changes: 11 additions & 15 deletions static/docs/commands-reference/commit.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,37 +90,33 @@ into play. It handles that last step of adding the file to the DVC cache.

## Examples

To explore `dvc commit` let's consider a simple pipeline with several stages:
the example workspace used in the [Get Started](/doc/get-started) section.
Let's employ a simple workspace with some data, code, ML models, pipeline
stages, such as the DVC project created in our [Get Started](/doc/get-started)
section. Then we can see what happens with `git commit` and `dvc commit` in
different situations.

<details>

### Click and expand to setup the project

This step is optional, and you can run it only if you want to run this examples
in your environment. First, you need to download the project:
Start by cloning our sample repo if you don't already have it:

```dvc
$ git clone https://github.com/iterative/example-get-started
$ cd example-get-started
```

Second, let's install the requirements. But before we do that, we **strongly**
recommend creating a virtual environment with
[virtualenv](https://virtualenv.pypa.io/en/stable/) or a similar tool:
Now let's install the requirements. But before we do that, we **strongly**
recommend creating a virtual environment with a tool such as
[virtualenv](https://virtualenv.pypa.io/en/stable/):

```dvc
$ cd example-get-started
$ virtualenv -p python3 .env
$ source .env/bin/activate
```

Now, we can install requirements for the project:

```dvc
$ pip install -r requirements.txt
```

Then download the precomputed data using:
Download the precomputed data using:

```dvc
$ dvc pull --all-branches --all-tags
Expand Down Expand Up @@ -221,7 +217,7 @@ $ ls .dvc/cache/70
And we've verified that `dvc commit` has saved the changes into the cache, and
that the new instance of `model.pkl` is in the cache.

## Example: Running without DVC
## Example: Running commands without DVC

It is also possible to execute the commands that are executed by `dvc repro` by
hand. You won't have DVC helping you, but you have the freedom to run any script
Expand Down
12 changes: 6 additions & 6 deletions static/docs/commands-reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,8 @@ remote. See `dvc remote` for more information.
The DVC cache is a hidden storage (by default located in the `.dvc/cache`
directory) for files that are under DVC control, and their different versions.
(See `dvc cache` and
[DVC internal files](/doc/user-guide/dvc-files-and-directories) for more
details.)
[DVC internal files](/doc/user-guide/dvc-files-and-directories#structure-of-cache-directory)
for more details.)

- `cache.dir` - set/unset cache directory location. A correct value must be
either an absolute path or a path **relative to the config file location**.
Expand Down Expand Up @@ -180,7 +180,7 @@ more about the state file that is used for optimization.
so that when it needs to cleanup the database it could sort them by the
timestamp and remove the oldest ones. Default quota is set to 50(percent).

## Examples: Core config options
## Example: Core config options

Set the `dvc` log level to `debug`:

Expand All @@ -200,7 +200,7 @@ $ dvc remote add myremote s3://bucket/path
$ dvc config core.remote myremote
```

## Examples: Default remotes
## Example: Default remotes

Use remote `myremote` by default:

Expand All @@ -227,9 +227,9 @@ which is equivalent to:
$ dvc config core.remote -u
```

## Examples: Cache config options
## Example: Cache config options

Set the cache directory to an absolute path:
Set the <abbr>cache directory</abbr> to an absolute path:

```dvc
$ dvc config cache.dir /mnt/cache
Expand Down
14 changes: 7 additions & 7 deletions static/docs/commands-reference/destroy.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ usage: dvc destroy [-h] [-q | -v] [-f]
## Description

It removes DVC-files, and the entire `.dvc/` meta directory from the current
workspace. Note that the DVC cache will normally be removed as well, unless it's
set to an external location with `dvc cache dir`. (By default a local cache is
located in the `.dvc/cache` directory.) If you were using
workspace. Note that the <abbr>DVC cache</abbr> will normally be removed as
well, unless it's set to an external location with `dvc cache dir`. (By default
a local cache is located in the `.dvc/cache` directory.) If you were using
[symlinks for linking data](/doc/user-guide/large-dataset-optimization) from the
cache, DVC will replace them with copies, so that your data is intact after the
DVC repository destruction.
Expand All @@ -31,7 +31,7 @@ DVC repository destruction.

- `-v`, `--verbose` - displays detailed tracing information.

## Example
## Examples

```dvc
$ dvc init
Expand All @@ -52,10 +52,10 @@ $ ls -a
.git code.py foo
```

## Example: External Cache Directory
## Example: External cache directory

By default, the cache location is `.dvc/cache`. Let's change the cache location
to `/mnt/cache` and then execute `dvc destroy` command:
By default, the <abbr>cache</abbr> location is `.dvc/cache`. Let's change the
cache location to `/mnt/cache` and then execute `dvc destroy` command:

```dvc
$ dvc init
Expand Down
Loading

0 comments on commit f9b9e33

Please sign in to comment.