Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: regular update (early August) #522

Merged
merged 16 commits into from
Aug 10, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
50212bd
import-url: try and update example; review term "Get Started" through…
jorgeorpinel Jul 30, 2019
c0cb7ed
update: change references to import stages being unlockable
jorgeorpinel Aug 4, 2019
be18aad
cmd ref: standardize example headers and expandables up to `dvc import`
jorgeorpinel Aug 4, 2019
a388712
cmd ref: standardize example headers and expandables up to `dvc update`
jorgeorpinel Aug 7, 2019
6946023
cmd ref: standardize example headers and expandables (finished)
jorgeorpinel Aug 7, 2019
e641349
term: update "workspace" glossary entry title and desc
jorgeorpinel Aug 8, 2019
f1dc617
cmd ref: updated shared example intro in checkout, commit, and fetch
jorgeorpinel Aug 8, 2019
2536b99
lint: Make YAML blocks valid
jorgeorpinel Aug 8, 2019
686561b
term: update "workspace" glossary entry and
jorgeorpinel Aug 9, 2019
fc28eb0
get-started: updates for more and better use of term "workspace"
jorgeorpinel Aug 9, 2019
8781a71
glossary: use new format with ES6 template literals
jorgeorpinel Aug 10, 2019
328b2ec
get-started: make noted text about git commit in add-files into a reg…
jorgeorpinel Aug 10, 2019
b949520
pipeline: small improvement in cmd ref short desc.
jorgeorpinel Aug 10, 2019
796e799
Merge remote-tracking branch 'upstream/master'
jorgeorpinel Aug 10, 2019
ee1cf35
tutorial: change `<<<<<<<` etc Git conflict markers to HTML symbols
jorgeorpinel Aug 10, 2019
6d1f6c1
term: Review "cache directory" usage and add <abbr> tags for glossary…
jorgeorpinel Aug 10, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 36 additions & 30 deletions src/Documentation/glossary.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,54 +5,60 @@ export default {
'relevant DVC concepts and terminologies which are frequently used.',
contents: [
{
name: 'Workspace directory',
name: 'Workspace',
match: ['workspace'],
desc:
'The **workspace** contains all of your DVC **project** files and ' +
"directories. It's typically also a Git **repository**. See also " +
'[`dvc init`](/doc/commands-reference/init).'
desc: `
By "workspace" we refer to the directory containing all your project files. For
example raw datasets, source code, ML models, etc. A workspace becomes a DVC
project when [\`dvc init\`](/doc/commands-reference/init) is run, and
[DVC-files](/doc/user-guide/dvc-file-format) are created in it. It\s typically
also a Git repository.
`

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as outdated.

This comment was marked as resolved.

Copy link
Contributor Author

@jorgeorpinel jorgeorpinel Aug 9, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If your question is whether the linter will fix lines that go over 80 chars, no. Neither with the previous or with this format does it fix them, neither with the hook nor run manually 🙁. It seems to leave strings alone.

Is that another issue to report to Aman?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can def check if it's possible to configure it to fix them. In any case, I don't mind the format you suggested, its' def easier to support manually

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @algomaster99 what do you think? (about pretty-quick being able to warp strings in js files)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks a bit strange, but let's try

Applied new format to all glossary entries in 8781a71

},
{
name: 'DVC cache',
name: 'DVC Cache',
match: ['DVC cache', 'cache', 'cache directory'],
desc:
'The DVC cache is a hidden storage (by default located in the ' +
'`.dvc/cache` directory) for files that are under DVC control, and ' +
'their different versions. For more details, please refer to this ' +
'[document](/doc/user-guide/dvc-files-and-directories' +
'#structure-of-cache-directory).'
desc: `
The DVC cache is a hidden storage (by default located in the \`.dvc/cache\`
directory) for files that are under DVC control, and their different versions.
For more details, please refer to this
[document](/doc/user-guide/dvc-files-and-directories#structure-of-cache-directory).
`
},
{
name: 'Data artifact',
name: 'Data Artifact',
match: ['data artifact', 'data artifacts'],
desc:
'Any **data** file or directory, as well as intermediate or final ' +
'result (such as extracted features or a ML model file) that is ' +
'under DVC control. Refer to [Data and Model Files Versioning]' +
'(/doc/use-cases/data-and-model-files-versioning) for more details.'
desc: `
Any data file or directory, as well as intermediate or final result (such as
extracted features or a ML model file) that is under DVC control. Refer to
[Data and Model Files Versioning](/doc/use-cases/data-and-model-files-versioning)
for more details.
`
},
{
name: 'Import stage',
name: 'Import Stage',
match: ['import stage', 'import stages'],
desc:
'Stage (DVC-file) created with the `dvc import` or `dvc import-url` ' +
'commands. They represent files or directories from external sources.'
desc: `
Stage (DVC-file) created with the \`dvc import\` or \`dvc import-url\`
commands. They represent files or directories from external sources.
`
},
{
name: 'Output',
match: ['output', 'outputs'],
desc:
'A file or a directory that is under DVC control. See `dvc add`, ' +
'`dvc run`, `dvc import`, `dvc import-url` commands.'
desc: `
A file or a directory that is under DVC control. See \`dvc add\` \`dvc run\`,
\`dvc import\`, \`dvc import-url\` commands.
`
},
{
name: 'External Dependency',
match: ['external dependency', 'external dependencies'],
desc:
'A DVC-file dependency with origin in an external source, for ' +
'example HTTP, SSH, Amazon S3, Google Cloud Storage remote ' +
'locations, or even other DVC repositories. See ' +
'[External Dependencies](/doc/user-guide/external-dependencies).'
desc: `
A DVC-file dependency with origin in an external source, for example HTTP, SSH,
Amazon S3, Google Cloud Storage remote locations, or even other DVC repositories.
See [External Dependencies](/doc/user-guide/external-dependencies).
`
}
]
}
4 changes: 2 additions & 2 deletions static/docs/commands-reference/add.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@ models). This way you bring data provenance and make your project reproducible.
location, by including a path in the provided value (e.g.
`-f stages/stage.dvc`).

## Examples: Single file
## Example: Single file

Take a file under DVC control:

Expand Down Expand Up @@ -158,7 +158,7 @@ $ file .dvc/cache/d8/acabbfd4ee51c95da5d7628c7ef74b
.dvc/cache/d8/acabbfd4ee51c95da5d7628c7ef74b: ASCII text
```

## Examples: Directory
## Example: Directory

What if you have not one dog picture, but hundreds of pictures of dogs and cats?
Your goal might be to build an algorithm to identify dogs and cats in pictures,
Expand Down
13 changes: 9 additions & 4 deletions static/docs/commands-reference/cache/dir.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# cache dir

Set/unset the cache directory location intuitively (compared to using
`dvc config cache`).
Set/unset the <abbr>cache directory</abbr> location intuitively (compared to
using `dvc config cache`).

## Synopsis

Expand Down Expand Up @@ -40,7 +40,12 @@ file location**. They are required in the latter form for the config file.

- `-h`, `--help` - prints the usage/help message, and exit.

## Examples: Using relative path
- `-q`, `--quiet` - do not write anything to standard output. Exit with 0 if no
problems arise, otherwise 1.

- `-v`, `--verbose` - displays detailed tracing information.

## Example: Using relative path

```dvc
$ dvc cache dir ../dir
Expand All @@ -54,7 +59,7 @@ $ cat .dvc/config
`../dir` has been resolved relative to the `.dvc/` dir, resulting in
`../../dir`.

## Examples: Using absolute path
## Example: Using absolute path

```dvc
$ dvc cache dir /path/to/dir
Expand Down
14 changes: 1 addition & 13 deletions static/docs/commands-reference/cache/index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# cache

Contains a helper command to set the cache directory location:
Contains a helper command to set the <abbr>cache directory</abbr> location:
[dir](/doc/commands-reference/cache/dir).

## Synopsis
Expand Down Expand Up @@ -34,15 +34,3 @@ config option, for more information on file links on different platforms.)
problems arise, otherwise 1.

- `-v`, `--verbose` - displays detailed tracing information.

## Example

The main use of this command is to apply the `-v` and `-q` options to
`dvc cache dir` which doesn't have them:

```dvc
$ dvc cache --verbose dir mycache
DEBUG: Writing '/Users/user/myproject/.dvc/config'.
$ dvc config cache.dir
../mycache
```
28 changes: 6 additions & 22 deletions static/docs/commands-reference/checkout.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,36 +103,21 @@ be pulled from a remote cache using `dvc pull`.

## Examples

To explore `dvc checkout` let's consider a simple workspace with several
[stages](/doc/commands-reference/run), and a few Git tags. Then with
`git checkout` and `dvc checkout` we can see what happens as we shift from tag
to tag.
Let's employ a simple workspace with some data, code, ML models, pipeline
stages, as well as a few Git tags, such as our
[get started example repo](https://github.com/iterative/example-get-started).
Then we can see what happens with `git checkout` and `dvc checkout` as we switch
from tag to tag.

<details>

### Click and expand to setup the project

This step is optional, and you can run it only if you want to run this examples
in your environment. First, you need to download the project:
Start by cloning our sample repo if you don't already have it:

```dvc
$ git clone https://github.com/iterative/example-get-started
```

Second, let's install the requirements. But before we do that, we **strongly**
recommend creating a virtual environment with
[virtualenv](https://virtualenv.pypa.io/en/stable/) or a similar tool:

```dvc
$ cd example-get-started
$ virtualenv -p python3 .env
$ source .env/bin/activate
```

Now, we can install requirements for the project:

```dvc
$ pip install -r requirements.txt
```

</details>
Expand All @@ -157,7 +142,6 @@ solving the problem:

```dvc
$ git tag

baseline <- first simple version of the model
bigram <- use bigrams to improve the model
```
Expand Down
26 changes: 11 additions & 15 deletions static/docs/commands-reference/commit.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,37 +90,33 @@ into play. It handles that last step of adding the file to the DVC cache.

## Examples

To explore `dvc commit` let's consider a simple pipeline with several stages:
the example workspace used in the [Get Started](/doc/get-started) section.
Let's employ a simple workspace with some data, code, ML models, pipeline
stages, such as the DVC project created in our [Get Started](/doc/get-started)
section. Then we can see what happens with `git commit` and `dvc commit` in
different situations.

<details>

### Click and expand to setup the project

This step is optional, and you can run it only if you want to run this examples
in your environment. First, you need to download the project:
Start by cloning our sample repo if you don't already have it:

```dvc
$ git clone https://github.com/iterative/example-get-started
$ cd example-get-started
```

Second, let's install the requirements. But before we do that, we **strongly**
recommend creating a virtual environment with
[virtualenv](https://virtualenv.pypa.io/en/stable/) or a similar tool:
Now let's install the requirements. But before we do that, we **strongly**
recommend creating a virtual environment with a tool such as
[virtualenv](https://virtualenv.pypa.io/en/stable/):

```dvc
$ cd example-get-started
$ virtualenv -p python3 .env
$ source .env/bin/activate
```

Now, we can install requirements for the project:

```dvc
$ pip install -r requirements.txt
```

Then download the precomputed data using:
Download the precomputed data using:

```dvc
$ dvc pull --all-branches --all-tags
Expand Down Expand Up @@ -221,7 +217,7 @@ $ ls .dvc/cache/70
And we've verified that `dvc commit` has saved the changes into the cache, and
that the new instance of `model.pkl` is in the cache.

## Example: Running without DVC
## Example: Running commands without DVC

It is also possible to execute the commands that are executed by `dvc repro` by
hand. You won't have DVC helping you, but you have the freedom to run any script
Expand Down
12 changes: 6 additions & 6 deletions static/docs/commands-reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,8 @@ remote. See `dvc remote` for more information.
The DVC cache is a hidden storage (by default located in the `.dvc/cache`
directory) for files that are under DVC control, and their different versions.
(See `dvc cache` and
[DVC internal files](/doc/user-guide/dvc-files-and-directories) for more
details.)
[DVC internal files](/doc/user-guide/dvc-files-and-directories#structure-of-cache-directory)
for more details.)

- `cache.dir` - set/unset cache directory location. A correct value must be
either an absolute path or a path **relative to the config file location**.
Expand Down Expand Up @@ -180,7 +180,7 @@ more about the state file that is used for optimization.
so that when it needs to cleanup the database it could sort them by the
timestamp and remove the oldest ones. Default quota is set to 50(percent).

## Examples: Core config options
## Example: Core config options

Set the `dvc` log level to `debug`:

Expand All @@ -200,7 +200,7 @@ $ dvc remote add myremote s3://bucket/path
$ dvc config core.remote myremote
```

## Examples: Default remotes
## Example: Default remotes

Use remote `myremote` by default:

Expand All @@ -227,9 +227,9 @@ which is equivalent to:
$ dvc config core.remote -u
```

## Examples: Cache config options
## Example: Cache config options

Set the cache directory to an absolute path:
Set the <abbr>cache directory</abbr> to an absolute path:

```dvc
$ dvc config cache.dir /mnt/cache
Expand Down
14 changes: 7 additions & 7 deletions static/docs/commands-reference/destroy.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ usage: dvc destroy [-h] [-q | -v] [-f]
## Description

It removes DVC-files, and the entire `.dvc/` meta directory from the current
workspace. Note that the DVC cache will normally be removed as well, unless it's
set to an external location with `dvc cache dir`. (By default a local cache is
located in the `.dvc/cache` directory.) If you were using
workspace. Note that the <abbr>DVC cache</abbr> will normally be removed as
well, unless it's set to an external location with `dvc cache dir`. (By default
a local cache is located in the `.dvc/cache` directory.) If you were using
[symlinks for linking data](/doc/user-guide/large-dataset-optimization) from the
cache, DVC will replace them with copies, so that your data is intact after the
DVC repository destruction.
Expand All @@ -31,7 +31,7 @@ DVC repository destruction.

- `-v`, `--verbose` - displays detailed tracing information.

## Example
## Examples

```dvc
$ dvc init
Expand All @@ -52,10 +52,10 @@ $ ls -a
.git code.py foo
```

## Example: External Cache Directory
## Example: External cache directory

By default, the cache location is `.dvc/cache`. Let's change the cache location
to `/mnt/cache` and then execute `dvc destroy` command:
By default, the <abbr>cache</abbr> location is `.dvc/cache`. Let's change the
cache location to `/mnt/cache` and then execute `dvc destroy` command:

```dvc
$ dvc init
Expand Down
Loading