use-cases: use 'dvc cache dir' command in shared dev server

[dmpetrov]

In the shared server use case https://dvc.org/doc/use-cases/shared-development-server we set up the shared cache by changing config dvc config cache.dir /path/to/dvc-cache while a separate command was introduced dvc cache dir /path/to/dvc-cache.

I'm trying to understand the motivation behind the command dvc cache dir... It feels like we should deprecate the command dvc cache dir or change the docs.

@efiop could you please explain the motivation behind the command?

UPDATE: also probably we should update dvc cache dir doc as well - it's not that clear from the doc (from #986 (comment))

[shcheklein]

The doc is just outdated.

There are two things behind this command:

treeverse/dvc#1284
treeverse/dvc#1421 (comment)
treeverse/dvc#1467

(didn't find specific discussion re the dvc cache dir itself, but the point is - we want to keep dvc config w/o any magic, w/o any transformations, while higher level commands like dvc cache dir provide some use case specific, higher level interface)

[dmpetrov]

@shcheklein thanks for the clarification.

we want to keep dvc config w/o any magic, w/o any transformations, while higher level commands like

It makes sense. The current dvc cache dir still seems to provide very little value for a dedicated command but it can be extended in the future. If we follow this logic dvc config cache.shared group should be also replaced to something higher level.

So, we can keep this issue to update the docs or close it if duplication. @jorgeorpinel you should know better.

[jorgeorpinel]

The doc is just outdated.

Why is https://dvc.org/doc/use-cases/shared-development-server outdated @shcheklein ? For an external cache you would probably prefer to use dvc config cache.dir instead of dvc cache dir because its easier to use the absolute path than a path relative to the project.

[shcheklein]

https://dvc.org/doc/command-reference/cache/dir#example-using-absolute-path - Absolute path /path/to/dir saved as is. (Probably we should update dvc cache dir doc as well - it's not that clear from the doc).

If we have two APIs and one is made to be user friendly why not advertise it?

The only value that comes to mind to keep the low level config manipulation is that we already have a lot of low level commands in the same doc (like group related, protected modes, etc).

It brings a good point if we don't have even a single place to recommend dvc cache dir in docs - why do we have it indeed. I understand and clearly remember the motivation behind it, but if we don't even recommend it in the doc that deals with changing cache location in the most common pattern, why bother?

(^^ this is not me trying to defend the dvc cache dir - it's a question that came to my mind, want to recalibrate my understanding of this command)

[jorgeorpinel]

Good points @shcheklein and sorry for taking so long to reply.

The only value that comes to mind to keep the low level config manipulation is that we already have a lot of low level commands in the same doc (like group related, protected modes, etc).

I think this is the reason to keep it as is: keeping the sequence of commands to a minimum complexity.

It brings a good point if we don't have even a single place to recommend dvc cache dir in docs - why do we have it

It's mentioned in a few docs (4). Not strongly, but more than dvc config cache.dir (just in this use case) at least... Perhaps we should first try to find more places to mention/link to it and if not, consider facing it out?

[efiop]

Also dvc cache dir has some magic sprinkled on it, it resolves your relpaths so they work with the location that the config is in. I'm sure you guys remember that issue from last year, where we've introduced that logic for local remotes because dvc cache dir ../path in repo root should be resolved to ../../path, because the config is in .dvc/config. Need to adjust the docs, that article is just way too old.