Common UI improvements

[dberenbaum]

Goals for UI:

1. Abstract UI implementation to apply high-level functions/classes to all UI components

• abstract cli ui utilities, move away from logger.info for display #5347 and introduce dvc.io module to consolidate io utilities #5348

2. Establish CLI/UI guidelines

3. Improve formatting, style, and functionality

@skshetry @efiop @pmrowla

[dberenbaum]

Referenced in #5282 (comment): Subcommand should usually be plural

[jorgeorpinel]

What about extracting all user messages to a central location for easy access to the prod team to help with? Could even help for future i18n (more context).

[dberenbaum]

See #5891 for --quiet guidelines

[dberenbaum]

@skshetry Are there other issues that can be linked here?

[jorgeorpinel]

We decided to adopt https://clig.dev as a base guideline for the CLI applications

p.s. would it be worth giving a quick talk to others on this? E.g. in an all hands meeting or in a bash perhaps (cc @casperdcl)

[jorgeorpinel]

The add command is quiet about what is done, at the end, there's only a progress bar (and, an ugly hint).

Not sure why you consider the progress bar to be quiet @skshetry . I think it communicates what's happening clearly enough (generally speaking).

💡 The exact phrases used in all messages can always be improved and to that effect I propose to consider an i18n-like effort to refactor the code extracting message strings to a separate module or config file anyone can easily propose updates on.

Idk what to think about the Git hints. I give them for granted at this point but thinking about it, I'm not sure. Are they useful realistically? Or should people know what to do after dvc add? Maye changing that to a more explicit message on what happened would be better e.g. 'data.csv' was cached. './data.csv.dvc' was created. so the user can decide what to do with the results.

Do we need progress bars at all

Yes. These operations can take a long time. It's important to know that there's progress.

How should DVC show progress bars/status on places where it might take a lot of time?

I see some problems with the current way (shown in the 2nd animation above):

1. The main progress bar starts at 0/2. Why 0? And it never prints the name data (first dir given)
2. "computing hashes" states it's only done once but it seems to happen more than once (once per target I guess?).
3. The 2nd top bar jumps from 0 to 2/2 and prints the name data2 (2nd dir given).

In all it is a bit confusing to try interpret those bars. Can there be a single, persistent master bar on top and then smaller bars for each subprocess underneath?

[iesahin]

I give them for granted at this point but thinking about it, I'm not sure. Are they useful realistically? Or should people know what to do after dvc add?

They are misleading. I think silence is better than providing an incorrect command. (Some of these files are .gitignored, see the difference between proposed git add and git status)

I believe building the command verbose doesn't add much value anyway. For the first few times, they may lead the user but after that, they are ignored. Verbosity must be optional, or at least could be turned off using config.

Even notoriously difficult tools like ffmpeg don't lecture about the usage as much as DVC. I don't feel it's the right attitude towards the users, when we tell "you should do git add after this command", we are signaling "you don't know how to use this tool and we don't trust you." This may be true for some new users, but for veterans these messages are a nuisance I believe.

[skshetry]

Or should people know what to do after dvc add?

I am hesitant to remove this on dvc add as I have seen a lot of users just copy and paste that. I don't expect everyone using DVC to understand git. They might not get very far, but for "Data Versioning" in DVC, it's enough. And, git add hint on dvc add is very stable. I do have plans for suppressing those hints messages.

For experiments and pipelines, I don't have strong opinions.

[skshetry]

Update on UI improvement:

1. Index was introduced in introduce index #6300.
2. In dvc add, some progress bars were merged and reduced. A few spinners during stage collection and graph checks were added. We still need to fix how we save files to cache which we decided on guideline: don't break the workspace #6252.
   Also see add: support rollback/recovery from partial/failed dvc add #6387.
3. dvc add duplicated targets are now ignored, instead of failing with ambiguous error.
4. Updater was changed to only show on dvc version, thanks to @efiop.
5. Updater message was changed from in-box to pip like output.
6. Introduced DVC_ANALYTICS env var thanks to @efiop.
7. fsspec callback was introduced by @isidentical and push/pull transfers were unified by @pmrowla, making it possible for us to have a single progress bar: progress: push and pull: single bytes-based bar #3682.

The #3682 is now being worked on. 🙂

[dberenbaum]

@skshetry Do you think we can close this and open any more specific UI issues, or is it helpful to keep this open?

[dberenbaum]

Closing as stale. Let's open more specific follow-ups as needed.