Skip to content

Commit

Permalink
ls: compute relparts once for every dir
Browse files Browse the repository at this point in the history
Cuts down `dvc ls . data/mnist/dataset --recursive` for me from ~25 to ~12
seconds.

`relparts` is an expensive operation as it calls `relpath` that calls `abspath`,
so we want to avoid calling it as much as possible.
  • Loading branch information
efiop committed Jan 8, 2023
1 parent 684fc32 commit 0e5674f
Showing 1 changed file with 5 additions and 4 deletions.
9 changes: 5 additions & 4 deletions dvc/repo/ls.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,11 +70,12 @@ def _ls(
if not recursive:
files.update(dirs)

parts = fs.path.relparts(root, fs_path)
if parts == (".",):
parts = ()

for name, entry in files.items():
entry_fs_path = fs.path.join(root, name)
relparts = fs.path.relparts(entry_fs_path, fs_path)
name = os.path.join(*relparts)
infos[name] = entry
infos[os.path.join(*parts, name)] = entry

if not recursive:
break
Expand Down

0 comments on commit 0e5674f

Please sign in to comment.