add: empty files add broken when cache mode is hardlinks

[shcheklein]

DVC Version

DVC version: 0.86.5+f67314.mod
Python version: 3.7.6
Platform: Darwin-18.2.0-x86_64-i386-64bit
Binary: False
Package: None
Cache: reflink - supported, hardlink - supported, symlink - supported
Filesystem type (cache directory): ('apfs', '/dev/disk1s1')
Filesystem type (workspace): ('apfs', '/dev/disk1s1')

Reproduce

dvc config cache.type hardlink
touch foo
dvc add foo

outputs:

ERROR: no possible cache types left to try out.

[skshetry]

Found the root cause of this. This happens because of the optimization that we do if the file is empty (i.e. 0 bytes size). We never create a hardlink for the file with 0 bytes size and therefore it fails when we try to verify if the hardlink was created.

dvc/dvc/remote/local.py

Lines 170 to 182 in a9bc65e

	# If there are a lot of empty files (which happens a lot in datasets),
	# and the cache type is `hardlink`, we might reach link limits and
	# will get something like: `too many links error`
	#
	# This is because all those empty files will have the same checksum
	# (i.e. 68b329da9893e34099c7d8ad5cb9c940), therefore, they will be
	# linked to the same file in the cache.
	#
	# From https://en.wikipedia.org/wiki/Hard_link
	# * ext4 limits the number of hard links on a file to 65,000
	# * Windows with NTFS has a limit of 1024 hard links on a file
	#
	# That's why we simply create an empty file rather than a link.

I think, is_hardlink() function should be aware of this optimization as well. I'll make a fix tomorrow.

P.S. This only failed if the file was empty (not even a newline). So:

echo > foo && dvc add foo # works because of extra-newline character added by `echo`
touch bar && dvc add bar # does not work as the file is empty

Also, subsequent runs would not fail because before this error is thrown out, the file gets cached (however, deletes file from workspace), i.e.

dvc init --no-scm
dvc config cache.type hardlink
touch foo && dvc add foo  # fails and deletes foo from workspace
touch foo && dvc add foo  # passes

[efiop]

@skshetry Great investigation! 🙏