Why don't patterns that match empty strings match empty files?

[jzinn]

In trying to use rg -l -g as a find replacement, I noticed that patterns that match empty strings don't match empty files.

For example:

% touch empty
% printf "a" > no_newline
% echo a > nonempty
% rg -l ''
no_newline
nonempty
% rg -l '.?'
no_newline
nonempty
% rg -l '^'
no_newline
nonempty

This is also how grep behaves.

But why?

[BurntSushi]

Sorry, I missed this question and it probably dipped too far into my email inbox and I ended up forgetting about it.

This is a good question, although ambiguous. On the one hand, maybe you're interested in "why does the implementation behave this way?" Or maybe you're interested in, "why did you choose to implement these semantics?" I suspect you're interested in the latter. But I'll answer the former too.

For the former, the answer is that when ripgrep goes to read a file, it has to know when to stop reading and thus stop searching. It knows this when read syscalls return 0 bytes, which conventionally indicates EOF. So in the current implementation, for an empty file, ripgrep issues a read call, sees that it's at EOF and then stops before ever executing a search.

For the latter question... Incidentally, I cannot remember ever explicitly thinking about what the behavior ought to be here. Interestingly, despite not considering it explicitly, ripgrep and grep do indeed behave the same. Looking at the POSIX specification for grep, I don't see anything that specifies the behavior for this particular case. So either they didn't consider it either, or they intentionally left it up to the implementation to decide.

So trying to reason about this from first principles, it does kind of seem like you could spin this either way, and it primarily depends on how you interpret "empty file."

1. An empty file contains no lines, and since greps only match lines and an empty file contains none, an empty file can never be matched.
2. An empty file contains a single empty line, and since the empty string regex matches the empty string, an empty file should be reported as a match for the query corresponding to the "null" or "empty string" regex.

Incidentally, if you did choose interpretation (2), does that mean, for example, that printf '' | wc -l should return 1? Because it doesn't! It returns 0. Similarly, if you go with interpretation (2), then printf 'foo\n' | grep '' should return two lines that match, because we previously declared that an empty string (which follows that last \n) is itself a line containing nothing. But only one line matches.

So it seems to me like interpretation (1) is the most consistent with other tools. You could say that the empty file is really a special case, but then you'd have to reconcile it with things like printf '' | wc -l returning 0.

[jzinn]

Thanks for your thoughtful answer.

I wasn't thinking deeply enough about greps being line-oriented tools.

POSIX defines an incomplete line as

A sequence of one or more non- <newline> characters at the end of the file.

Because the definition specifies "one or more" and not "zero or more", the empty file does not have an incomplete line for a grep tool to test.

POSIX seems to specify your interpretation (1).

[BurntSushi]

Ah interesting, nice find! Thanks. :)