Deadlock on syscalls like rename() and sendfile()

[dimakuv]

When a syscall operates on two files, there seems to be a possibility of a deadlock.

A trivial example would be:

1. First process does rename("foo.txt", "bar.txt")
2. Second process does rename("bar.txt", "foo.txt")

Roughly at the same time. Another example would be sendfile().

Or maybe there is no problem actually? The Server can impose some ordering to prevent deadlocks?

[pwmarcz]

I'm assuming the objects have some kind of IDs (ideally strings, so you can incorporate path), so maybe we could use that to take locks in order? That doesn't even have to be imposed by server.

[mkow]

Lock hierarchy sounds doable, but you need to be careful and normalize paths and document that this assumes we can't have two views of the same files in the filesystem (i.e. you can't have the same directory mounted in two places).