Size of syscall.Dirent not always consistent with FUSE

[lechner]

Hi,

This is from Debian packaging. On i386 the Getdents test in version 1.4.2 fails because the syscall.Dirent data structure gocryptfs uses has inconsistent sizes between architectures.

While the inode and offset members can vary in scenarios like this, they are not the issue here. The golang runtime uses only the getdents64 syscall and via cgo tweaks struct dirent to its liking (although I might have preferred struct dirent64 ). The issue here are the data types for name length and file type, as well as the padding for alignment.

On all architectures, both go-fuse and libfuse return at most 280 bytes. On amd64, that size matches the syscall.Dirent data type gocryptfs uses, but on i386 the latter drops to 276 bytes. It causes an error in internal/syscallcompat/getdents_linux.go:

if int(s.Reclen) > sizeofDirent {
    tlog.Warn.Printf("Getdents: corrupt entry #%d: Reclen=%d > %d. Returning EBADR",
        numEntries, s.Reclen, sizeofDirent)
    return nil, syscall.EBADR
}

Here is a small program to illustrate the various sizes. This is the output for amd64:

gocryptfs
unsafe.Sizeof (syscall.Dirent{})                 =  280

go-fuse
unsafe.Sizeof (gofuse_Dirent{}) + 256            =  280

libfuse
sizeof (struct libfuse_dirent) + 256             =  280

readdir(3), provided by libc
sizeof (struct dirent)                           =  280
sizeof (struct dirent64)                         =  280

getdents(3) syscall, not wrapped by libc
sizeof (struct linux_dirent) with 256[]          =  280

proposal to bring go-fuse in line with gocryptfs
unsafe.Sizeof (PROPOSED_gofuse_Dirent{}) + 256   =  280

For i386, the sizes are not nearly as consistent:

gocryptfs
unsafe.Sizeof (syscall.Dirent{})                 =  276

go-fuse
unsafe.Sizeof (gofuse_Dirent{}) + 256            =  280

libfuse
sizeof (struct libfuse_dirent) + 256             =  280

readdir(3), provided by libc
sizeof (struct dirent)                           =  276
sizeof (struct dirent64)                         =  276

getdents(3) syscall, not wrapped by libc
sizeof (struct linux_dirent) with 256[]          =  268

proposal to bring go-fuse in line with gocryptfs
unsafe.Sizeof (PROPOSED_gofuse_Dirent{}) + 256   =  276

Which data structure should be used in gocryptfs? Bonus question: Is FUSE safe on i386? Thank you!

[rfjakob]

Wow, good find.

The FUSE side of things should be fine, as it has its own fuse_dirent definition at https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?h=v4.12#n707 .

struct fuse_dirent {
	uint64_t	ino;
	uint64_t	off;
	uint32_t	namelen;
	uint32_t	type;
	char name[];
};

This is independent of what the getdents64 syscall uses.

Looks like the syscall.Dirent definition of i386 is wrong. Will check.

[rfjakob]

syscall.Dirent (definition) seems to match what Linux has (definition) - or at least it is NOT smaller than what Linux has.

What is Reclen in the warning you get?

[lechner]

Like you I think golang does the right thing.

The print statement unintentionally reverses the values (my quote was patched) but it was that s.Reclen from FUSE is 280 while the space reserved via syscall.Dirent was 276.

Here is the build log.

[lechner]

I am thinking about using Linux's FUSE dirent until go-fuse exports its _Dirent type. Would you like to see a pull request? Thank you!

[rfjakob]

No the FUSE binary interface and the syscall interface are independent, using the FUSE data type for the getdents syscall would break things worse

…

On Jan 25, 2018 16:55, "Felix Lechner" ***@***.***> wrote: I am thinking about using Linux's FUSE dirent <https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/uapi/linux/fuse.h?h=v4.12#n707> until go-fuse exports its _Dirent type. Would you like to see a pull request? Thank you! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#197 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AARgf4sYJ2kV7ZKjzqEOo7dRpQ5jTtiCks5tOKQMgaJpZM4RsIFS> .

[rfjakob]

It looks like the assumption that reclen cannot be longer than the Dirent is just wrong. Linux can insert arbitrary padding in between. And it does, to improve alignment, apparently: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/readdir.c?h=v4.15-rc9#n173

[rfjakob]

Fixed by f3838c0 .