I/O Error when opening files with custom process umask

[Toilal]

When opening a file with O_EXCL flag, nfs4j always receives 0000 filemode, making subsequent writes on this file failing. There's an easy reproduction case with inplace sed.

test.txt

test

test.sh

#!/bin/bash
sed -i 's/test/hello' test.txt

Using trace gives more information on what's hapening from the NFS client.

trace.sh

#!/bin/bash
strace -f -e trace=file test.sh

It shows this line, that should it should be created with 0600 flag.

open("./sedAaF5hB", O_RDWR|O_CREAT|O_EXCL, 0600) = 4

In nfs4j, it seems to only occurs with nfs 4.1 mount, as it goes throw OperationOPEN.java#L136 and then get a create filemode = 0000 at OperationOPEN.java#L150.

[Toilal]

After analysing source code of sed, this is caused by some tricks with umask system call.

It can be reproduced with the following python script

#!/usr/bin/env python3
import os

filename = './sedBk0bug'
try:
    os.remove(filename)
except:
    pass

oldmask = os.umask(0o700)
fd = os.open(filename, os.O_RDWR | os.O_CREAT, 0o600)
os.umask(oldmask)

os.write(fd, b'hello')
os.close(fd)

Without umask invocation, it just work as expected over NFS, but with umask invocation it fails.

[Toilal]

It seems to be related to some flaws in NFS spec, see this draft that add umask support for NFS4.2 : https://tools.ietf.org/id/draft-ietf-nfsv4-umask-05.html#RFC7862