x/sys/unix: Solaris lacks Flock (plus related constants)

[calmh]

Primarily needed by Bolt which currently builds on "everything" except Solaris.

# github.com/boltdb/bolt
./bolt_unix.go:24: undefined: syscall.LOCK_SH
./bolt_unix.go:26: undefined: syscall.LOCK_EX
./bolt_unix.go:30: undefined: syscall.Flock
./bolt_unix.go:30: undefined: syscall.LOCK_NB
./bolt_unix.go:44: undefined: syscall.Flock
./bolt_unix.go:44: undefined: syscall.LOCK_UN
./bolt_unix.go:61: undefined: syscall.Mmap
./bolt_unix.go:86: undefined: syscall.Munmap
./bolt_unix.go:95: undefined: syscall.SYS_MADVISE

[minux]

Have you tried golang.org/x/sys/unix? (The issue subject mentions x/sys, but your code still references the syscall pakage)

[calmh]

Yes, it seems not implemented there either, unless I'm missing it.

jb@syno:~/src/golang.org/x/sys (master) $ git grep 'func Mmap'
unix/syscall_bsd.go:func Mmap(fd int, offset int64, length int, prot int, flags int) (data []byte, err error) {
unix/syscall_linux.go:func Mmap(fd int, offset int64, length int, prot int, flags int) (data []byte, err error) {
jb@syno:~/src/golang.org/x/sys (master) $ 

(similar for Flock; hits, but not in something that looks like it would build on Solaris)

[4ad]

We can fix this trivially in x/sys, but it still won't solve your problem, as bolt has to be changed as well to start using x/sys instead of syscall.

[calmh]

That's not a problem though. Bolt I can fix, but I'm not sure of the changes necessary in x/sys.

[binarycrusader]

The Solaris OS itself does not currently support flock(); could you use FcntlFlock() instead?

[calmh]

It compiles and doesn't return an error... Although I didn't perform any deeper testing of it.

jb@zlogin4:~ $ uname -a
SunOS zlogin4.nym.se 5.11 joyent_20150709T171818Z i86pc i386 i86pc Solaris
jb@zlogin4:~ $ cat test.c 
#include <sys/file.h>
#include <stdio.h>
#include <stdlib.h>

int main() {
    FILE *fp;
    fp = fopen("test.c", "r");
    if (fp == NULL) {
        perror("fopen");
        exit(1);
    }
    if (flock(fileno(fp), LOCK_EX) != 0) {
        perror("flock");
        exit(1);
    }
    printf("test.c locked\n");
    return 0;
}

jb@zlogin4:~ $ gcc -o test test.c
jb@zlogin4:~ $ ./test 
test.c locked
jb@zlogin4:~ $ 

Nevertheless that's the least important part of it, as you note there are other ways to do locking anyway.

[binarycrusader]

I was referring to Solaris, not OpenSolaris derivatives such as SmartOS.

Solaris had an flock method historically but it was not semantically compatible with Linux and was removed from later versions of Solaris.

Since the Solaris port of Go is targeted to both Solaris and OpenSolaris-based derivatives, flock is not supportable on Solaris for Go for now.

I also think the precise semantics of whatever version of flock available on SmartOS are would need to be verified to be the same as other OS' given what I know about the original sources.

[akolb-imd]

I have a working implementation for locking using fcntl() interface that is currently used by etcd. I made version for bolt as well.

[akolb1]

// flock acquires an advisory lock on a file descriptor.
func flock(f *os.File, exclusive bool, timeout time.Duration) error {
    var t time.Time
    for {
        // If we're beyond our timeout then return an error.
        // This can only occur after we've attempted a flock once.
        if t.IsZero() {
            t = time.Now()
        } else if timeout > 0 && time.Since(t) > timeout {
            return ErrTimeout
        }
        var lock syscall.Flock_t
        lock.Start = 0
        lock.Len = 0
        lock.Pid = 0
        lock.Whence = 0
        lock.Pid = 0
        if exclusive {
            lock.Type = syscall.F_WRLCK
        } else {
            lock.Type = syscall.F_RDLCK
        }
        err := syscall.FcntlFlock(f.Fd(), syscall.F_SETLK, &lock)
        if err == nil {
            return nil
        } else if err != syscall.EAGAIN {
            return err
        }

        // Wait for a bit and try again.
        time.Sleep(50 * time.Millisecond)
    }
}

// funlock releases an advisory lock on a file descriptor.
func funlock(f *os.File) error {
    var lock syscall.Flock_t
    lock.Start = 0
    lock.Len = 0
    lock.Type = syscall.F_UNLCK
    lock.Whence = 0
    return syscall.FcntlFlock(uintptr(f.Fd()), syscall.F_SETLK, &lock)
}

[ksedgwic]

@akolb-imd Thanks! I'm working on boltdb for Solaris as well.

I think I have a patch to x/sys/unix which fixes the build and provides mmap ... is this of interest?

-- Ken

[akolb1]

@ksedgwic Yes, please. I am especially interested in your mmap fix. I hacked golfing to provide mmap, but fixing it in bolt is definitely better.

BTW, why bolt uses mmap? It is very OS-specific and in many cases isn't even faster. Would be better to avoid it altogether.

[akolb1]

@ksedgwic Does the flock/funlock fix look reasonable?

[akolb1]

That's what I did for Solaris build:

--- a/src/syscall/syscall_solaris.go
+++ b/src/syscall/syscall_solaris.go
@@ -526,3 +526,17 @@ func writelen(fd int, buf *byte, nbuf int) (n int, err error) {
        }
        return
 }
+
+var mapper = &mmapper{
+       active: make(map[*byte][]byte),
+       mmap:   mmap,
+       munmap: munmap,
+}
+
+func Mmap(fd int, offset int64, length int, prot int, flags int) (data []byte, err error
+       return mapper.Mmap(fd, offset, length, prot, flags)
+}
+
+func Munmap(b []byte) (err error) {
+       return mapper.Munmap(b)
+}

[ksedgwic]

I'm new to boltdb so can't speak to the reason they used mmap.

My patch is for x/sys/unix itself. It appears the solaris build has been broken for some time:

 http://build.golang.org/?repo=golang.org%2fx%2fsys

Here is my x/sys patch:

diff --git a/unix/mksyscall_solaris.pl b/unix/mksyscall_solaris.pl
index e81edb9..9ec19e0 100755
--- a/unix/mksyscall_solaris.pl
+++ b/unix/mksyscall_solaris.pl
@@ -269,7 +269,10 @@ print <<EOF;

 package $package

-import "unsafe"
+import (
+       "syscall"
+       "unsafe"
+)
 EOF

 print "import \"golang.org/x/sys/unix\"\n" if $package ne "unix";
diff --git a/unix/syscall_solaris.go b/unix/syscall_solaris.go
index ea5ac78..d5886ee 100644
--- a/unix/syscall_solaris.go
+++ b/unix/syscall_solaris.go
@@ -527,3 +527,17 @@ func writelen(fd int, buf *byte, nbuf int) (n int, err error) {
        }
        return
 }
+
+var mapper = &mmapper{
+       active: make(map[*byte][]byte),
+       mmap:   mmap,
+       munmap: munmap,
+}
+
+func Mmap(fd int, offset int64, length int, prot int, flags int) (data []byte, err error) {
+       return mapper.Mmap(fd, offset, length, prot, flags)
+}
+
+func Munmap(b []byte) (err error) {
+       return mapper.Munmap(b)
+}
diff --git a/unix/zsyscall_solaris_amd64.go b/unix/zsyscall_solaris_amd64.go
index 4c2f8bf..a05befa 100644
--- a/unix/zsyscall_solaris_amd64.go
+++ b/unix/zsyscall_solaris_amd64.go
@@ -5,92 +5,96 @@

 package unix

-import "unsafe"
+import (
+       "syscall"
+       "unsafe"
+)

 var (
-       modlibc      = syscall.newLazySO("libc.so")
-       modlibsocket = syscall.newLazySO("libsocket.so")
-
-       procgetgroups    = modlibc.NewProc("getgroups")
-       procsetgroups    = modlibc.NewProc("setgroups")
-       procfcntl        = modlibc.NewProc("fcntl")
-       procaccept       = modlibsocket.NewProc("accept")
-       procsendmsg      = modlibsocket.NewProc("sendmsg")
-       procAccess       = modlibc.NewProc("access")
-       procAdjtime      = modlibc.NewProc("adjtime")
-       procChdir        = modlibc.NewProc("chdir")
-       procChmod        = modlibc.NewProc("chmod")
-       procChown        = modlibc.NewProc("chown")
-       procChroot       = modlibc.NewProc("chroot")
-       procClose        = modlibc.NewProc("close")
-       procDup          = modlibc.NewProc("dup")
-       procExit         = modlibc.NewProc("exit")
-       procFchdir       = modlibc.NewProc("fchdir")
-       procFchmod       = modlibc.NewProc("fchmod")
-       procFchown       = modlibc.NewProc("fchown")
-       procFpathconf    = modlibc.NewProc("fpathconf")
-       procFstat        = modlibc.NewProc("fstat")
-       procGetdents     = modlibc.NewProc("getdents")
-       procGetgid       = modlibc.NewProc("getgid")
-       procGetpid       = modlibc.NewProc("getpid")
-       procGeteuid      = modlibc.NewProc("geteuid")
-       procGetegid      = modlibc.NewProc("getegid")
-       procGetppid      = modlibc.NewProc("getppid")
-       procGetpriority  = modlibc.NewProc("getpriority")
-       procGetrlimit    = modlibc.NewProc("getrlimit")
+       modlibc = newLazySO("libc.so")
+       modlibsocket = newLazySO("libsocket.so")
+
+       procgetgroups = modlibc.NewProc("getgroups")
+       procsetgroups = modlibc.NewProc("setgroups")
+       procfcntl = modlibc.NewProc("fcntl")
+       procaccept = modlibsocket.NewProc("accept")
+       procsendmsg = modlibsocket.NewProc("sendmsg")
+       procAccess = modlibc.NewProc("access")
+       procAdjtime = modlibc.NewProc("adjtime")
+       procChdir = modlibc.NewProc("chdir")
+       procChmod = modlibc.NewProc("chmod")
+       procChown = modlibc.NewProc("chown")
+       procChroot = modlibc.NewProc("chroot")
+       procClose = modlibc.NewProc("close")
+       procDup = modlibc.NewProc("dup")
+       procExit = modlibc.NewProc("exit")
+       procFchdir = modlibc.NewProc("fchdir")
+       procFchmod = modlibc.NewProc("fchmod")
+       procFchown = modlibc.NewProc("fchown")
+       procFpathconf = modlibc.NewProc("fpathconf")
+       procFstat = modlibc.NewProc("fstat")
+       procGetdents = modlibc.NewProc("getdents")
+       procGetgid = modlibc.NewProc("getgid")
+       procGetpid = modlibc.NewProc("getpid")
+       procGeteuid = modlibc.NewProc("geteuid")
+       procGetegid = modlibc.NewProc("getegid")
+       procGetppid = modlibc.NewProc("getppid")
+       procGetpriority = modlibc.NewProc("getpriority")
+       procGetrlimit = modlibc.NewProc("getrlimit")
        procGettimeofday = modlibc.NewProc("gettimeofday")
-       procGetuid       = modlibc.NewProc("getuid")
-       procKill         = modlibc.NewProc("kill")
-       procLchown       = modlibc.NewProc("lchown")
-       procLink         = modlibc.NewProc("link")
-       proclisten       = modlibsocket.NewProc("listen")
-       procLstat        = modlibc.NewProc("lstat")
-       procMkdir        = modlibc.NewProc("mkdir")
-       procMknod        = modlibc.NewProc("mknod")
-       procNanosleep    = modlibc.NewProc("nanosleep")
-       procOpen         = modlibc.NewProc("open")
-       procPathconf     = modlibc.NewProc("pathconf")
-       procPread        = modlibc.NewProc("pread")
-       procPwrite       = modlibc.NewProc("pwrite")
-       procread         = modlibc.NewProc("read")
-       procReadlink     = modlibc.NewProc("readlink")
-       procRename       = modlibc.NewProc("rename")
-       procRmdir        = modlibc.NewProc("rmdir")
-       proclseek        = modlibc.NewProc("lseek")
-       procSetegid      = modlibc.NewProc("setegid")
-       procSeteuid      = modlibc.NewProc("seteuid")
-       procSetgid       = modlibc.NewProc("setgid")
-       procSetpgid      = modlibc.NewProc("setpgid")
-       procSetpriority  = modlibc.NewProc("setpriority")
-       procSetregid     = modlibc.NewProc("setregid")
-       procSetreuid     = modlibc.NewProc("setreuid")
-       procSetrlimit    = modlibc.NewProc("setrlimit")
-       procSetsid       = modlibc.NewProc("setsid")
-       procSetuid       = modlibc.NewProc("setuid")
-       procshutdown     = modlibsocket.NewProc("shutdown")
-       procStat         = modlibc.NewProc("stat")
-       procSymlink      = modlibc.NewProc("symlink")
-       procSync         = modlibc.NewProc("sync")
-       procTruncate     = modlibc.NewProc("truncate")
-       procFsync        = modlibc.NewProc("fsync")
-       procFtruncate    = modlibc.NewProc("ftruncate")
-       procUmask        = modlibc.NewProc("umask")
-       procUnlink       = modlibc.NewProc("unlink")
-       procUtimes       = modlibc.NewProc("utimes")
-       procbind         = modlibsocket.NewProc("bind")
-       procconnect      = modlibsocket.NewProc("connect")
-       procmmap         = modlibc.NewProc("mmap")
-       procmunmap       = modlibc.NewProc("munmap")
-       procsendto       = modlibsocket.NewProc("sendto")
-       procsocket       = modlibsocket.NewProc("socket")
-       procsocketpair   = modlibsocket.NewProc("socketpair")
-       procwrite        = modlibc.NewProc("write")
-       procgetsockopt   = modlibsocket.NewProc("getsockopt")
-       procgetpeername  = modlibsocket.NewProc("getpeername")
-       procgetsockname  = modlibsocket.NewProc("getsockname")
-       procsetsockopt   = modlibsocket.NewProc("setsockopt")
-       procrecvfrom     = modlibsocket.NewProc("recvfrom")
-       procrecvmsg      = modlibsocket.NewProc("recvmsg")
+       procGetuid = modlibc.NewProc("getuid")
+       procKill = modlibc.NewProc("kill")
+       procLchown = modlibc.NewProc("lchown")
+       procLink = modlibc.NewProc("link")
+       proclisten = modlibsocket.NewProc("listen")
+       procLstat = modlibc.NewProc("lstat")
+       procMkdir = modlibc.NewProc("mkdir")
+       procMknod = modlibc.NewProc("mknod")
+       procNanosleep = modlibc.NewProc("nanosleep")
+       procOpen = modlibc.NewProc("open")
+       procPathconf = modlibc.NewProc("pathconf")
+       procPread = modlibc.NewProc("pread")
+       procPwrite = modlibc.NewProc("pwrite")
+       procread = modlibc.NewProc("read")
+       procReadlink = modlibc.NewProc("readlink")
+       procRename = modlibc.NewProc("rename")
+       procRmdir = modlibc.NewProc("rmdir")
+       proclseek = modlibc.NewProc("lseek")
+       procSetegid = modlibc.NewProc("setegid")
+       procSeteuid = modlibc.NewProc("seteuid")
+       procSetgid = modlibc.NewProc("setgid")
+       procSetpgid = modlibc.NewProc("setpgid")
+       procSetpriority = modlibc.NewProc("setpriority")
+       procSetregid = modlibc.NewProc("setregid")
+       procSetreuid = modlibc.NewProc("setreuid")
+       procSetrlimit = modlibc.NewProc("setrlimit")
+       procSetsid = modlibc.NewProc("setsid")
+       procSetuid = modlibc.NewProc("setuid")
+       procshutdown = modlibsocket.NewProc("shutdown")
+       procStat = modlibc.NewProc("stat")
+       procSymlink = modlibc.NewProc("symlink")
+       procSync = modlibc.NewProc("sync")
+       procTruncate = modlibc.NewProc("truncate")
+       procFsync = modlibc.NewProc("fsync")
+       procFtruncate = modlibc.NewProc("ftruncate")
+       procUmask = modlibc.NewProc("umask")
+       procUnlink = modlibc.NewProc("unlink")
+       procUtimes = modlibc.NewProc("utimes")
+       procbind = modlibsocket.NewProc("bind")
+       procconnect = modlibsocket.NewProc("connect")
+       procmmap = modlibc.NewProc("mmap")
+       procmunmap = modlibc.NewProc("munmap")
+       procsendto = modlibsocket.NewProc("sendto")
+       procsocket = modlibsocket.NewProc("socket")
+       procsocketpair = modlibsocket.NewProc("socketpair")
+       procwrite = modlibc.NewProc("write")
+       procgetsockopt = modlibsocket.NewProc("getsockopt")
+       procgetpeername = modlibsocket.NewProc("getpeername")
+       procgetsockname = modlibsocket.NewProc("getsockname")
+       procsetsockopt = modlibsocket.NewProc("setsockopt")
+       procrecvfrom = modlibsocket.NewProc("recvfrom")
+       procrecvmsg = modlibsocket.NewProc("recvmsg")
+
 )

 func getgroups(ngid int, gid *_Gid_t) (n int, err error) {
@@ -906,3 +910,5 @@ func recvmsg(s int, msg *Msghdr, flags int) (n int, err error) {
        }
        return
 }
+
+

[ksedgwic]

Please let me know if the patch works for you as well; if it does I'll submit it to x/sys folks ...

[akolb1]

@ksedgwic Thanks I'll try it. And since you are there, can you also add

--- a/src/syscall/zsysnum_solaris_amd64.go
+++ b/src/syscall/zsysnum_solaris_amd64.go
@@ -8,6 +8,7 @@ package syscall

 // TODO(aram): remove these before Go 1.3.
 const (
+        SYS_IOCTL          = 54  // { int sys_ioctl(int fd, \
        SYS_EXECVE = 59
        SYS_FCNTL  = 62
 )

[ksedgwic]

@akolb1 I'll try out the flock code straightaway and let you know how it goes.

re: SYS_IOCTL, will do ...

[akolb1]

@ksedgwic The flock diff was for bolt but something similar can be done in golang to provide flock on Solaris. I have not checked the golang implementation.

BTW, now the newJoyent version of illumos provides flock() but I don't think that golfing can be that specific (unless it is possible to detect whether flock() is provided by the library)

[ksedgwic]

@akolb1 It's working! I can run the "bolt bench" command!

I had to add Madvise support to solaris as well.

Here is the complete patch for x/sys:
http://www.bonsai.com/ken/public/xsys_solaris.patch

Here is the complete patch for bolt:
http://www.bonsai.com/ken/public/bolt_solaris.patch

I think we should be careful about introducing flock into x/sys for solaris. Since it is not supported directly by the underlying OS/libraries we'd have to guarantee that we'd considered all possible semantics, not just the ones that bolt needs (example, can you flock a directory?)

[akolb1]

@ksedgwic Have you tried running bolt tests? They fail for me.

Regarding flock(), here is Joyent implementation of flock() for Linux brand - we can do exactly the same in GO:

long
lx_flock(uintptr_t p1, uintptr_t p2)
{
        int                     fd = (int)p1;
        int                     operation = (int)p2;
        struct flock            fl;
        int                     cmd;
        int                     ret;

        if (operation & LX_LOCK_NB) {
                cmd = F_FLOCK;
                operation &= ~LX_LOCK_NB; /* turn off this bit */
        } else {
                cmd = F_FLOCKW;
        }

        switch (operation) {
                case LX_LOCK_UN:
                        fl.l_type = F_UNLCK;
                        break;
                case LX_LOCK_SH:
                        fl.l_type = F_RDLCK;
                        break;
                case LX_LOCK_EX:
                        fl.l_type = F_WRLCK;
                        break;
                default:
                        return (-EINVAL);
        }

        fl.l_whence = 0;
        fl.l_start = 0;
        fl.l_len = 0;
        fl.l_sysid = 0;
        fl.l_pid = 0;

        ret = fcntl(fd, cmd, &fl);

        return ((ret == -1) ? -errno : ret);
}

[ksedgwic]

@akolb1 Three tests fail for me; I think they are flock relateddb_test.go:57:

--- FAIL: TestOpen_Timeout (0.00s)
db_test.go:86:

--- FAIL: TestOpen_Wait (0.00s)
db_test.go:257:

--- FAIL: TestOpen_ReadOnly (0.00s)

I'm looking at the first test now, looks like we failing to prevent a second open of the same db ...

[ksedgwic]

Ok, I think I understand the issue. An important difference between flock and fcntl is how they deal with threads in the same process:

http://stackoverflow.com/questions/29611352/what-is-the-difference-between-locking-with-fcntl-and-flock

My read is that fcntl ignores the fd (and threads) and considers whether the process holds the lock.
Flock locks per fd.

The unit test TestOpen_Timeout is opening the same file twice from the same process; so flock based locking will prevent that (test PASSES) and fcntl locking will allow it (test FAILS).

[akolb1]

I see. Is this specific to the test or not actually depends on such semantics? I would argue that the test is broken. Also there is an important difference is that fcntl-based locking usually works on NFS while lockf doesn't.

[akolb1]

@ksedgwic Your Solaris patch looks great (I don't understand GO bindings to libc so I can't comment on these). For madvise support - are Solaris-specific NUMA advise flags supported? (MADV_ACCESS_LWP, MADV_ACCESS_MANY, MADV_ACCESS_DEFAULT) ?

[akolb1]

@ksedgwic Do you plan to push your golang patches to the upstream?

[ksedgwic]

@akolb1 I plan on pushing the x/sys patch upstream. I would like to get it all working first.

I think the flock semantics may actually be important. Consider a multi-threaded server of some sort which opens the boltdb separately from each thread. With flock it all works fine; their access is consistent. With fcntl it breaks. I'm currently investigating a fix, will know in an hour or so how it's going ...

[ksedgwic]

If multiple threads have the same file open with different fds:
flock prevents corruption because it only allows one fd exclusive access at a time.
fcntl allows corruption because it allows multiple threads access (ignores per-fd locking).

[akolb-imd]

Moving this offline.

On August 14, 2015 at 2:08:25 PM, Ken Sedgwick (notifications@github.com) wrote:

If multiple threads have the same file open with different fds:
flock prevents corruption because it only allows one fd exclusive access at a time.
fcntl allows corruption because it allows multiple threads access (ignores per-fd locking).

Hmm - fcntl provides locks at the file level so it doesn’t matter whether you go to the file through the same fd or through different fds - so why would it allow corruption from multiple threads?

[ksedgwic]

Because if different threads in the same process open the file with different fds, they will be given concurrent access, which will cause corruption. See: https://gist.github.com/MerlinDMC/3197f4d13f8145c457e4

[akolb-imd]

On Aug 14, 2015, at 2:35 PM, Ken Sedgwick notifications@github.com wrote:

Because if different threads in the same process open the file with different fds, they will be given concurrent access, which will cause corruption. See: https://gist.github.com/MerlinDMC/3197f4d13f8145c457e https://gist.github.com/MerlinDMC/3197f4d13f8145c457e

This shouldn’t be the case - fcntl is implemented at vnode, so it shouldn’t give concurrent access.

So I wrote a simple test in go to verify this. It isn’t multithreaded - you need to open two windows to try it. The source is attached.

In window 1:

./locks

In window2:

./locks
Can not grab lock: resource temporarily unavailable

So I go through two different opens with different fds.

[ksedgwic]

Right, but if you implement both parts of the test in the same process (like a multithreaded server) then it will fail and allow concurrent exclusive access. This is what the failing unit test does. This is what a reasonable multi-threaded server might do.

See the problem?

In other words, the problem never happens when you have separate processes (like your test), that part works fine. The problem only happens when both lockers are inside the same process, even with two different fds.

[binarycrusader]

Guys, the x/sys/UNIX build failure is already covered by #10086:

#10086

I've already submitted a CL and it has been reviewed.

[ksedgwic]

@binarycrusader Looks great!

I'm pretty sure we're still going to need to fix the specific flock != fcntl issue, but that is probably better handled on the boltdb side (where the issue, and the fix would lie).

Thanks!

[akolb1]

@ksedgwic Oh, I see, you are right. I'll investigate this a bit further.

[ksedgwic]

@binarycrusader

1. I don't see the "mapper" wrapper that linux and bsd use to register active mmap'd regions in your CL:
   x/sys/unix/syscall_bsd.go:553
   x/sys/unix/syscall_linux.go:932
   Won't we need this on Solaris as well?
2. I don't see the Madvise call wrapped ... we need that too.

[akolb1]

On the recent illumos-joyent the man page for fcntl says:

       Two types of file locks are supported: POSIX-style and OFD-style. OFD-
       style locks are associated with the open file description (not
       descriptor) instead of with a process. Either type is advisory by
       default, but POSIX-style locks can be mandatory if, and only if,
       mandatory locking has been enabled on the file being locked.  Each type
       of lock may be created through two different interfaces. POSIX-style
       locks are created via the F_SETLK, F_SETLK64, F_SETLKW, or F_SETLKW64
       commands to this system call or by use of the lockf(3C) routine. There
       is no difference between locks created via one mechanism or the other.
       Likewise, OFD-style locks are created via the F_OFD_SETLK,
       F_OFD_SETLK64, F_OFD_SETLKW, or F_OFD_SETLKW64 commands to this system
       call or by use of the Linux/BSD-compatible flock(3C) routine. Note that
       this system call supports the creation of range-specified OFD-style
       file locks, while flock(3C) does not. However, the current
       implementation of OFD-style locking is limited to locking the entire
       file. This limitation might be removed in the future.


       The essential distinction between POSIX-style locks and OFD-style locks
       lie in how ownership of a lock is scoped. POSIX locks are scoped to a
       process. All POSIX locks associated with a file for a given process are
       removed when any file descriptor for that file is closed by that
       process or the process holding that file descriptor terminates. POSIX-
       style locks are not inherited by a child process created using fork(2).
       An OFD-style lock is scoped to the file description for a file, not the
       process or open file descriptor. Thus all file descriptors referring to
       the same description (i.e. those created via the F_DUPFD, F_DUP2FD,
       F_DUPFD_CLOEXEC, or F_DUP2FD_CLOEXEC commands to the fcntl(2) system
       call, or those created via the dup(2) system call, or those inherited
       by a child process created via fork(2)) reference the same lock, but a
       file descriptor obtained via a separate open(2) call on the same file
       will reference a different lock.  A lock is removed only on the last
       close(2) of the description, or when the lock is explicitly unlocked.


       Locks of both styles are compatible. A file that has been locked with
       one style of lock will be regarded as locked when creation of a lock of
       either style is attempted, and information about the lock will be
       provided via any of the F_GETLK, F_GETLK64, F_OFD_GETLK, or
       F_OFD_GETLK64 commands to this system call if that lock would conflict
       with an attempt to create the specified lock regardless of whether the
       specified lock is of the same style as the conflicting extant lock.
       Because ownership of OFD-style locks is scoped to the open description
       rather than the calling process, the l_pid field of a lock descriptor
       for any such lock will always be set to -1.


       When a shared lock is set on a segment of a file, other callers
       (regardless of whether in the same or different process and of whether
       referenced via the same open file) will be able to set shared locks on
       that segment or a portion of it. A POSIX-style shared lock prevents any
       other process from setting an exclusive lock on any portion of the
       protected area. A OFD-style shared lock prevents any caller (even
       callers in the same process) from setting an exclusive lock on any
       portion of the protected area, unless the caller makes the request
       against a file descriptor referencing the same open file against which
       the shared lock was created, in which case the lock will be downgraded
       to a shared lock with respect to the specified region. A request for a
       shared lock of either style will fail if the file descriptor was not
       opened with read access.

But this isn't part of Solaris or other illumos distributions.

[akolb1]

@ksedgwic It seems that for bolt fcntl locks are is fine.

[binarycrusader]

@ksedgwic the changes I submitted for review are only for fixing the build, they don't add support for any calls that aren't supported today.

[fazalmajid]

Here's more color on the status of implementing a proper flock in Illumos:
https://www.illumos.org/issues/3252
In any case, even if Illumos gets a real flock (not a broken one emulated on top of fcntl with its release-on-any-close semantics), that would not be a solution for Oracle flavors of Solaris.

[ksedgwic]

I've submitted a change for x/sys/unix which adds mmap, munmap and madvise
for solaris:

https://go-review.googlesource.com/#/c/13720/

Ken

On Wed, Aug 19, 2015 at 3:22 PM, Fazal Majid notifications@github.com
wrote:

Here's more color on the status of implementing a proper flock in Illumos:
https://www.illumos.org/issues/3252
In any case, even if Illumos gets a real flock (not a broken one emulated
on top of fcntl with its release-on-any-close semantics), that would not be
a solution for Oracle flavors of Solaris.

—
Reply to this email directly or view it on GitHub
#11113 (comment).

Ken Sedgwick
Bonsai Software, Inc.
http://www.bonsai.com/ken/
(510) 269-7334
ken@bonsai.com
Public Key: http://www.bonsai.com/ken/ken.asc
GPG Fingerprint: 851E 3B07 E586 0843 9434 5CC7 4033 3B9B 3F3F 9640

[akolb1]

@ksedgwic Your change will not make it to 1.5 - correct? Can you send me your Solaris patch?

[akolb1]

@ksedgwic Also I noticed that your change didn't have SYS_IOCTL for Solaris.

[akolb1]

@ksedgwic What should I do to apply golang/sys to standard golfing source tree? I tried copying files, but Solaris files are in unix package instead of syscall package.

[akolb1]

@ksedgwic Oh, I see. It isn't a patch to golfing itself but rather a GO package. I was thinking that this is part of the core golang.

[ianlancetaylor]

Fixed for x/sys/unix by #21410.