Compilation errors in illumos

[gdamore]

It would appear that under cgo 1.4, it doesn't work on illumos systems:

../../../../boltdb/bolt/bolt_unix.go:26: undefined: syscall.Flock
../../../../boltdb/bolt/bolt_unix.go:26: undefined: syscall.LOCK_EX
../../../../boltdb/bolt/bolt_unix.go:26: undefined: syscall.LOCK_NB
../../../../boltdb/bolt/bolt_unix.go:40: undefined: syscall.Flock
../../../../boltdb/bolt/bolt_unix.go:40: undefined: syscall.LOCK_UN
../../../../boltdb/bolt/bolt_unix.go:55: undefined: syscall.Mmap
../../../../boltdb/bolt/bolt_unix.go:75: undefined: syscall.Munmap

It looks like you're using a lot of Unix system calls that are simply not actually portable. Sadly, these calls do exist in the C library under the hood, but apparently cgo doesn't have them exposed for Solaris.

This prevents me from being able to use an otherwise very promising key value store in my project.

[benbjohnson]

@gdamore There's an open issue for using flock() on Solaris. I didn't realize that the mmap() and munmap() calls were not available though. Bolt will support Solaris-based distributions but I haven't had the time so far to implement the changes.

[gdamore]

I can try to code it up and send you a patch if you like.

Sent from my iPhone

On Feb 17, 2015, at 5:10 AM, Ben Johnson notifications@github.com wrote:

@gdamore There's an open issue for using flock() on Solaris. I didn't realize that the mmap() and munmap() calls were not available though. Bolt will support Solaris-based distributions but I haven't had the time so far to implement the changes.

—
Reply to this email directly or view it on GitHub.

[benbjohnson]

Yeah, that'd be great. I'm happy to test the changes on Linux & Mac OS X if you get them working on Illumos. Hopefully those changes should translate over to SmartOS too.

[gdamore]

On Feb 17, 2015, at 12:03 PM, Ben Johnson notifications@github.com wrote:

Yeah, that'd be great. I'm happy to test the changes on Linux & Mac OS X if you get them working on Illumos. Hopefully those changes should translate over to SmartOS too.

They will. ;-)

The only thing I’m not sure about is the Mmap stuff… I don’t know why that is missing from syscall..

- Garrett

—
Reply to this email directly or view it on GitHub #305 (comment).

[benbjohnson]

I'm looking at this issue but having problems getting a Solaris instance up and running. The Vagrant boxes out there use VirtualBox (I'm on VMWare) and DigitalOcean doesn't support Solaris distros. :-/

I may have to build a Vagrant box from scratch but first I need to figure out how to do that. :)

[chrislusf]

Waiting on this fix.

[ghost]

Hi Ben,

If I were to supply you with a SmartOS VM (illumos based distro) in our cloud would that help with this issue? We're keen to use this on SmartOS...

Cheers,

Alasdair

[benbjohnson]

@alasdairrr Yeah, that'd be great. I burned about 4 hours trying to get it running in Vagrant and I haven't had the time to invest to try again. I don't think the fix itself is that hard so a VM would be a huge help.

[gdamore]

I’m super sorry I’ve not had time to work on this. I switched to goleveldb for my immediate needs, and haven’t had cycles to go back and fix this. But doing so is important to making some applications, such as Consul.io, work on illumos. Its been on my TODO list for a while now, but I’m completely swamped.

- Garrett

On May 26, 2015, at 8:27 AM, Ben Johnson notifications@github.com wrote:

@alasdairrr https://github.com/alasdairrr Yeah, that'd be great. I burned about 4 hours trying to get it running in Vagrant and I haven't had the time to invest to try again. I don't think the fix itself is that hard so a VM would be a huge help.

—
Reply to this email directly or view it on GitHub #305 (comment).

[benbjohnson]

No problem. Hopefully it'll be pretty straightforward once I get a VM up and running.

[ghost]

Hi Ben,

I've created the VM, bolt.cloud.ec - could you let me know your ssh public key? My email is al at everycity dot co dot uk

Cheers!

[caquino]

Hi, any updates on this issue? I'm also interested in running consul on openindiana/illumos.
Thanks

[benbjohnson]

@caquino I've been slammed for time lately but I'm planning on working on this after GopherCon (July 7 - 10th).

[pannon]

@caquino would be great to have this fixed as Consul.io fails to build on SmartOS/Illumos. If you still need a VM to test let me know.

[akolb1]

golang/go#11113 discusses mmap fixes for golang
For locks I have a patch that works:

diff --git a/bolt_solaris.go b/bolt_solaris.go
new file mode 100644
index 0000000..89c99df
--- /dev/null
+++ b/bolt_solaris.go
@@ -0,0 +1,112 @@
+// +build solaris
+
+// Solaris system call interface
+
+package bolt
+
+import (
+       "fmt"
+       "os"
+       "syscall"
+       "time"
+       "unsafe"
+)
+
+// flock acquires an advisory lock on a file descriptor.
+func flock(f *os.File, exclusive bool, timeout time.Duration) error {
+       var t time.Time
+       for {
+               // If we're beyond our timeout then return an error.
+               // This can only occur after we've attempted a flock once.
+               if t.IsZero() {
+                       t = time.Now()
+               } else if timeout > 0 && time.Since(t) > timeout {
+                       return ErrTimeout
+               }
+               var lock syscall.Flock_t
+               lock.Start = 0
+               lock.Len = 0
+               lock.Pid = 0
+               lock.Whence = 0
+               lock.Pid = 0
+               if exclusive {
+                       lock.Type = syscall.F_WRLCK
+               } else {
+                       lock.Type = syscall.F_RDLCK
+               }
+               err := syscall.FcntlFlock(f.Fd(), syscall.F_SETLK, &lock)
+               if err == nil {
+                       return nil
+               } else if err != syscall.EAGAIN {
+                       return err
+               }
+
+               // Wait for a bit and try again.
+               time.Sleep(50 * time.Millisecond)
+       }
+}
+
+// funlock releases an advisory lock on a file descriptor.
+func funlock(f *os.File) error {
+       var lock syscall.Flock_t
+       lock.Start = 0
+       lock.Len = 0
+       lock.Type = syscall.F_UNLCK
+       lock.Whence = 0
+       return syscall.FcntlFlock(uintptr(f.Fd()), syscall.F_SETLK, &lock)
+}
+
+// mmap memory maps a DB's data file.
+func mmap(db *DB, sz int) error {
+       // Truncate and fsync to ensure file size metadata is flushed.
+       // https://github.com/boltdb/bolt/issues/284
+       if !db.NoGrowSync && !db.readOnly {
+               if err := db.file.Truncate(int64(sz)); err != nil {
+                       return fmt.Errorf("file resize error: %s", err)
+               }
+               if err := db.file.Sync(); err != nil {
+                       return fmt.Errorf("file sync error: %s", err)
+               }
+       }
+
+       // Map the data file to memory.
+       b, err := syscall.Mmap(int(db.file.Fd()), 0, sz, syscall.PROT_READ, syscall.MAP_SHARED)
+       if err != nil {
+               return err
+       }
+
+       // Advise the kernel that the mmap is accessed randomly.
+       if err := madvise(b, syscall.MADV_RANDOM); err != nil {
+               return fmt.Errorf("madvise: %s", err)
+       }
+
+       // Save the original byte slice and convert to a byte array pointer.
+       db.dataref = b
+       db.data = (*[maxMapSize]byte)(unsafe.Pointer(&b[0]))
+       db.datasz = sz
+       return nil
+}
+
+// munmap unmaps a DB's data file from memory.
+func munmap(db *DB) error {
+       // Ignore the unmap if we have no mapped data.
+       if db.dataref == nil {
+               return nil
+       }
+
+       // Unmap using the original byte slice.
+       err := syscall.Munmap(db.dataref)
+       db.dataref = nil
+       db.data = nil
+       db.datasz = 0
+       return err
+}
+
+// NOTE: This function is copied from stdlib because it is not available on darwin.
+func madvise(b []byte, advice int) (err error) {
+       _, _, e1 := syscall.Syscall(syscall.SYS_MADVISE, uintptr(unsafe.Pointer(&b[0])), uintptr(len(b)), uintptr(advice))
+       if e1 != 0 {
+               err = e1
+       }
+       return
+}
diff --git a/bolt_unix.go b/bolt_unix.go
index 17ca318..6eef6b2 100644
--- a/bolt_unix.go
+++ b/bolt_unix.go
@@ -1,4 +1,4 @@
-// +build !windows,!plan9
+// +build !windows,!plan9,!solaris

 package bolt

[akolb1]

The patch requires change in the test because the lock test fails on Solaris. This is caused by the fact that fcntl-based locking doesn't protect within the same pid.