x/sys: add unsafe mmap

[piotr-sneller]

The current implementation of the Mmap wrappers for the Unix-derived OSes does not dispatch directly to the relevant syscall but relies on the mmapper layer. The mmapper adds address range validation and returns EINVAL for the ranges not matching the content of the mapping registry. While it might be arguably correct for most of the use cases, it also prevents partial Munmaps and similar valid parts of the pretty rich mmap semantics. I suggest that the existing implementation should be left as-is, but undisturbed access to the underlying OS primitives should also be provided. The scope of the extension would be minimal, as the low-level part already is there -- the package should just export more entry points. The naming details are irrelevant to me, so I leave them unspecified: I will be equally happy with MmapUnsafe as with MmapRaw.

The only suggestion I'd make is the return type: the functions should operate with uintptr, as their WinAPI VirtualAlloc() counterparts do. Wrapping the uintptr in a slice just to unwrap it in the follow-up step to recover the uintptr adds no value. I'll make the slice when I'm done.

[ianlancetaylor]

Can you write down the exact new API that you propose adding? Thanks.

[ncruces]

I need this in github.com/ncruces/go-sqlite3.

I'm currently go:linknaming mmap (I know, I'm sorry!) because I need unix.MAP_FIXED (which is unusable with unix.Mmap as is). This is understandably discouraged, and on a path to being curtailed in #67401.

I know my package is pretty much irrelevant, but exposing the autogenerated bits that allow mmap to be called from a bunch of different Unixes would help. For example, I have no chance to test z/OS, but IBM is incentivized to add a portable version to x/sys/unix.

Hence, answering @ianlancetaylor, I'd like to propose x/sys/unix exports:

func MmapPtr(addr uintptr, length uintptr, prot int, flag int, fd int, pos int64) (ret uintptr, err error)
func MunmapPtr(addr uintptr, length uintptr) (err error)

And maybe, for the platforms that support it:

func MremapPtr(oldaddr uintptr, oldsize uintptr, newaddr uintptr, newsize uintptr, flags int) (ret uintptr, err error)

These match the portability layer already in x/sys/unix.

[database64128]

Related discussion: #58625

[rsc]

It seems like the pointers should at least be pointers. What about:

func MmapPtr(addr unsafe.Pointer, length uintptr, prot int, flag int, fd int, pos int64) (ret unsafe.Pointer, err error)
func MunmapPtr(addr unsafe.Pointer, length uintptr) (err error)
func MremapPtr(oldaddr unsafe.Pointer, oldsize uintptr, newaddr unsafe.Pointer, newsize uintptr, flags int) (ret unsafe.Pointer, err error)

?

[ncruces]

I don't know that I'm qualified to answer that.

I'll just say this is not Go managed memory, and as said above x/sys/windows.VirtualAlloc uses uintptr.

That was my reasoning, but I'm happy to be overridden by someone more knowledgeable. 🙂

[aclements]

In general, the unix syscall packages try to be as type-safe as possible, so using unsafe.Pointer would force code to talk about these pointers as pointers, and also to use unsafe to get at these unsafe APIs. I don't think it's a strong requirement, but I think it's nice to have. The Windows syscall packages tend to follow the Windows C APIs more literally and are much more lax about safely wrapping them, so I wouldn't put much weight on VirtualAlloc's signature.

[ncruces]

If this is a likely approve, I don't minding submitting a CL myself.
Whatever is easiest for the Go team.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group