doc: mention "purego" build tag convention somewhere

[dsnet]

As the number of Go implementations continues to increase, the number of cases where the unsafe package is unlikely to work properly also rises. Currently, there is appengine, gopherjs, and possibly wasm where pointer arithmetic is not allowed.

Currently, protobuf and other packages special cases build tags for appengine and js and may need to add others in the near future. It does not scale very well to blacklist specific known Go implementations where unsafe does not work.

My proposal is to document safe as a community agreed upon tag meaning that unsafe should not be used. It is conceivable that this concept be extended to the compiler rejecting programs that use unsafe when the safe tag is set, but I'm currently more interested as a library owner in knowing whether to avoid unsafe in my own packages.

\cc @zombiezen @dneil @neelance @shurcooL

[dsnet]

The code history of protobuf seems to indicate that this very same concept was discussed but not pursued further. I'd like to push this more since I see this distinction in at least 2 packages I own.

golang/protobuf#154

[mdempsky]

I think standardizing a build tag to indicate whether package unsafe is available makes sense.

It is conceivable that this concept be extended to the compiler rejecting programs that use unsafe when the safe tag is set

I disagree. Currently, build tags are strictly a build-system concept. I'd argue the compiler should remain ignorant of them. cmd/compile already has a -u flag that prevents importing package unsafe, and the build system can arrange to pass -u as appropriate.

[dmitshur]

/cc @bradfitz who also ran into this with go4.org/reflectutil, and seemed to like "safe" at the time.

[bradfitz]

@shurcooL, I'm still fine with "safe", as long as it's defined (i.e. "code that doesn't import "unsafe").

But does it also mean no assembly?

Those are the sorts of things that should be clarified, if this is to be blessed somehow. (our own use, wiki page, etc)

[dmitshur]

@dsnet Can you clarify if your proposal is about documenting safe to have a very specific meaning and applied to all packages?

Or is it about documenting the fact that safe is a commonly used build tag for a given purpose, but individual projects still have get final say on the exact meaning of the safe build tag for their own needs?

[flimzy]

Would this proposal be codified in the standard library somehow, perhaps by adding a !safe build tag to the unsafe package? Or would it live purely in documentation?

[davecb]

In a previous life, we had to identify versions of libraries, and rapidly found out that that was too coarse a measure. We eventually attached a label via the linker to each entry point*, and could tell if, for example, a call to memcpy allowed overap or not. '

We also used it in migration work, to identify parts of programs that could not be supported on a different OS or hardware platform.

You arguably should consider labelling parts of the unsafe library with supported and unsupported by target OS, language or whatever, not the whole library if only one operation is unavailable.

--dave
[* a description of using per-entry-point labels for a different purposes is at https://leaflessca.wordpress.com/2017/02/12/dll-hell-and-avoiding-an-np-complete-problem/ ]

[andlabs]

Would this unsafe build tag also affect code that uses cgo? SWIG?

How would this build tag interact with the standard library, where both are used a lot? Does no unsafe mean no reflect as well?

What is the unsafe policy on nacl? is there anything in nacl that we could use for this?

@shurcooL it sounds like the latter at minimum, the former ideally.

[dsnet]

I propose that "safe" be soft signal that a library should have memory safety (i.e., makes no assumptions about how objects are laid out in memory, the architecture endianess, semantics regarding registers, etc).

Thus, the "safe" tag has the following properties:

• This is just a hint for library authors who want to write code that is highly portable. There is no logic in the compiler or the build tool to enforce this.
  • Thus, the standard library doesn't have to use it. Portability is achieve by either requiring a fork of the standard library (as is the case for gopherjs) or via a series of build tags as the mainline standard library does for various architectures.
• Use of reflect is allowed since it doesn't allow you to violate memory safety.
• Use of cgo is allowed. We already have a build tag for that, which is cgo. In practice, safe implies that cgo is not used since it is difficult to use cgo without pointers (for which you need unsafe).
• Use of assembly is forbidden. Any reasonable use of assembly makes assumption about how memory is laid out on the stack and/or heap.

Thus, appengine and gopherjs are example toolchains that would always set the safe tag.

[rsc]

Based on discussion with proposal-review:

• It seems reasonable for appengine, gopherjs, and wasm, all of which declare their own "restricted build" tag, to agree on a common one.
• It should mean no asm, no cgo, no unsafe. (It's impossible to use cgo without unsafe.)
• Nothing in the standard distribution would care; this is really about coordination between these other non-standard environments. Where do you propose to document this?
• A better name than "safe" would be nice. "purego"?

[neelance]

On WebAssembly: The wasm backend that I'm working on uses a linear memory, so unsafe is fully supported. It also has its own asm instructions, just like other architectures. Cgo is not supported (unless someone wants to do a crazy integration with emscripten).

[cznic]

A better name than "safe" would be nice. "purego"?

👍 for purego, already using it in my projects for some years.

[dsnet]

I support purego as well. Even if WebAssembly supports unsafe (which is great to hear!), there is always still the use case where someone wants to compile with a pure-Go version for a variety of reasons.

I don't have any great suggestions for where to document this, but perhaps the godoc for go/build?

[dmitshur]

I wanted to point out that in colloquial usage, I've seen "pure Go" most commonly refer to packages that don't use cgo (but can use unsafe, assembly). Seeing it mean "no unsafe and no assembly as well" would require some calibration. But maybe it's fine.

The math/big package contains some precedent on this: it defines a math_big_pure_go tag, which is being used as proposed here (no assembly, no unsafe, no cgo).

[rsc]

OK, purego it is.

[gopherbot]

Change https://go.dev/cl/561935 mentions this issue: crypto: use and test purego tag consistently

[aykevl]

I'm working on TinyGo, and combining these three concepts together seems like a bad idea to me (no assembly, no cgo, and no unsafe).
TinyGo:

• Supports unsafe, though it has a slightly different memory layout for some types. It should still be possible to write portable unsafe code in most cases (e.g. unsafe.String was a great addition for this).
• Supports CGo, though it has some features missing compared to the main Go toolchain. These missing features can probably be implemented when needed.
• Does not support Go assembly. I've tried it, and the only way I got it to work was one giant hack.

Furthermore, there is already a perfectly good tag for CGo support: the cgo build tag. I don't think we need a new build tag that also says something about CGo support.
I see there are some systems (like appengine) where unsafe is not allowed, I would suggest using a different build tag for that than the one that controls Go assembly support.

Right now we set the purego build tag by default to get crypto packages to work, but I don't really like it because it's not very clearly defined right now and I'd rather not limit things like unsafe. A build tag like noasm and a separate build tag like nounsafe (or whatever) would be much better in my opinion.