cmd/go: build: add -static flag

[kolyshkin]

This is a proposal to add -static flag to go build.

Producing a static build with go already requires a non-trivial amount of flags passed to go build, which on Linux currently amounts to something like:

-ldflags '-extldflags "-fno-PIC -static"' -buildmode pie -tags 'osusergo netgo static_build' [1]

...and this magic string keeps growing.

It would be awesome to encapsulate this sacred knowledge internally, exposing it via a new -static flag for go build and friends.

In addition to the above benefit of hiding the complexity, -static can also bring us more niceties, such as:

• automatic static build tag third-party software can rely upon. Currently using static_build tag seems like a de-facto standard, used a lot, but it has to be explicitly defined like in [1] above.
• providing --static flag to pkg-config invocations (those initiated by // #cgo pkg-config: lib lines in the source code). It will solve another issue (linking against a proper set of libraries for both static and dynamic link cases) for which a somewhat verbose workaround is currently required (for example, see ploop_link_static.go and ploop_link_dynamic.go).

See also

• cmd/go: #cgo pkg-config: add conditional --static flag to pkg-config #12058 (original proposal for adding conditional --static to pkg-config)
• os/user: LookupId panics on Linux+glibc static build #24787 (initial version of this very proposal)

[rsc]

This seems OK but the tag should just be "static" not "static_build" (it's a build tag already). And we should make os/user and net do the right thing - whatever that is - instead of having to hard-code their tags too.

Note that the assumption is that -static does not disable cgo; it just makes the cgo uses statically linked. FWIW, I don't know how well statically linked cgo works, but apparently well enough.

It sounds like maybe the os/user and net non-cgo restrictions only apply on Linux (more precisely, GNU/Linux, since this is a glibc restriction).

[rsc]

If anyone wants to work on this for Go 1.12, help is appreciated.

[amenzhinsky]

I get the idea of combining netgo and usergo tags that looks a bit magical, but what's the reason of using the pie builmode since it forces the toolchain to use an external linker (gcc, clang) and many build systems, like alpine docker images, come without one.

[kolyshkin]

This seems OK but the tag should just be "static" not "static_build" (it's a build tag already).

static_build was proposed as it is used in many packages (seems like a de facto standard... I'm too lazy to provide examples but please let me know if you need any).

Still, I agree it's better to have static as it is simple and straightforward (and hopefully won't collide with anything). Proposal changed.

And we should make os/user and net do the right thing - whatever that is

Right (and current netgo and osusergo flags might stay for backward compatibility, as they might also be used to choose one implementation over another, with no regard to static/dynamic linking).

Note that the assumption is that -static does not disable cgo; it just makes the cgo uses statically linked.

Great.

many build systems, like alpine docker images, come without one

I'm afraid I was overly brief describing the idea, let me emphasize.

The idea (specifically, the first part of it) is to hide the burden of choosing the build flags required in order to obtain a working static binary (on a best effort/knowledge basis). The above example is particularly valid for Linux/glibc. In case of Linux/musl (i.e. Alpine), for example, there is probably no need to use osusergo or netgo tags.

[iamoryanmoshe]

Any chance you elaborate on what a static build does differently than regular build?

I would like to work on this proposal but I need further clarification.

[kolyshkin]

Any chance you elaborate on what a static build does differently than regular build?

It should

1. Set the needed build options (this is platform-dependent, Linux/glibc example is above).
2. Set the needed build tags (for Linux/glibc this currently amounts to osusergo netgo).
3. Set the static build tag.
4. Add the --static flag to pkg-config invocations (those triggered by // #cgo pkg-config: <lib> ... lines in the source code)

[vikramcse]

Can I work on this as my first contribution to go?

[ianlancetaylor]

@vikramcse The main requirements here are access to a range of different systems in order to test the work, and the knowledge required to write a good test for whether the program is statically linked. The actual patch is probably not too difficult, I hope. If you still want to give this a try, please go ahead.

[vikramcse]

Thanks @ianlancetaylor, I should start from simpler issue

[tmm1]

We build golang binaries across a variety of platforms, preferring static builds wherever possible. Here are the flags we're currently using:

windows: -tags netgo -ldflags '-H=windowsgui -extldflags "-static"'
linux/bsd: -tags netgo -ldflags '-extldflags "-static"'
macos: -ldflags '-s -extldflags "-sectcreate __TEXT __info_plist Info.plist"'
android: -ldflags -s

On macos and android, we need to be able to pull in system frameworks and libraries so we opted not to build static binaries.

[kolyshkin]

@tmm1 thanks for the info! Just noticed you should also use osusergo tag, at least for Linux.

[noerw]

To get a statically linked PIE executable I needed the following invocation (I'm mentioning this here, as I found this documented nowhere on the net, scraped it together from this thread, and this stackoverflow answer)

CGO_ENABLED=1 go build -buildmode=pie -tags 'osusergo,netgo,static,' -ldflags '-linkmode=external -extldflags "-static-pie"' .

The proposed -static flag would need to handle this case, when -buildmode=pie is also present.

[fgm]

Since this has been dragging on for a couple of years already, would it not be a better solution to include the instructions for static builds in the release notes for each version, since they keep on changing over time ? That would obviate the need for an actual code change.

[mvdan]

I think we could update the list at the top of this thread continuously for that purpose. Also, this doesn't make go build -static unnecessary, because you don't want everyone to have to carry a long command and keep it up to date.

@MaoJianwei we understand Go does support static builds - this thread is about making them easier to achieve.

[fweimer-rh]

@MaoJianwei we understand Go does support static builds - this thread is about making them easier to achieve.

And hopefully less confusing, too: Often, static linking is used to make it easier to move binaries between different systems (of the same architecture), but that doesn't really work with glibc: there, dynamically linked binaries are compatible with glibc updates, but any glibc change (including certain minor version updates) may break statically linked binaries that use NSS or other forms of dlopen. So static linking for portability only works if CGO is disabled on the Go side and glibc does not enter the picture.

[fgm]

@mvdan that's a start. The bonus if doing it in release notes is that each version could have relevant info, while this thread would likely be evergreen, only for the latest/next (which one ?) version. Or maybe a table per version here ?

[MaoJianwei]

Thanks, @fgm @mvdan @fweimer-rh .
I see. The commands and needs for statically linked compilation may vary between golang versions, so it is difficult to do the work to provide a convenient --static cmd parameter for go build.

[tv42]

@MaoJianwei go build -static is part of Go and would get updated to do the right thing before release -- writing a test for it should be easy enough.

[tv42]

Let me rephrase: The proposed go build -static would be part of Go, and would be updated to do the right thing.