Makefile build options do not generate static binaries #48

atc0005 · 2020-10-03T08:33:40Z

Same here as with atc0005/check-mail#94.

atc0005 · 2020-10-03T10:55:14Z

Specifying CGO_ENABLED=0 produced failures:

WARN[0000] github.com/atc0005/mysql2sqlite/internal/dbqs.VerifyDBConn: attempt 1 of 6 to verify database connection failed: Binary was compiled with 'CGO_ENABLED=0', go-sqlite3 requires cgo to work. This is a stub

atc0005 · 2020-10-03T11:04:54Z

ubuntu@mysql2sqlite-testing:~/go-mysql2sqlite$ git diff Makefile
diff --git a/Makefile b/Makefile
index bcc36ff..8723fae 100644
--- a/Makefile
+++ b/Makefile
@@ -37,7 +37,7 @@ VERSION                               = $(shell git describe --always --long --dirty)

 # The default `go build` process embeds debugging information. Building
 # without that debugging information reduces the binary size by around 28%.
-BUILDCMD                               =       go build -mod=vendor -a -ldflags="-s -w -X $(VERSION_VAR_PKG).Version=$(VERSION)"
+BUILDCMD                               =       go build -mod=vendor -tags 'osusergo,netgo' -a -ldflags "-extldflags '-static' -s -w -X $(VERSION_VAR_PKG).version=$(VERSION)"
 GOCLEANCMD                             =       go clean -mod=vendor ./...
 GITCLEANCMD                            =       git clean -xfd
 CHECKSUMCMD                            =       sha256sum -b

ubuntu@mysql2sqlite-testing:~/go-mysql2sqlite$ make linux
Building release assets for linux ...
Building mysql2sqlite 386 binaries
Building mysql2sqlite amd64 binaries
# github.com/atc0005/mysql2sqlite/cmd/mysql2sqlite
/tmp/go-link-753298436/000010.o: In function `unixDlOpen':
/home/ubuntu/go-mysql2sqlite/vendor/github.com/mattn/go-sqlite3/sqlite3-binding.c:39981: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
Generating mysql2sqlite checksum files
Building check_mysql2sqlite 386 binaries
Building check_mysql2sqlite amd64 binaries
# github.com/atc0005/mysql2sqlite/cmd/check_mysql2sqlite
/tmp/go-link-293828066/000010.o: In function `unixDlOpen':
/home/ubuntu/go-mysql2sqlite/vendor/github.com/mattn/go-sqlite3/sqlite3-binding.c:39981: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
Generating check_mysql2sqlite checksum files
Completed build tasks for linux

ubuntu@mysql2sqlite-testing:~/go-mysql2sqlite$ file release_assets/mysql2sqlite/mysql2sqlite-7a77dab-dirty-linux-amd64
release_assets/mysql2sqlite/mysql2sqlite-7a77dab-dirty-linux-amd64: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, BuildID[sha1]=d00f44845d74d0677794ae875b7920d3f626bb78, stripped

Running the application produces the desired results, at least on the same distro/arch that generated the binary.

atc0005 · 2020-10-03T11:05:47Z

This page (https://stackoverflow.com/questions/7526255/unknown-reference-to-dlopen-in-dlopen/7532277) has this to say:

In my system, gcc test.c -ldl -static is enough to make the application run, but

gcc -static -ldl -lc

should fix the problem in your system.

So, add -lc then?

atc0005 · 2020-10-03T11:18:19Z

That didn't seem to make a difference.

ubuntu@mysql2sqlite-testing:~/go-mysql2sqlite$ git diff Makefile
diff --git a/Makefile b/Makefile
index bcc36ff..18b4bd2 100644
--- a/Makefile
+++ b/Makefile
@@ -37,7 +37,7 @@ VERSION                               = $(shell git describe --always --long --dirty)

 # The default `go build` process embeds debugging information. Building
 # without that debugging information reduces the binary size by around 28%.
-BUILDCMD                               =       go build -mod=vendor -a -ldflags="-s -w -X $(VERSION_VAR_PKG).Version=$(VERSION)"
+BUILDCMD                               =       go build -mod=vendor -tags 'osusergo,netgo' -a -ldflags "-extldflags '-static -ldl -lc' -s -w -X $(VERSION_VAR_PKG).version=$(VERSION)"
 GOCLEANCMD                             =       go clean -mod=vendor ./...
 GITCLEANCMD                            =       git clean -xfd
 CHECKSUMCMD                            =       sha256sum -b

ubuntu@mysql2sqlite-testing:~/go-mysql2sqlite$ make clean && make linux
Removing object files and cached files ...
Removing any existing release assets
Building release assets for linux ...
Building mysql2sqlite 386 binaries
Building mysql2sqlite amd64 binaries
# github.com/atc0005/mysql2sqlite/cmd/mysql2sqlite
/tmp/go-link-263648963/000010.o: In function `unixDlOpen':
/home/ubuntu/go-mysql2sqlite/vendor/github.com/mattn/go-sqlite3/sqlite3-binding.c:39981: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
Generating mysql2sqlite checksum files
Building check_mysql2sqlite 386 binaries
Building check_mysql2sqlite amd64 binaries
# github.com/atc0005/mysql2sqlite/cmd/check_mysql2sqlite
/tmp/go-link-969147616/000010.o: In function `unixDlOpen':
/home/ubuntu/go-mysql2sqlite/vendor/github.com/mattn/go-sqlite3/sqlite3-binding.c:39981: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
Generating check_mysql2sqlite checksum files
Completed build tasks for linux

atc0005 · 2020-10-03T11:24:28Z

Mixed findings when searching further. It is unclear to me whether static binaries are reliable where CGO is concerned.

atc0005 · 2020-10-03T11:58:23Z

The more I think about it, the more I think I should hold off with attempts to truly generate static binaries for this project. This may change once I put some effort into #18.

For now, I may try what appears to be a "safe" set of Go build tags:

-tags 'osusergo netgo'

atc0005 · 2020-10-03T12:14:13Z

The Gitea project produces static binaries. Probably a good idea to see how they're pulling off stable builds.

https://github.com/go-gitea/gitea/blob/v1.12.5/Makefile

atc0005 · 2020-10-03T12:19:48Z

.PHONY: release-linux
release-linux: | $(DIST_DIRS)
	@hash xgo > /dev/null 2>&1; if [ $$? -ne 0 ]; then \
		GO111MODULE=off $(GO) get -u src.techknowlogick.com/xgo; \
	fi
	CGO_CFLAGS="$(CGO_CFLAGS)" GO111MODULE=off xgo -go $(XGO_VERSION) -dest $(DIST)/binaries -tags 'netgo osusergo $(TAGS)' -ldflags '-linkmode external -extldflags "-static" $(LDFLAGS)' -targets 'linux/amd64,linux/386,linux/arm-5,linux/arm-6,linux/arm64,linux/mips64le,linux/mips,linux/mipsle' -out gitea-$(VERSION) .
ifeq ($(CI),drone)
	cp /build/* $(DIST)/binaries
endif

Looks like they're using a similar collection of flags/settings that I was. Need to try building their codebase to review the output.

atc0005 · 2020-10-03T12:24:03Z

Need to try building their codebase to review the output.

https://github.com/techknowlogick/xgo

My solution to the challenge of cross compiling Go code with embedded C/C++ snippets (i.e. CGO_ENABLED=1) is based on the concept of lightweight Linux containers. All the necessary Go tool-chains, C cross compilers and platform headers/libraries have been assembled into a single Docker container, which can then be called as if a single command to compile a Go package to various platforms and architectures.

Ah. Talk about a rabbit hole.

atc0005 · 2020-10-04T09:43:21Z

If you take this literally:

/tmp/go-link-383478856/000010.o: In function `unixDlOpen':
/home/ubuntu/go-mysql2sqlite/vendor/github.com/mattn/go-sqlite3/sqlite3-binding.c:39981: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

this seems to suggest that if I build on Ubuntu 18.04 and glibc is at 2.23.0, then that version of glibc is needed?

ubuntu@mysql2sqlite-testing:~/go-mysql2sqlite$ dpkg -l | grep -E '^ii\s+libc[0-9]'
ii  libc6:amd64                      2.23-0ubuntu11.2                           amd64        GNU C Library: Shared libraries
ii  libc6-dev:amd64                  2.23-0ubuntu11.2                           amd64        GNU C Library: Development Libraries and Header Files

That exact version, or that version and newer?

atc0005 · 2020-10-04T10:40:48Z

Light reading seems to suggest a hard requirement with "strange" behavior if the versions don't line up.

Then, there is the licensing aspect. It sounds like if I distribute compiled binaries with dynamic linkage there is no trouble, but if static builds are provided which pull in glibc code then the LGPL comes into play. This still isn't very clear to me.

EDIT:

The first solution that may come to mind is to just link the glibc statically. However, that rarely works, as various warnings will tell you. The glibc just doesn’t like that. Also, it might create certain legal problems. Some people, the FSF in particular, interpret static linking as copying, and thus any binaries linked statically with the glibc that we distribute would have to abide by the GPL.

https://dominik.honnef.co/posts/2015/06/statically_compiled_go_programs__always__even_with_cgo__using_musl/

atc0005 · 2020-10-04T11:21:24Z

https://bugzilla.redhat.com/show_bug.cgi?id=111298#c2

Statically linked programs which are not self-contained and depend
on the installed glibc (be it because of using NSS or iconv (which
both dlopen glibc internally), using dlopen directly or even using
locale support) are really the least portable thing you can build.

and

So, the only supported way of using NSS/iconv/dlopen in statically
linked programs is if they are compiled/linked against the same glibc
as they are run with.

atc0005 · 2020-10-04T11:51:50Z

A decent bit of reading later and it appears that I'm left with a few options:

Spend time on Review other potential replacements for mattn/go-sqlite3 package #18 to find a SQLite package that does not depend on CGO
- this package would need to be promising, well maintained, etc.
Setup multiple LXD containers to provide architecture-specific build environments for musl libc
- e.g., 32-bit, 64-bit, and so on
Use xgo + multiple Docker containers (one per build) to perform builds
- upside: clean environment per build, easily reproducible
- upside: Gitea project uses this approach, so there would be good starter material to work from
- downside: complicated vs "go build" or "make all" from a WSLv1 environment
Skip binary builds for this project
- skip them going forward
- update docs to note caveats (might be needed anyway)
- remove binary builds provided with v0.1.0 release (might also be needed anyway)

atc0005 · 2020-10-04T13:12:19Z

Giving xgo a try by way of building Gitea from source. Once you crawl down the rabbit hole, you end up at this xgo Dockerfile:

that is used to generate this Docker image:

ubuntu@ubuntu-1804-virtual-machine:~/Desktop/gitea$ docker image ls | grep -E 'SIZE|xgo'
REPOSITORY           TAG                 IMAGE ID            CREATED             SIZE
techknowlogick/xgo   go-1.14.x           be60bf7e5281        3 weeks ago         6.41GB

When running make release-linux, this is the output:

ubuntu@ubuntu-1804-virtual-machine:~/Desktop/gitea$ make release-linux
mkdir -p dist/binaries dist/release
CGO_CFLAGS="-g -O2 -DSQLITE_MAX_VARIABLE_NUMBER=32766" GO111MODULE=off xgo -go go-1.14.x -dest dist/binaries -tags 'netgo osusergo ' -ldflags '-linkmode external -extldflags "-static"  -X "main.MakeVersion=GNU Make 4.1" -X "main.Version=1.12.5" -X "main.Tags="' -targets 'linux/amd64,linux/386,linux/arm-5,linux/arm-6,linux/arm64,linux/mips64le,linux/mips,linux/mipsle' -out gitea-master .
Checking docker installation...
Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:02:36 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:01:06 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Checking for required docker image techknowlogick/xgo:go-1.14.x... not found!
Pulling techknowlogick/xgo:go-1.14.x from docker registry...
go-1.14.x: Pulling from techknowlogick/xgo
f08d8e2a3ba1: Pull complete 
3baa9cb2483b: Pull complete 
94e5ff4c0b15: Pull complete 
1860925334f9: Pull complete 
77565c898953: Pull complete 
8410d8790e2a: Pull complete 
c551134d4a0f: Pull complete 
ac8e2d6a71a2: Pull complete 
fd6e8b4c06a4: Pull complete 
e15985d3596f: Pull complete 
bf419048f31c: Pull complete 
67de5729275b: Pull complete 
0ec4681ddfb5: Pull complete 
7ac18eaea482: Pull complete 
af2c9cce3263: Pull complete 
0dd68d60a364: Pull complete 
c14d186e0af1: Pull complete 
a7b49aa545e6: Pull complete 
b5e48bc7f37f: Pull complete 
bf4677d2f250: Pull complete 
Digest: sha256:b607bbd3c3def22bfd5272bad1254650dac83c572bc28bfcbace994bf149d1c8
Status: Downloaded newer image for techknowlogick/xgo:go-1.14.x
docker.io/techknowlogick/xgo:go-1.14.x
Cross compiling ....
Enabled Go module support
Using vendored Go module dependencies
Building /source/go.mod...
Compiling for linux/amd64...
# code.gitea.io/gitea
loadinternal: cannot find runtime/cgo
Compiling for linux/386...
# code.gitea.io/gitea
loadinternal: cannot find runtime/cgo
Bootstrapping linux/arm-5...
Compiling for linux/arm-5...

it continues from there, that's as far as it has gotten as of this writing.

atc0005 · 2020-10-04T13:28:36Z

Results of the build:

ubuntu@ubuntu-1804-virtual-machine:~/Desktop/gitea$ tree dist/
dist/
├── binaries
│   ├── gitea-master-linux-386
│   ├── gitea-master-linux-amd64
│   ├── gitea-master-linux-arm-5
│   ├── gitea-master-linux-arm-6
│   ├── gitea-master-linux-arm64
│   ├── gitea-master-linux-mips
│   ├── gitea-master-linux-mips64le
│   └── gitea-master-linux-mipsle
└── release

2 directories, 8 files
ubuntu@ubuntu-1804-virtual-machine:~/Desktop/gitea$ file dist/binaries/*
dist/binaries/gitea-master-linux-386:      ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, BuildID[sha1]=2107c37e63227068000209cfa7a6fed8de39c357, not stripped
dist/binaries/gitea-master-linux-amd64:    ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.2.0, BuildID[sha1]=2e7789d9ed84e5c961dfdddd37c8a880cd89ffff, not stripped
dist/binaries/gitea-master-linux-arm-5:    ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, for GNU/Linux 3.2.0, BuildID[sha1]=08514b5051fa57f9e617c5a3af87734c638780fb, not stripped
dist/binaries/gitea-master-linux-arm-6:    ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, for GNU/Linux 3.2.0, BuildID[sha1]=b220cd11cbfcbe2338de3d175d0badb86dfce1ee, not stripped
dist/binaries/gitea-master-linux-arm64:    ELF 64-bit LSB executable, ARM aarch64, version 1 (GNU/Linux), statically linked, for GNU/Linux 3.7.0, BuildID[sha1]=4b5a87688c2e4bf802fd83f489acf2be72dbe8cb, not stripped
dist/binaries/gitea-master-linux-mips:     ELF 32-bit MSB executable, MIPS, MIPS32 rel2 version 1 (SYSV), statically linked, for GNU/Linux 3.2.0, BuildID[sha1]=386cdbd0c818ec22a9f25a2afc59cb49923678d1, not stripped
dist/binaries/gitea-master-linux-mips64le: ELF 64-bit LSB executable, MIPS, MIPS64 rel2 version 1 (SYSV), statically linked, BuildID[sha1]=6a16d7cd4610f4add7c31ce768097c238255909a, for GNU/Linux 3.2.0, not stripped
dist/binaries/gitea-master-linux-mipsle:   ELF 32-bit LSB executable, MIPS, MIPS32 rel2 version 1 (SYSV), statically linked, for GNU/Linux 3.2.0, BuildID[sha1]=d1b5ec4af35163b9c7e8596971902eeb10c4af9c, not stripped

atc0005 · 2020-10-04T13:31:21Z

So, xgo doesn't use a container per architecture, (afaict), it uses one large container with lots of cross-compiling tools within to handle the various architectures.

The end result is static binaries based off of glibc. That still leaves the licensing question in the air: what is the final license for static binaries generated from Go code + glibc?

atc0005 · 2020-10-04T13:39:57Z

https://blog.filippo.io/easy-windows-and-linux-cross-compilers-for-macos/

Maybe most importantly, it's based on the musl C standard library. This means that the binaries will only run on a musl-based system, like Alpine. However, if you build them as static binaries by passing -static as a LDFLAG they will run anywhere, including in scratch Docker containers. musl is specifically engineered to support fully static binaries, which is not recommended with glibc.

atc0005 · 2020-10-04T13:42:30Z

zeromq/libzmq#2849 (comment)

You cannot statically link to glibc - it uses runtime plugins for things like DNS resolution.
Also it's licensed under the LGPL, which means you can do dynamic linking, but static linking means your program's source code also has to be released under the LGPL.

Besides that, it looks like a problem with Golang, by searching for that error on google this is the second result:

golang/go#21421

atc0005 · 2020-10-04T14:04:24Z

Results of the build:

Results from trying to run the x64 build:

ubuntu@ubuntu-1804-virtual-machine:~/Desktop/gitea$ ./dist/binaries/gitea-master-linux-amd64 --help
Trace/breakpoint trap (core dumped)

ubuntu@ubuntu-1804-virtual-machine:~/Desktop/gitea$ strace ./dist/binaries/gitea-master-linux-amd64
execve("./dist/binaries/gitea-master-linux-amd64", ["./dist/binaries/gitea-master-lin"...], 0x7ffcbf337170 /* 59 vars */) = 0
brk(NULL)                               = 0x4e77000
brk(0x4e781c0)                          = 0x4e781c0
arch_prctl(ARCH_SET_FS, 0x4e77880)      = 0
uname({sysname="Linux", nodename="ubuntu-1804-virtual-machine", ...}) = 0
readlink("/proc/self/exe", "/home/ubuntu/Desktop/gitea/dist/"..., 4096) = 65
brk(0x4e991c0)                          = 0x4e991c0
brk(0x4e9a000)                          = 0x4e9a000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
arch_prctl(ARCH_SET_FS, 0x3d89d10)      = 0
--- SIGTRAP {si_signo=SIGTRAP, si_code=SI_KERNEL} ---
+++ killed by SIGTRAP (core dumped) +++
Trace/breakpoint trap (core dumped)

I think I ran into that yesterday when testing. Something about the -linkmode external setting, though in this case the build environment is within a container and appears to work well for the Gitea team, so perhaps I'm still doing something wrong. Maybe make release-linux is not sufficient to fully prep the build environment.

atc0005 · 2020-10-04T14:43:02Z

I opened an issue in the Gitea GH repo asking for their feedback. Maybe something about their build process (ignoring the error that I received) generates binaries that don't link against the GNU C Library.

atc0005 · 2020-10-04T15:10:40Z

https://github.com/zgoat/goatcounter#building-from-source

Or to build a statically linked binary:

$ go build -ldflags="-X main.version=$(git log -n1 --format='%h_%cI')" \
    -tags osusergo,netgo,sqlite_omit_load_extension \
    -ldflags='-extldflags=-static' \
    ./cmd/goatcounter

atc0005 · 2020-10-04T21:02:52Z

https://www.reddit.com/r/golang/comments/8m4xrh/do_linux_golang_binaries_depend_on_libc/

As far as I can tell, the v0.2.5 release provided binaries that are *not* statically linked against glibc and thus would not be held to LGPL license requirements. AFAICT. Update Makefile to disable use of cgo for binary generation in an effort to avoid potential inclusion in the future. This is likely *not* the last time I'll need to update the Makefile build options for static binary generation, but hopefully this will do for now. refs #94 refs atc0005/mysql2sqlite#48

atc0005 · 2020-10-07T10:59:05Z

https://github.com/mattermost/mattermost-load-test-ng/blob/master/Makefile

atc0005 · 2020-10-07T11:25:52Z

I tried enabling cgo explicitly by providing CGO_ENABLED=1 for each build command, but this produced odd results for Windows:

Building release assets for windows ...
Building mysql2sqlite 386 binaries
# runtime/cgo
gcc: error: unrecognized command line option ‘-mthreads’; did you mean ‘-pthread’?
Building check_mysql2sqlite 386 binaries
# runtime/cgo
gcc: error: unrecognized command line option ‘-mthreads’; did you mean ‘-pthread’?
Makefile:166: recipe for target 'windows' failed
make: *** [windows] Error 2

Per this GH issue (specifically GH user tiaguinho's feedback):

mattn/go-sqlite3#303 (comment)

the fix is to either use a Docker container with everything needed already included, or install some additional packages and specify specific options for the Windows builds:

First install this two packages

apt-get install gcc-multilib
apt-get install gcc-mingw-w64

After that I use this command to compile

GOOS=windows GOARCH=386 CGO_ENABLED=1 CXX_FOR_TARGET=i686-w64-mingw32-g++ CC_FOR_TARGET=i686-w64-mingw32-gcc go build

This works for me.

If this is the route used, then the docs will need to be updated to reflect this build requirement.

instead of the intended (but not well noted in the Makefile) static executables. This commit includes changes noted in upstream `golang/go` issues which appear to work as intended for others. I also include doc comments and reference links describing the use of the updated build options list and some problems that I encountered during testing of `linkmode=external`. - refs GH-48 - refs golang/go 38789 - refs golang/go 26492

The previous release generated Windows binaries that were broken (GH-53) and Linux binaries that were unintentionally dynamic (GH-48). This commit provides an updated Makefile and GitHub workflow which provides multiple build options for both dynamic and static linking, but defaults to the previous dynamically linked behavior. The Makefile build options now explicitly enable the `CGO_ENABLED` environment variable so that all builds have the required cgo functionality enabled. A `docker` Makefile recipe is provided to generate binaries using new Docker images from the `atc0005/go-ci` project based on the official Golang Alpine Linux image. The result is statically linked binaries based on the musl C library instead of glibc. The intent is to help prevent licensing issues surround the GNU C library's LGPL licensing (which I do not fully understand). Multiple build tags are specified for static builds which enable Go-specific replacements for common glibc-provided features: - `osusergo` - `netgo` and a build tag specific to disabling SQLite extensions, which we do not use with this specific project: - `sqlite_omit_load_extension` Minor documentation updates have been included which update the build requirements and specific steps for building binaries for this project. Further updates are likely needed to add polish. A Docker Compose file has been included for kicking off multiple static binary builds in parallel, but it may end up getting tossed in a later PR if we don't make sufficient use of it. - refs GH-48 - refs golang/go 38789 - refs golang/go 26492 - refs atc0005/go-ci#85

The previous release generated Windows binaries that were broken (GH-53) and Linux binaries that were unintentionally dynamic (GH-48). This commit provides an updated Makefile and GitHub workflow which provides multiple build options for both dynamic and static linking, but defaults to the previous dynamically linked behavior. The GitHub Workflow jobs based on the Makefile build tasks use a 20 minute timeout vs the 10 minute timeout used previously. The Makefile build options now explicitly enable the `CGO_ENABLED` environment variable so that all builds have the required cgo functionality enabled. A `docker` Makefile recipe is provided to generate binaries using new Docker images from the `atc0005/go-ci` project based on the official Golang Alpine Linux image. The result is statically linked binaries based on the musl C library instead of glibc. The intent is to help prevent licensing issues surround the GNU C library's LGPL licensing (which I do not fully understand). Multiple build tags are specified for static builds which enable Go-specific replacements for common glibc-provided features: - `osusergo` - `netgo` and a build tag specific to disabling SQLite extensions, which we do not use with this specific project: - `sqlite_omit_load_extension` Minor documentation updates have been included which update the build requirements and specific steps for building binaries for this project. Further updates are likely needed to add polish. A Docker Compose file has been included for kicking off multiple static binary builds in parallel, but it may end up getting tossed in a later PR if we don't make sufficient use of it. - refs GH-48 - refs golang/go 38789 - refs golang/go 26492 - refs atc0005/go-ci#85

The previous release generated Windows binaries that were broken (GH-53) and Linux binaries that were unintentionally dynamic (GH-48). This commit provides an updated Makefile and GitHub workflow which provides multiple build options for both dynamic and static linking, but defaults to the previous dynamically linked behavior. The GitHub Workflow jobs based on the Makefile build tasks use a 20 minute timeout vs the 10 minute timeout used previously. Also, `CGO_ENABLED` has been explicitly set in order to ensure that cgo support is enabled. The Makefile build options now explicitly enable the `CGO_ENABLED` environment variable so that all builds have the required cgo functionality enabled. A `docker` Makefile recipe is provided to generate binaries using new Docker images from the `atc0005/go-ci` project based on the official Golang Alpine Linux image. The result is statically linked binaries based on the musl C library instead of glibc. The intent is to help prevent licensing issues surround the GNU C library's LGPL licensing (which I do not fully understand). Multiple build tags are specified for static builds which enable Go-specific replacements for common glibc-provided features: - `osusergo` - `netgo` and a build tag specific to disabling SQLite extensions, which we do not use with this specific project: - `sqlite_omit_load_extension` Minor documentation updates have been included which update the build requirements and specific steps for building binaries for this project. Further updates are likely needed to add polish. A Docker Compose file has been included for kicking off multiple static binary builds in parallel, but it may end up getting tossed in a later PR if we don't make sufficient use of it. - refs GH-48 - refs golang/go 38789 - refs golang/go 26492 - refs atc0005/go-ci#85

The previous release generated Windows binaries that were broken (GH-53) and Linux binaries that were unintentionally dynamic (GH-48). This commit provides an updated Makefile and GitHub workflow which provides multiple build options for both dynamic and static linking, but defaults to the previous dynamically linked behavior. The GitHub Workflow jobs based on the Makefile build tasks use a 20 minute timeout vs the 10 minute timeout used previously. Also, `CGO_ENABLED` has been explicitly set in order to ensure that cgo support is enabled. The Makefile build options now explicitly enable the `CGO_ENABLED` environment variable so that all builds have the required cgo functionality enabled. A `docker` Makefile recipe is provided to generate binaries using new Docker images from the `atc0005/go-ci` project based on the official Golang Alpine Linux image. The result is statically linked binaries based on the musl C library instead of glibc. The intent is to help prevent licensing issues surround the GNU C library's LGPL licensing (which I do not fully understand). Multiple build tags are specified for static builds which enable Go-specific replacements for common glibc-provided features: - `osusergo` - `netgo` and a build tag specific to disabling SQLite extensions, which we do not use with this specific project: - `sqlite_omit_load_extension` Minor documentation updates have been included which update the build requirements and specific steps for building binaries for this project. Further updates are likely needed to add polish. Several reference links have also been added. A Docker Compose file has been included for kicking off multiple static binary builds in parallel, but it may end up getting tossed in a later PR if we don't make sufficient use of it. - refs GH-48 - refs golang/go 38789 - refs golang/go 26492 - refs atc0005/go-ci#85

atc0005 added bug Something isn't working builds labels Oct 3, 2020

atc0005 added this to the Next Release milestone Oct 3, 2020

atc0005 self-assigned this Oct 3, 2020

atc0005 changed the title ~~Makefile build options does not generate static binaries~~ Makefile build options do not generate static binaries Oct 3, 2020

atc0005 mentioned this issue Oct 6, 2020

Makefile build options do not generate static binaries atc0005/check-mail#94

Closed

atc0005 mentioned this issue Oct 6, 2020

Explicitly disable use of cgo atc0005/check-mail#101

Merged

This was referenced Oct 7, 2020

Windows builds are broken for v0.1.0 release #53

Closed

Add images for building static, cgo-enabled projects atc0005/go-ci#85

Closed

atc0005 mentioned this issue Oct 8, 2020

Refactor binary generation options #54

Merged

atc0005 closed this as completed in #54 Oct 8, 2020

This was referenced Oct 9, 2020

Review other potential replacements for mattn/go-sqlite3 package #18

Open

Go | Docker | Build static binary using musl C library atc0005/notes#35

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Makefile build options do not generate static binaries #48

Makefile build options do not generate static binaries #48

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020 •

edited

Loading

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 7, 2020

atc0005 commented Oct 7, 2020

Makefile build options do not generate static binaries #48

Makefile build options do not generate static binaries #48

Comments

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 3, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020 • edited Loading

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 4, 2020

atc0005 commented Oct 7, 2020

atc0005 commented Oct 7, 2020

atc0005 commented Oct 4, 2020 •

edited

Loading