Use staticlib

[cxtuttle]

Adds the ability to compile everything statically with an installed duckdb libs.

I tend to compile similar to the following, but I mostly need the parquet extension:

export CGO_LDFLAGS="-lduckdb_static -lduckdb_re2 -lduckdb_pg_query -lduckdb_fmt -lduckdb_utf8proc -lduckdb_hyperloglog -lduckdb_miniz -lduckdb_fastpforlib -licu_extension -lparquet_extension -lvisualizer_extension -ljemalloc_extension -lduckdb_mbedtls -lduckdb_fsst -ldl -lm -lstdc++"

go build --tags=duckdb_use_staticlib cmd/test_duck.go

[marcboeker]

@cxtuttle Thanks for the PR :)
Could you please elaborate a bit on your use case? If I understand you correctly you need a special version of DuckDB bundled that has additional extension and featured compiled in?
Could you explain what your steps are, to build a duckdb_static so that we can add a small guide to the README.md for users that want to build a custom version of DuckDB like you are doing?
Thanks!

[disq]

I found this useful when trying to compile a binary for an older platform, e.g. Debian Buster and not wanting to bundle libduckdb.so. Still needed to compile duckdb from source (to get all the .a files) all the dependencies for it to work because of duckdb/duckdb#9475 but it helped.

[marcboeker]

@disq We ship a precompiled static version of DuckDB that is used by default if you run go build. There is no need to build it from source anymore. Or do you mean something else?

[disq]

@marcboeker The precompiled static version doesn't seem to have other dependencies so linking becomes an issue with unknown symbols popping up everywhere, at least on the latest releases. I had to do this to be able to compile it "fully statically" (using the staticlib feature in this PR):

export CGO_LDFLAGS="-lduckdb_static -lduckdb_re2 -lduckdb_pg_query -lduckdb_fmt -lduckdb_utf8proc -lduckdb_hyperloglog -lduckdb_miniz -lduckdb_fastpforlib -lparquet_extension -ljemalloc_extension -lduckdb_mbedtls -lduckdb_fsst -ldl -lm -lstdc++ -L/home/ec2-user/duckdb-0.9.2/build/release/third_party/miniz -L/home/ec2-user/duckdb-0.9.2/build/release/third_party/fmt -L/home/ec2-user/duckdb-0.9.2/build/release/third_party/libpg_query -L/home/ec2-user/duckdb-0.9.2/build/release/third_party/imdb -L/home/ec2-user/duckdb-0.9.2/build/release/third_party/re2 -L/home/ec2-user/duckdb-0.9.2/build/release/third_party/mbedtls -L/home/ec2-user/duckdb-0.9.2/build/release/third_party/fastpforlib -L/home/ec2-user/duckdb-0.9.2/build/release/third_party/fsst -L/home/ec2-user/duckdb-0.9.2/build/release/third_party/utf8proc -L/home/ec2-user/duckdb-0.9.2/build/release/third_party/hyperloglog -L/home/ec2-user/duckdb-0.9.2/build/release/extension/parquet -L/home/ec2-user/duckdb-0.9.2/build/release/extension/jemalloc -L/home/ec2-user/duckdb-0.9.2/build/release/test/helpers -L/home/ec2-user/duckdb-0.9.2/build/release/src -L/home/ec2-user/duckdb-0.9.2/build/release/tools/sqlite3_api_wrapper"

sudo yum install glibc-static
sudo yum install libstdc++-static
CGO_ENABLED=1 CGO_CFLAGS="-I/home/ec2-user" go build -tags=duckdb_use_staticlib -o my_duckdb_binary -buildmode=exe -ldflags '-s -w -linkmode external -extldflags=-static' .

This was done because my aim was to compile it on a current system and then run it on an older debian (Buster)
Which didn't solve everything (load/installing extensions became a nuisance, which had solutions but I didn't find it acceptable) so in the end I had to give up.

[marcboeker]

@disq I agree. It would be sensible to ship a fully static precompiled version with go-duckdb, eliminating the need for users to build their own. Can you please share the list of unknown symbols from your system so that I can reproduce this.

[taniabogatsch]

From my understanding, there are two (related) points here.

1. Compiling duckdb statically with a custom duckdb library.
2. Dependencies and linking of the pre-compiled or custom duckdb libraries.

What is the status of this PR? As there has not been any activity in a while.

[loicalleyne]

FWIW would love to have this to improve the portability of my programs.

[KoduIsGreat]

Any word on the status of this?

[marcboeker]

@KoduIsGreat Unfortunately not. Maybe @taniabogatsch has some more information on this?

[taniabogatsch]

Unfortunately, no - I think this PR is stale - maybe some of the people commenting are interested in picking it up? I am not really in the matter of (cross) compiling this with different builds of duckdb, is this PR related to #279 (and other issues around static linking)?

Potentially, we also saw some related issues, e.g., #330?