Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

With go 1.9.4, static build of applications fails with "invalid pkg-config package name: --static" #137

Closed
7 tasks
bolgarov10 opened this issue Feb 17, 2018 · 8 comments

Comments

@bolgarov10
Copy link

bolgarov10 commented Feb 17, 2018

Description

For security reasons, Go 1.9.4 compiler uses whitelist to filter out most of the flags that can be used in //cgo <...> directive, see here:
golang/go#23672

(See also golang/go#23749 for lots of other people complains and some workarounds)

How to reproduce

Use Go 1.9.4. compiler;
Get libdrkafka, build it;
Get confluent-kafka-go package;
cd to .../src/github.com/confluentinc/confluent-kafka-go/kafkatest/go_verifiable_consumer ;
run "go build -tags static";
see the error message: "go build github.com/confluentinc/confluent-kafka-go/kafka: invalid pkg-config package name: --static"

Checklist

Please provide the following information:

  • confluent-kafka-go and librdkafka version (LibraryVersion()):
  • Apache Kafka broker version:
  • Client configuration: ConfigMap{...}
  • Operating system:
  • Provide client logs (with "debug": ".." as necessary)
  • Provide broker log excerpts
  • Critical issue
@edenhill
Copy link
Contributor

We'll have to wait for Golang to be updated to allow the -static flag (mentioned on the referenced issues).
There isn't much we can do short of hard-coding the linker flags from the pkg-config command.

timothyb89 added a commit to monasca/monasca-docker that referenced this issue Feb 20, 2018
Building of confluent-kafka-go is currently broken on go v1.9.4 which
is currently shipping with alpine:edge [1]. This switches the image
build to alpine:3.7 which does now have this problem, however we may
run into issues upgrading to later versions if confluent-kafka-go
doesn't find a workaround for the underlying problem.

[1] confluentinc/confluent-kafka-go#137

Signed-off-by: Tim Buckley <timothy.jas.buckley@hpe.com>
@schnie
Copy link

schnie commented Feb 28, 2018

I just hit this issue rebuilding a docker image (alpine 3.7). I'm having some trouble getting any workarounds to work. Does anybody have any kind of hack to get around this until it's fixed? What would be the best way to hard code the linker flags? Just getting into go recently, so I may be missing something obvious.

@chstout
Copy link
Contributor

chstout commented Feb 28, 2018

-tags static appears to work fine with go 1.10, if upgrading go version is an option

@edenhill
Copy link
Contributor

edenhill commented Mar 4, 2018

Since pkg-config --static is problematic (with and without Go) I'm looking into providing separate .pc files for dynamic and static linking of librdkafka, that should solve this issue as well.

@reaVen-
Copy link

reaVen- commented Mar 13, 2018

I see this issue being closed, does that mean I can now statically build a go application that uses confluent-kafka-go inside a small alpine image? How would I go about doing that? and does anyone have a dockerfile that uses multi-staging? eg you first make install librdkafka, go get confluent-kafka then transfer the executable to a new clean alpine image? seems like a standard use-case

@edenhill
Copy link
Contributor

I haven't tried building on Alpine, please report your findings.

@reaVen-
Copy link

reaVen- commented Mar 13, 2018

I switched to a pure Go implementation, which resulted in about a 6.5 MB container. When I have the rest of the system up (Probably in a week or so), I will measure the performance of that small container, against a normal librdkafka-client container. And then decide if I need to switch back and solve this compile issue I had when trying to get everything down to one single binary file.

I'll be sure to report my findings unless someone beats me too it.

@edenhill
Copy link
Contributor

Maybe we should have an alpine-docker build in .travis.yml to verify that it works if this is a common use-case (PRs welcome!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants