-
Notifications
You must be signed in to change notification settings - Fork 805
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix docker-push.sh execution #3919
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Protoc is apparently incompatible with alpine images, it has something to do with musl and glibc incompatibilities. In the alpine images, `sh` complains `command not found`, but after some more digging you can see `ldd` has more info: ``` ldd .build/bin/protoc /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by .build/bin/protoc) Error relocating .build/bin/protoc: __strftime_l: symbol not found ``` After a bit of experimenting, it seems that the semi-common tricks of "install `libc6-compat` or `gcompat`" do not work for protoc. I'm not really sure why. So our options are basically one of: - Abandon alpine for this build section. It just builds binaries that are copied to the next step, so it won't affect the final image size anyway. - Keep alpine, build protoc from source, hope it compiles successfully against musl - Keep alpine, but don't do codegen in this build Option 3 seemsed the quickest. After some toying around and (as always) learning something new about Make, I found this tactic. It appears to be well-supported and consistent, so I'm OK with using it. Binaries are still downloaded/built during release builds with this approach, but that seems like a relatively low-risk / low-cost consequence. This all also means that devs will download and build these tools as part of a normal `make bins`, but similarly will not run codegen unless needed. Since that kinda helps "prime" the dev environment, that seems totally fine / possibly even a good thing. Codegen *should* still run when necessary though.
mantas-sidlauskas
approved these changes
Jan 25, 2021
mkolodezny
approved these changes
Jan 25, 2021
yux0
approved these changes
Jan 25, 2021
github-actions bot
pushed a commit
to vytautas-karpavicius/cadence
that referenced
this pull request
Feb 4, 2021
**Why this commit exists:** "build and push master" buildkite steps are failing, due to both: - their images not having `unzip` available - `protoc` failing to run on alpine after adding `unzip`, e.g. `/bin/sh: .build/bin/protoc: not found` - pip v21 dropped support for python2 entirely, and `cqlsh` hasn't been updated in nearly 4 years and is stuck on 2.7 This fixes that by 1) adding unzip, 2) not executing thrift/proto codegen during those release builds (and in other locations), and 3) preventing pip from going beyond v20.x --- **Lots more details:** Protoc is apparently incompatible with alpine images, it has something to do with musl and (g?)libc incompatibilities. In the alpine images, `sh` complains `.build/bin/protoc: not found`, but after some more digging you can see `ldd` has more info: ``` ldd .build/bin/protoc /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by .build/bin/protoc) Error relocating .build/bin/protoc: __strftime_l: symbol not found ``` After a bit of experimenting, it seems that the semi-common tricks of "install `libc6-compat` or `gcompat`" do not work for protoc. I'm not really sure why. So our options are basically one of: - Abandon alpine for this build section. It just builds binaries that are copied to the next step, so it won't affect the final image size anyway. (this builds successfully, but I have not checked the resulting binaries) - Keep alpine, build protoc from source, hope it compiles successfully against musl - Keep alpine, but don't do codegen in this build (this works) Option 3 seemed worth trying in depth. After some toying around and (as usual) learning something new about Make, I found this tactic. It appears to be well-supported and consistent, so I'm OK with using it, though it's rather strange at first glance. Binaries are still downloaded/built during release builds with this approach, but that seems like a relatively low-risk / low-cost consequence. This all also means that devs will download and build these tools as part of a normal `make bins`, but similarly will not run codegen unless needed. Since that kinda helps "prime" the dev environment, that seems totally fine / possibly even a good thing. Codegen *should* still run when necessary though.
yux0
pushed a commit
to yux0/cadence
that referenced
this pull request
May 4, 2021
**Why this commit exists:** "build and push master" buildkite steps are failing, due to both: - their images not having `unzip` available - `protoc` failing to run on alpine after adding `unzip`, e.g. `/bin/sh: .build/bin/protoc: not found` - pip v21 dropped support for python2 entirely, and `cqlsh` hasn't been updated in nearly 4 years and is stuck on 2.7 This fixes that by 1) adding unzip, 2) not executing thrift/proto codegen during those release builds (and in other locations), and 3) preventing pip from going beyond v20.x --- **Lots more details:** Protoc is apparently incompatible with alpine images, it has something to do with musl and (g?)libc incompatibilities. In the alpine images, `sh` complains `.build/bin/protoc: not found`, but after some more digging you can see `ldd` has more info: ``` ldd .build/bin/protoc /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f9b1f366000) Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by .build/bin/protoc) Error relocating .build/bin/protoc: __strftime_l: symbol not found ``` After a bit of experimenting, it seems that the semi-common tricks of "install `libc6-compat` or `gcompat`" do not work for protoc. I'm not really sure why. So our options are basically one of: - Abandon alpine for this build section. It just builds binaries that are copied to the next step, so it won't affect the final image size anyway. (this builds successfully, but I have not checked the resulting binaries) - Keep alpine, build protoc from source, hope it compiles successfully against musl - Keep alpine, but don't do codegen in this build (this works) Option 3 seemed worth trying in depth. After some toying around and (as usual) learning something new about Make, I found this tactic. It appears to be well-supported and consistent, so I'm OK with using it, though it's rather strange at first glance. Binaries are still downloaded/built during release builds with this approach, but that seems like a relatively low-risk / low-cost consequence. This all also means that devs will download and build these tools as part of a normal `make bins`, but similarly will not run codegen unless needed. Since that kinda helps "prime" the dev environment, that seems totally fine / possibly even a good thing. Codegen *should* still run when necessary though.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why this commit exists:
"build and push master" buildkite steps are failing, due to both:
unzip
availableprotoc
failing to run on alpine after addingunzip
, e.g./bin/sh: .build/bin/protoc: not found
cqlsh
hasn't been updated in nearly 4 years and is stuck on 2.7This fixes that by 1) adding unzip, and 2) not executing thrift/proto codegen during those release builds (and in other locations).
Lots more details:
Protoc is apparently incompatible with alpine images, it has something to do with musl and (g?)libc incompatibilities.
In the alpine images,
sh
complains.build/bin/protoc: not found
, but after some more digging you can seeldd
has more info:After a bit of experimenting, it seems that the semi-common tricks of "install
libc6-compat
orgcompat
" do not work for protoc.I'm not really sure why.
So our options are basically one of:
Option 3 seemed worth trying in depth. After some toying around and (as usual) learning something new about Make, I found this tactic.
It appears to be well-supported and consistent, so I'm OK with using it, though it's rather strange at first glance.
Binaries are still downloaded/built during release builds with this approach, but that seems like a relatively low-risk / low-cost consequence.
This all also means that devs will download and build these tools as part of a normal
make bins
, but similarly will not run codegen unless needed.Since that kinda helps "prime" the dev environment, that seems totally fine / possibly even a good thing. Codegen should still run when necessary though.