PROD-3069: Cross-compile self contained binaries #8
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jossemargt-cto-ai
force-pushed
the
feature/xcompile-self-contained-binaries
branch
4 times, most recently
from
7854d2d to
d2a7e44
Compare
jossemargt-cto-ai
requested review from
kc-dot-io,
CampbellBrett and
nitishkumar71
jossemargt-cto-ai
changed the title
Up to this iteration we used to link our binaries to the GNU C library (AKA glibc), which works fine for most of GNU distributions. However, it is well known that managing said system dependency comes at a great maitenance cost, so it would be best to ship self-contained binaries by migrating into the musl C library. Turns out, the only dependency that tied us to glibc was OpenSSL which is no longer required for our HTTP client. So by changing some cargo configurations we were able to do the switch we were aiming for. This change came with the additional benefit of enabling cross compiliation without relying on chip emulation, which is way faster! To the point of reducing the CI time and billing by at least 5x. Signed-off-by: Jonnatan Jossemar Cordero <jossemar.cordero@cto.ai>
jossemargt-cto-ai
force-pushed
the
feature/xcompile-self-contained-binaries
branch
2 times, most recently
from
6625939 to
5948918
Compare
jossemargt-cto-ai
force-pushed
the
feature/xcompile-self-contained-binaries
branch
2 times, most recently
from
24cd9f6 to
7934e79
Compare
Since cross compiling requires a couple of extra setup, we decided to move the build logic into a reusable workflow so there's only one way of doing things regarding CI. Signed-off-by: Jonnatan Jossemar Cordero <jossemar.cordero@cto.ai>
Signed-off-by: Jonnatan Jossemar Cordero <jossemar.cordero@cto.ai>
jossemargt-cto-ai
force-pushed
the
feature/xcompile-self-contained-binaries
branch
from
7934e79 to
a54fb86
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why
Although we managed to generate binaries for multiple architectures on #6, we still had plenty opportunities of improvement like:
What
By switching from
OpenSSLintorustlsfrom one of the dependencies, we were able to move from glibc into musl. By using the latter C standard library the rust compiler is able to build self contained binaries.Extra
The real changes are on the
Cargo.tomland the.github/workflows/wc_buid_binaries.ymlfiles, the rest is code grooming.By the way, with taking the chipset emulation out of the picture we reduced the cross-build time from ~30 minute to ~7 minutes 🎉🎉🎉