Support static linkage on all platforms

[bobsummerwill]

At least for Linux and OS X, our libraries are always dynamically linked. This produces massive bloat in our deployment process, because it means that dead-code stripping isn't possible.

The cross-build ZIPs, for example, weigh in at ~230Mb each.

We have supported static linkage in the past, but the CMake code paths have likely not been used for months and will be broken.

Static linkage is especially important for mobile and embedded scenarios.

[smartbitcoin]

Great idea. just wondering why nobody complain about the 200M+ binary when they do apt-get :)

[bobsummerwill]

Those are that size too, eh? Makes sense.

I really notice this when pushing to a smartwatch which only has 4Gb of flash storage. Not only does the push take forever (and often timeout), but the binaries just chew up space which I don't have, and memory which we don't have. I would stunned if a dead-stripped release binary weighed in at more than 20Mb or so, @smartbitcoin.

[chriseth]

We certainly won't statically link for package managers that support dynamic linking like deb or homebrew (?), for everything else it makes a lot of sense.

[bobsummerwill]

This issue is talking about having the option to static link, not switching the default.

We would static link our own libraries, and uncommon ones like CryptoPP, etc. Not boost or glibc or pthreads, of course.

The cross-build ZIPs contain 61 .so files required by eth. That is bonkers. For desktop OS X and Linux builds we will be depending on even more (with qt5 and v8 and llvm added).

Static linking lets us dead-strip.

[nerdralph]

Static linking gets you a lot more than basic dead code elimination (i.e. good old -fgc-sections) when you use LTO.
http://hubicka.blogspot.ca/2016/03/building-libreoffice-with-gcc-6-and-lto.html

defining ETH_STATIC will create static libs (EthCompilerSettings.cmake sets BUILD_SHARED_LIBS OFF)
I think setting ETH_STATIC should result in static libs and statically-linked executables.

I also think the shared lib .deb packages cause more problems than they solve. Shared lib version conflicts has been a big problem for literally decades. For a relevant example, try testing a self-build .deb of genoil's ethminer fork on the same machine that has cpp-ethereum installed from the ppa.

I actually found this issue because I was trying to build a static version of genoil's branch so I wouldn't have the shared lib conflicts. After I noticed there was incomplete support for static builds, I decided to look upstream to see if it was already solved.

[bobsummerwill]

^^ This!

Yes, @nerdralph. I know and completely agree with everything you say about.

I think that it's likely that when we take the time to get static linkage working, we're only going to want dynamic linkage for a handful of libraries (system libraries, boost, OpenCL, perhaps).

But yeah - first we just need to do the work, and then assess it from there.

After we have done the repro reorgs and had a good try add upstream Genoil changes, I would like to massively collapse the number of libraries within cpp-ethereum - essentially bundling EVERYTHING into a single libethereum.a. I see little reason for individual DLLs for each of these sub-modules, which are never used independently.

[rainbreak]

I built a fully static solc and soltest by building on Alpine with musl. I think these should work on any x86_64-linux.

Source here: https://github.com/rainbeam/solidity-static (see releases for binaries).

solc weighs in at 9.1MB and soltest is 19MB.

I also had success with building static eth, although it isn't included in the above and I didn't test it much.

A question I have: if soltest runs and passes its tests, is this sufficient to say that solc is stable?

[bobsummerwill]

Fantastic, @rainbeam.

Do you happen to know what the executable size for your eth was? Or could built it again and find out?

I can believe that we could get to "pretty tiny".

WRT - Running and passing tests. I don't know how extensive soltest is. @chriseth? Other parts of the C++ codebase could really use more in the way of tests.

That is good news, though. Your binaries are probably pretty usable, for sure. You could build some of your own .sol files too and check you get the same binaries from both too.

[rainbreak]

@bobsummerwill I can probably recreate the eth binary. The script wasn't that different from the solidity build. Will report back on that.

[bobsummerwill]

Thanks, @rainbeam!

[rainbreak]

@bobsummerwill: my memory was a bit optimistic. I succeeded in building eth on Alpine, but not in static linking against system libraries. Problem seems to be libcurl / libssh2. Tracking here: rainbreak/eth-static#1

[bobsummerwill]

np, @rainbeam.
Please do keep me in the loop. Best wishes!

[rainbreak]

@bobsummerwill I built fully static eth (against musl). Binary size is 19MB (unstripped).

With symbol stripping, all of the binaries come down in size a fair bit: solc - 3MB; soltest - 9MB; eth - 10MB.

Doing this just required some cmake modifications to force use of static libraries. eth seems to work ok, but I've not properly tested it out.

[bobsummerwill]

Nice, @rainbeam!

If you have a working branch which you are hacking away on, please feel free to share, or log a messy pull request. Would love to see the kind of scope of edits you needed. Thanks!

[rainbreak]

Will do if I've time at the weekend. It started off methodically but descended into hacking so the work is currently in a container. The changes aren't that big - mostly inside webthree-helpers.

[bobsummerwill]

It's all good - thanks, @rainbeam!

[nerdralph]

Which gcc version, and did you try lto?
On Apr 28, 2016 22:33, "Bob Summerwill" notifications@github.com wrote:

It's all good - thanks, @rainbeam https://github.com/rainbeam!

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#337 (comment)

[rainbreak]

@nerdralph: gcc 5.30 and no lto. There isn't a gcc 6 package for Alpine Linux yet.

[rainbreak]

@bobsummerwill see referenced PRs ^.

I've automated the build and there is an example binary here:

https://github.com/rainbeam/eth-static/releases.

[bobsummerwill]

Thanks, @rainbeam! I had missed that and thanks for the new PRs :-)

https://github.com/rainbeam/eth-static

[bobsummerwill]

Work is happening in #495.