Strip binaries #40
Strip binaries #40
Conversation
Stripped binaries are removed of unused objects and become lighter in resource usage and have better performance.
|
travis is failing with I wanted to remove the Also, to take advantage of ccache I think the full path needs to remain the same between builds, would you be open to a patch about that? |
|
Hello @fauno, thanks for your contribution. I didn't had time yet to review fully the impact of this change, but two points I would like to bring before anything:
I'm leaving these notes here and let me know if you have explored those. Last but not least, I'm interesting to better understand your motivation of this change, if a concrete scenario or need raised it. Thank you in advance and looking forward your feedback. Cheers. |
|
Luis Lavena <notifications@github.com> writes:
Hello @fauno, thanks for your contribution.
thanks! usually i run something like `find -name .so | xargs strip` on
my gems directory, but since i wanted to start using gem-compiler to
reduce cpu/energy usage i thought it would be a nice feature (amongst
the other things i mentioned)
I didn't had time yet to review fully the impact of this change, but
two points I would like to bring before anything:
1. The changes requires to be portable across Ruby and
compilers. There might not be `strip` or might be called differently
in other platforms. This information is exposed by `RbConfig::CONFIG`
and should be used instead of hardcoding. This also includes the file
extensions. Ironically, Ruby's extensions have `.so`, even on Windows.
sure, i can make it check for *nix systems instead of file extension.
what compiler does ruby use on windows?
2. Shelling out using backticks might not scale correctly. What would
happen when things fail? How the user should be informed?
i can use popen3 instead and check if `strip` is available, though if
you have a build env you probably have it already.
gem-compiler relies on RubyGems' own building process, so perhaps we
can rely on them. If not, perhaps the issue could be solved directly
by `extconf.rb` itself and not at gem-compiler.
wouldn't that require changing every gem? in the long run it would
benefit everyone but that's a lot of pull requests :P
I'm leaving these notes here and let me know if you have explored
those. Last but not least, I'm interesting to better understand your
motivation of this change, if a concrete scenario or need raised it.
well, i've been packaging for a distro (https://parabola.nu) for several
years and recently found out that binary gems aren't stripped, which is
common practice while packaging (usually the packager does this for you,
if you run `file /usr/bin/anything` it will say 'stripped' at the end).
stripped binaries are much smaller in size and memory use, since you
don't have to load unneeded objects. for example, i just tested against
a ruby 2.3.7 i had from an old rbenv and the ruby interpreter went from
15 to 3MB.
https://en.wikipedia.org/wiki/Strip_(Unix)
i could run a few benchmarks if you want, i would need to find a binary
gem that has it's own though :)
Thank you in advance and looking forward your feedback.
thanks for the feedback!
…
|
RubyInstaller uses MinGW (gcc) and there is also Visual Studio (MSVC). I wouldn't recommend excluding things by that method. I took a quick look and Ruby already collects if strip is available using In relation to process execution and capture, we should be following RubyGems' own approach, similar to the following, using https://github.com/rubygems/rubygems/blob/master/lib/rubygems/ext/builder.rb#L62-L81 I will not have time to tackle yet (probably in a few weeks) but feel free to update this PR and we will sync back soon. Once again, thank you for your contribution and the details on the motivation, indeed makes a really useful case and looking forward integrate such changes to improve the usability of this tool. Cheers. |
|
hi, i didn't forget about this but i've been doing other things. i did a
quick test with popen3 and even added a flag to optionally keep symbols,
but found out that `RbConfig::CONFIG['STRIP']` detects strip with some
flags that do remove debug symbols, but don't strip everything, so the
binaries are still identified as "not stripped".
do you think it would be ok to detect if strip is present via RbConfig
but still use `strip --strip-unneeded` via popen3? (i remember
`--strip-all` preventing them from loading).
…--
D
|
Don't worry, not going anywhere 😄 Appreciate the time you take to collaborate and investigate how we can reach the best option.
I think we can work with that. Will be tricky to test striping since the test fake an extension, so doesn't produce a valid shared object file. I will try at some point to workout some integration tests that generates a real C extension, but I think is OK for now. One last nitpick: can you sort Thank you for your time and patience with my feedback. And thank you for your contribution! |
Introduce `--strip` to perform stripping on compiled extensions (using same command as defined by `RbConfig::CONFIG["STRIP"]`). Or using a custom stripping command using `--strip-cmd`, allowing different executable and flags to be used. This is not enabled by default. Closes #40.
|
Hi! I was just thinking of this and found out the notification email! Great! Thanks! We've been using gem-compiler in production for a few months now and everything's great. Here are some examples for creating a binary gem repository for Alpine (ruby < 2.7 requires a patch on rubygems to correctly detect the linux-musl target):
|
|
This is excellent @fauno!!! Really happy to see this project being put to a good use (outside of me, 😁). Just wanted to let you know that simplified symbol stripping UI/UX and reduced to just Only need to finish some include/exclude pattern globing changes and will release the new version, probably this weekend. Cheers |
Stripped binaries are removed of unused objects and become lighter in
resource usage and have better performance.