Automate building of binaries

[andrew]

We need to make it easier to rebuild binaries for each platform, perhaps using vagrant boxes and puppet.

[robbschiller]

+1 - Wish I could help on this... Vagrant can automate?

[deanmao]

Do you know of anywhere we could get access to other architectures like solaris etc? What does joyent use? I could supply the 32bit & 64bit machines for x86.

[andrew]

Maybe @dscape can hook us up with one at @nodejitsu

[dscape]

We are working on this actively. Since SASS is such an important part of the workflow for front end engineers I'm going to flag this module, and probe interest.

On the longer run yes we are working super hard to make this a possibility for everyone

[dean-m]

@dscape does joyent also have linux x86 32bit & 64bit machines too?

[dscape]

no idea, check their site is the best guess

[silentrob]

Joyent does have Solaris, and linux machines. Im not there anymore so I can't hook you up - sorry.

[dscape]

@pgte do you think this is easy to do?

[andrew]

https://github.com/nodejitsu/module-foundry/ looks like a great solution to this problem

[nschonni]

We have, however, tested module-foundry against other platforms for the most popular native Node.js modules:
...
17 1 407 37 node-sass

Wonder if they'll share 😉

[nschonni]

I came across http://www.appveyor.com/ after it landed in Grunt. I haven't been able to get the binary building working yet, but it maybe a solution for the Windows part of the binaries if I/we can get it running.

https://ci.appveyor.com/project/nschonni/node-sass

/cc @LaurentGoderre

[andrew]

Looks promising for windows build if you can get it working 👍

[LaurentGoderre]

I remain skeptical. It took me a week to have it setup to build x32 and x64 correctly.

[nschonni]

@FeodorFitsner have you seen any examples of people doing NodeJS C++ bindings on AppVeyor?

[FeodorFitsner]

Yes, sure. This project for example:
https://ci.appveyor.com/project/ChrisWilliams/node-serialport

there were some issues and found the solution: http://help.appveyor.com/discussions/problems/216-problem-while-compiling-with-x64-platform

Wilhelm (@BergWerkGIS) helped a lot too.

[wilhelmberg]

Here are some other C++ projects I'm working on with @springmeyer and the support of @FeodorFitsner to build them on AppVeyor:

https://ci.appveyor.com/project/springmeyer/node-srs
https://ci.appveyor.com/project/springmeyer/node-sqlite3
https://ci.appveyor.com/project/springmeyer/node-pre-gyp
https://ci.appveyor.com/project/springmeyer/node-mapnik
https://ci.appveyor.com/project/springmeyer/node-zipfile

[deepak1556]

@andrew @nschonni i have a working appveyor configuration for node-sass setup here https://github.com/deepak1556/node-sass/tree/appveyor_setup . Also wats your opinion about using https://github.com/mapbox/node-pre-gyp .

[nschonni]

❤️ ❤️ ❤️ Awesome @deepak1556! Do you want to PR the .app-veyor.yml file? I also saw you have some changes in the binding.gyp that make sense https://github.com/deepak1556/node-sass/compare/appveyor_setup

RE: node-pre-gyp. I looked at it before, but I didn't go forward with it here since we have the binaries repo instead of S3. If you want to put together something in a separate PR we could take a look again.

[tbranyen]

For NodeGit (https://github.com/nodegit/nodegit) I'm using node-pre-gyp and a Vagrant box setup on a Mac mini where I can install OS X 64bit, Windows 32/64bit, and Linux 32/64bit and automate OS X and Linux with a script. I've looked into vagrant rdp, but it's still a mystery to me as to how I'd utilize it like vagrant ssh --command.

[LaurentGoderre]

For Windows, installing only the 64 bit should suffice because you can easily build 32bits on 64bits Windows

[springmeyer]

@tbranyen - would be interested if you have notes on how your set up the vagrant box to provide for node-pre-gyp users. I've been automating OSX/Linux using travis and Windows using appveyor, but vagrant as a backup sounds like another good thing to document.

[am11]

Perhaps we can use puppet and vagrant and publish a recipe on node-sass-binaries repo, so anyone can just spawn all the environments and get the required releases.
http://www.linux.com/news/software/applications/694157-setup-your-dev-environment-in-nothing-flat-with-puppet

We can then use headless Linux: https://vagrantcloud.com/hashicorp/boxes/precise32 and https://vagrantcloud.com/hashicorp/boxes/precise64.

Also, IE team has released some vagrant boxes at modern.ie, so we can get x86 and x64 versions from there: http://blogs.msdn.com/b/ie/archive/2014/09/16/expanding-offers-and-vm-delivery-choices-on-modern-ie.aspx. I think this is evaluation version. We can install OpenSSH, VS2013 Express desktop and git. It's better that we keep both 32 and 64 bit version separate; as it is causing issues when we have one of them installed and other one standalone on Win-x64 edition.

(don't know how to bring Macintosh box to the mix though)

Furthermore, we can publish those prepared vagrant boxes to https://vagrantcloud.com/discover/featured, so anyone (with ample RAM) can just download and run the node-sass' build process on all machines in parallel. ⚡

[andrew]

Very interesting post by @edgarsilva about building multi-platform binaries: http://cylonjs.com/blog/2014/11/19/creating-multiplatform-precompiled-binaries-for-node-modules/

[am11]

Brilliant! Thank you for posting the link. 👍

[am11]

The "uncool" news is; vagrant cloud is not as open as it used to be, after migrating to Atlas. We cannot search boxes without registration. Plus it mentions: "This will be a paid product once out of tech preview. Pricing will be per node."

[saper]

Both Travis CI and Appveyor can push artifacts to a GitHub release: http://www.appveyor.com/docs/deployment/github I think this is the simplest way to do it out of CI, once we decide robots are in charge :)

[xzyfer]

I have looked into the Travis CI approach in the past. It'll work for oxs
but IIRC it's not compatible with hoops we jump for old centos support in
the Linux builds. Also Travis is using docker so we can't use our docker
script because they don't allow docker in docker for security reasons.
On 22 Aug 2015 09:57, "Marcin Cieślak" notifications@github.com wrote:

Both Travis CI and Appveyor can push artifacts to a GitHub release:
http://www.appveyor.com/docs/deployment/github I think this is the
simplest way to do it out of CI, once we decide robots are in charge :)

—
Reply to this email directly or view it on GitHub
#56 (comment).

[saper]

I am thinking about simple

deploy:
  provider: releases
  api_key: $GITHUB
  file: XXXX

setup that just works, no docker script required. Just upload the generated binding file.

Funny Linux setup won't work (unless you get the whole process working for Travis - also possible).
FreeBSD won't work as well - but those can still be build manually and added to the release.

Regarding old CentOS support - I am ready to re-visit sass/libsass#995 and sass/libsass#996 to make sure libsass standard autoconf setup works for old CentOS. Need only somebody to test it for me. Then no patching should be necessary - and we could use stock libsass. The only problem is that we might need autoconf setup but I see two solutions there:

1. Make node-sass setup using externally-built libsass with autoconf
2. Define a set of config.h constants in the libsass.gyp file once old CentOS is detected.

What is the distinguishing feature of the "old CentOS"? Old compiler? Which version is it? gcc 4.7 ?

[xzyfer]

Then no patching should be necessary - and we could use stock libsass

We don't patch LibSass anymore. @am11 put together a nice Docker setup that builds against an old gcc which seems to have solved our problem - https://gist.github.com/xzyfer/f5623546cb73b601b9b8

FreeBSD won't work as well

@am11 mentioned this in chat, does this help with automating FreeBSD binaries?

also to run the docker in travis, found: https://github.com/rocker-org/rocker/blob/56cb59cc262e9d2c02f9e11cfacab65f0b659c5f/.travis.yml#L10-L24 via http://stackoverflow.com/a/28223092.

[saper]

So it is only a matter of having CentOS 5 libc and libstc++.so from their g++ (I think it is 4.8.2 now) around, good.

[saper]

Pretty neat solution for Windows with @appveyor: #1135

[xzyfer]

I think this is largely sorted now. There are improvements to be made but for the most part binary generation is now automated.

[saper]

I was wondering if this means also automatically triggered builds?

[xzyfer]

IMHO the spirit of this issue is resolved. We should open new issues for specific improvements. There is no value in keeping this 2 year old issue open.

[saper]

We have enough new ones :)

[saper]

It would be good to document this, so test somebody else may try to reproduce the steps.