build: organise iojs.org/dist/

[rvagg]

Partial duplicate of #40 and nodejs/build#40 but I want to put it here as a v1.0.0 milestone.

I've gone with an iojs.org/download/ directory for now so we can experiment a little but TC wants a /dist/ directory organised roughly the same as nodejs.org/dist/ to help version managers and ease transition. I'm planning on doing this with symlinks for releases and leaving the nightlies in /download/.

• Need a catalogue for version managers, getting late but it would be good to have input from version manager authors like @ljharb, I'll try and have something up within the next few hours if I can manage it
• Need a latest symlink
• Need to GPG sign SHASUMS256.txt upon release (I don't want to provide a SHA-1 version of this like on nodejs.org and I don't think nightlies will be signed)
• Need to alias & symlink iojs.org/api/ to point to latest release docs/api/ directory

[zeke]

I would love to add iojs version support to semver.io, so Heroku users can use iojs in their builds, and we can display up-to-date iojs version info on various npm webpages.

Is the http://nodejs.org/dist/ directory structure a similar enough template against which to pre-emptively add iojs support to the semver.io codebase?

// @hunterloftis

[hunterloftis]

@zeke would love a PR for that, working on some related stuff now in buildpack.

So far I've been expecting something like this:
https://iojs.org/download/test/v1/iojs-v1.0.0-alpha20150107-linux-x64.tar.gz

(replacing test with release)

👍 @rvagg on mimicking the format of nodejs.org/dist since it'd make pulling from one or the other that much simpler.

[snostorm]

Hmm, I just decided to call the website's compiled asset folder (as per my un-approved PRs) dist. This actually isn't a conflict, it just might be confusing for 5 seconds for somebody puling the https://github.com/iojs/iojs.github.io project to see a dist folder there.

References: nodejs/iojs.org#26 & nodejs/build#31

Perhaps better go with ./build/ or ./public/ on the website instead? Or non-issue?

[rvagg]

the plan is to mirror /dist/ pretty closely with the following differences so far:

• at the top level, no nightlies, npm and patch directories (we'll be putting nightlies in /download/ where we can experiment with other formats)
• in individual directories:
  • the "x64" directory is now "win-x64" and there is a matching "win-x86" where only the iojs.exe and iojs.lib files will live
  • no SHASUMS.txt., only SHASUMS256.txt.
  • no 32-bit darwin build
  • there will be an armv7l linux build as well (possibly an armv6l too for raspberry pi but that's more complicated, working on it!)
  • no sunos builds
  • the windows x64 .msi is in top level
  • no .exp or .pdb files
  • no openssl-cli files

this is all being modelled in https://iojs.org/download/nightly/ if you want to see it live as it evolves

[jbergstroem]

Could we optionally also provide a .tar.xz?

/tmp $ du -h iojs-v1.0.0-nightly2015011284fa1f8c46.tar.*
 18M    iojs-v1.0.0-nightly2015011284fa1f8c46.tar.gz
 12M    iojs-v1.0.0-nightly2015011284fa1f8c46.tar.xz

[rvagg]

@jbergstroem I did think it would be great for us to migrate to .xz only but there's a desire (not so much on my part) to maintain compatibility with nodejs.org/dist/

This'll need a Makefile change unfortunately but I'll look in to it.

[jbergstroem]

@rvagg Instead of replacing, just do both?

Edit: Want me to do a pull request that checks for the xz binary and creates a tarball if it exists? The simplest possible solution would be to test for xz ($(shell which xz) or possibly wildcard to avoid subshell), do a similar compress as gzip, add -k to gzip/xz flags and rm the tarball once done. I guess removing .gz from the TARBALL variable input would be a more suiting target name as well.

Edit2: Pull request shortly.

[jbergstroem]

@rvagg make distclean should remove all build artefacts. How about moving generated stuff to out/?

[rvagg]

@jbergstroem perhaps, but I'd rather leave that for discussion by others with a longer history in the project, make a separate PR for distclean

[rvagg]

I've created index.json and index.tab files at the top-level of the nightly directory and will use the same format for /dist/. The code to generate these files will go into the iojs/build repo and will accept pull requests of course. I'd like feedback from version managers on this please. You have less than 24 hours before we go live with this and it'll be harder to change.

/cc @ljharb (and others?)

[rvagg]

updated comment, the index.tab link was wrong sorry

[ljharb]

index.tab looks like exactly what I need. It might be nice to also include the sha256sum in the same file?

[rvagg]

@ljharb how important is that? I've tried to keep the file relatively small in anticipation of lots of nightlies and lots of releases, the sha256sums are huge and you should be able to pull up the file for each one using the information in that file.

[rvagg]

Application used to generate these files is @ nodejs/build#32

[ljharb]

@rvagg it's not critical, since based on a version number I can get to the sum's text file.

[rvagg]

I believe this is ready to roll, basically the same format as nightlies except files don't have the -nightlyYYYYMMDDcommit on their names and the SHASUMS256.txt will be signed by my gpg key. I have /dist/ ready to link in to /download/release/

[snostorm]

For the "to do" list: let https://github.com/heroku/semver.io know this is live? They just merged in to master basic io.js support but using a workaround (since dist isn't live yet). See heroku/semver.io#26

[hunterloftis]

@snostorm is it live? I get a 404 at https://iojs.org/dist

edit: ah I see, you mean let us know when it goes live. Please do! :)

[snostorm]

@hunterloftis my understanding is the changes are queued up for deploying to the iojs.org server, but not yet in production. @rvagg is going to be the one putting it live. (There is a https://iojs.org/dist-test/ he just mentioned on IRC.)

I mostly cross-linked the issues so we remembered let you know it was live :)

[hunterloftis]

Nice, I'll hop on IRC.

[jbergstroem]

@rvagg Is there a reason the c-ares version was left out of index.json/tab? Guessing because it doesn't live in process.versions? I could probably fix that..

[rvagg]

@jbergstroem http_parser too, perhaps even node-gyp although it's a dep of npm ...

PR against https://github.com/iojs/build/tree/master/dist/dist-indexer and we can discuss

[rvagg]

I believe this is ready to roll, basically the same format as nightlies except files don't have the -nightlyYYYYMMDDcommit on their names and the SHASUMS256.txt will be signed by my gpg key. I have /dist/ ready to link in to /download/release/