Binary releases

[japaric]

Steps required:

@azerupi's share of work

• Add this project to your AppVeyor account. And let me know! Instructions, only steps 1 and 2.
• Generate an encrypted GitHub Personal Access Token for Travis and paste it here. Instructions
• Generate an encrypted GitHub Personal Access Token for AppVeyor and paste it here. You can reuse the token you got in the previous step. Instructions
• Decide what will be the contents of the release tarball (other than the mdbook binary). mdBook has to comply with the MPL-2.0 license, and the license dictates that:

You must inform recipients of the Executable Form how they can obtain a copy of such Source Code Form by reasonable means in a timely manner, at a charge no more than the cost of distribution to the recipient.

So, I guess that the release tarball must include a link to this repository somewhere, you could add the link to the output of mdbook --help or add the link to the README and ship the README in the release tarball. I leave this decision to you.

• Decide which targets test/release for. Here's the list of targets supported by rust-everywhere:
  • i686-apple-darwin (32-bit OSX)
  • i686-pc-windows-gnu (32-bit Windows, MinGW)
  • i686-pc-windows-msvc (32-bit Windows, MSVC)
  • i686-unknown-linux-gnu (32-bit Linux)
  • x86_64-apple-darwin (64-bit OSX)
  • x86_64-pc-windows-gnu (64-bit Windows, MinGW)
  • x86_64-pc-windows-msvc (64-bit Windows, MSVC)
  • x86_64-unknown-linux-gnu (64-bit Linux)
  • x86_64-unknown-linux-musl. (64-bit Linux, statically linked binaries)

Recall that more targets means longer CI testing time. AppVeyor is particularly slow because it doesn't do parallel build jobs (i.e. one target at a time).

@japaric's share of work

• Set up testing 32/64 bit OSX on Travis CI Travis CI: expand to test and deploy for Linux and Mac #118
• Set up testing 32 bit Linux on Travis CI Travis CI: expand to test and deploy for Linux and Mac #118
• Set up testing 32/64 bit Windows on AppVeyor set up AppVeyor to test and deploy on Windows #117
• Set up deployments on Travis and AppVeyor set up AppVeyor to test and deploy on Windows #117 Travis CI: expand to test and deploy for Linux and Mac #118

When all that is done, every time a new git tag is pushed the CI infrastructure will build a new binary release and upload it here. Then it's @azerupi's turn again:

• Push a release candidate version, e.g. v0.0.11-rc1, to test that the deployment works
  correctly.
• Push a non-rc tag, e.g. v0.0.11, and inform the world 🚀.
• Celebrate 🍻.

@azerupi You can ask me any questions here or on mozilla IRC network my handle is japaric.

[azerupi]

Travis:

Z1k7WqX7z+tT4+SzTh4tBBzf11VaADB4AWuEczHtylaEb/0hRs8gaiHCNSVHm/QTp0QPWQR2Vw7uKMhVuxG7I8X7h31j3A7ulYBh/iVk0DVIrtrn2Q4WOED9CpoXLuLtk2nxo9MBViFW7mw4nJe9H2Tn9o/9oEYBuwzekvW5mh4muqUuCVTr8eQVYbs3jbC9pQy5oYjOLeUnlL9Cey5VN/nAhzAtyFP+6lIMri0PKit4JtkFou/O1MEpFYlP3VGC2lFiWuByocPKBT/L45FecS9qoHq+i6+ZCPDH2eu46nuYsDbLKAkPdGvf1MdPBPwoj0vSnZbgaTisQ4hIoBngQQQPZlPaGtcdd6g6asxSfnbA9cQhClI5oZJmg+ksxQE+peE8pnbmZ10Ix0PpIkkfWdQeMdUUCQarOTkTK54Munw+X+kp1lH19j6+krQPLBYr95fPRd4b5tWsJD2+pb/UOYFEEJxMNoUHyLCrtdCO7imOwrSUcv51+Z8UudqfPpKQeszrJcntL4owip35r3sF5TsE9YfW5qssLC164IylvP32y1AcfL1jqg8b+zrqLZKanjvDOJ1dtHHuwKqxcwf7PhAf0YjAtVSH9OIYcDzmDa0EMLrq7EK0fs6NAeb5qt6CML7pZrRS3fmOxN53Fbmj81qm6TmjQjDe4dmZlELgNow=

AppVeyor:

QQhjKVyz7mpjlyGhlXytbFQQfKFQWTahHkD+B0NzIUoEVqO7ZLWjnoWasvLqW4nE

I don't think that there should be anything else besides the binary in the tarball, actually do we need to make a tarball at all in this case, can't we just ship the binary directly?

Adding a link to the GitHub repository at the end of mdbook --help sounds like a good idea 👍

Thanks for doing this! :)

[japaric]

actually do we need to make a tarball at all in this case, can't we just ship the binary directly?

Each target will upload a binary and each binary must be named differently or one upload will overwrite the other. We could name them using the target triple as a suffix: mdbook-x86_64-unknown-linux-gnu, mdbook-x86_64-apple-darwin, etc. Or use some custom naming: mdbook-linux64, mdbook-win32gnu, mdbook-win32msvc. Thoughts?

I personally like the naming convention the Rust releases use: mdbook-v0.0.11-x86_64-unknown-linux-gnu.tar.gz :-).

I forgot to ask which targets you want to release binaries for. I'll add the list to the top comment.

[azerupi]

Each target will upload a binary and each binary must be named differently or one upload will overwrite the other.

Right, in that case the tarball makes sense to have the same name for binaries independent of version and target.

I forgot to ask which targets you want to release binaries for.

As many as possible? 😉
I would say at least the 3 big ones: Windows, OS X and Linux
Do we need to make different tarballs for 32/64 bits?

[japaric]

Right, in that case the tarball makes sense to have the same name for binaries independent of version and target.

😄

As many as possible?

Sure. I've added the list of targets to the top comment. There a 9 targets. Let me know if you want to "opt-out" of any of them. I don't really know what's the difference between the gnu and msvc variants on windows land, but hopefully the windows user will know the difference 😃.

Do we need to make different tarballs for 32/64 bits?

Yeah, 32-bit and 64-bit need to run as different build jobs so they end in different tarballs.

[azerupi]

I don't really know what's the difference between the gnu and msvc variants on windows land

I don't know either, but isn't having all options going to be confusing for users (if they don't know the difference)?

Also what's the difference between gnu and musl linux? You say "libraries statically linked", but Rust already links statically by default right?

[japaric]

I don't know either, but isn't having all options going to be confusing for users (if they don't know the difference)?

I asked on IRC, and the Rust on windows expert recommends msvc. :-)

Also what's the difference between gnu and musl linux? You say "libraries statically linked", but Rust already links statically by default right?

rustc links statically to Rust libraries by default, yes. But it dynamically links to C libraries by default. You can check this on mdbook itself:

$ cargo build --target x86_64-unknown-linux-gnu

$ file target/debug/mdbook
target/debug/mdbook: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), DYNAMICALLY LINKED, interpreter /usr/x86_64-pc-linux-gnu/lib/ld-linux-x86-64.so.2, for GNU/Linux 2.6.34, BuildID[sha1]=f1d6fb45f51f0c7dbc943c714997475ac4db215c, not stripped

$ ldd target/debug/mdbook
        linux-vdso.so.1 (0x00007ffd7af3e000)
        libc.so.6 => /usr/x86_64-pc-linux-gnu/lib/libc.so.6 (0x00007f9786a8a000)
        /usr/x86_64-pc-linux-gnu/lib/ld-linux-x86-64.so.2 (0x00007f9786e2e000)
        libm.so.6 => /usr/x86_64-pc-linux-gnu/lib/libm.so.6 (0x00007f9786794000)
        libdl.so.2 => /usr/x86_64-pc-linux-gnu/lib/libdl.so.2 (0x00007f9786590000)
        libpthread.so.0 => /usr/x86_64-pc-linux-gnu/lib/libpthread.so.0 (0x00007f9786373000)
        libgcc_s.so.1 => /usr/x86_64-pc-linux-gnu/lib/libgcc_s.so.1 (0x00007f978615d000)
        librt.so.1 => /usr/x86_64-pc-linux-gnu/lib/librt.so.1 (0x00007f9785f55000)

That binary can only run on Linux systems that use glibc.

On the other hand compiling against musl produces a statically linked binary:

$ cargo build --target x86_64-unknown-linux-musl
file target/x86_64-unknown-linux-musl/debug/mdbook
target/x86_64-unknown-linux-musl/debug/mdbook: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), STATICALLY LINKED, BuildID[sha1]=5456d7616dc525b1fbbbdeb25c97f02c43eaebff, not stripped

This binary can run on any Linux system regardless of the libc it has installed. The tradeoff is that the musl binary tends to be bigger in size.

[azerupi]

the Rust on windows expert recommends msvc

MSVC it is then 😉

This binary can run on any Linux system regardless of the libc it has installed. The tradeoff is that the musl binary tends to be bigger in size.

I guess it's worth having both then, Linux users tend to know more often what they need.

[japaric]

The releases seem to work \o/. I tested the windows release under wine and mdbook.exe init && mdbook.exe build works correctly.

[azerupi]

Yes! That's great news 👍
Thanks a lot for setting this up!

[japaric]

FYI, rsnotify 2.5.5 should be build-able for the i686-apple-darwin target. If you want to extend the test/release matrix, let me know.

[azerupi]

Yes! please 😉
That would complete the set of binaries for the major platforms

[azerupi]

Sorry to bother you, but I have a doc-test that is failing on only some platforms:

https://travis-ci.org/azerupi/mdBook/builds/118838692

But I have no clue as to what I am doing wrong, do you have an idea?

[japaric]

Short answer:

Rewrite your test as:

    /// ```
    /// # extern crate mdbook;
    /// #
    /// # fn main() {
    ///     let author = mdbook::config::Author::new("John Doe").set_email(Some("john@doe.org"));
    /// # }
    /// ```

Note: closing the doctest fence ('```') and space between the# and the start of code.

There was an issue (rust-lang/rust#31907) with doctests and --target; they basically don't work together. This has been fixed in the nightlies by making cargo ignore the doctest when the --target flag is passed. I "backported" this fix to stable and beta by marking all doctests as ignored (with this sed script) when cross compiling.

Now the problem is that for some reason the transformed doctest:

    /// ``` ignore,
    /// #extern crate mdbook;
    /// #
    /// #fn main() {
    ///     let author = mdbook::config::Author::new("John Doe").set_email(Some("john@doe.org"));
    /// #}
    ///

doesn't get ignored even though is marked with ignore. This seems like a rustdoc bug. The problem goes away if you rewrite the test as shown at the beginning of this comment. weird

[azerupi]

I didn't even notice I was missing the closing fences, ugh..

I am a bit surprised that a doc-test without closing fences passes though, if it would have failed on every platform I would probably have noticed. But since it was only failing on a couple of platforms I assumed something weird was going on with the test matrix.

Thanks for the help 😃