Add nightly/cronjob CI

[Manishearth]

While we shouldn't run every PR on this, we should set up a "nightly" CI that:

• runs every night
• runs the normal CI build (but on the latest Rust nightly). Warnings are disabled (i.e. the clippy job is skipped)
• runs additional jobs like make full-data, or running clippy with the pinned nightly, or whatever.

I think the way to do this would be to:

• Update all of cargo-make to use cargo-make env vars. Allow it to be overridden with a real world NIGHTLY_RUST_OVERRIDE variable
• Make all other users of nightly rust in this repo respect this key (it's one gnu makefile and one rust script)
• Use on.schedule.cron = "/7 * * * * " or whatever in build-test.yml
• Every step in build-test.yml starts off with a step that:
  • checks for the on.schedule.cron variable, and if true:
    • sets an environment variable called NIGHTLY_RUST_OVERRIDE
    • rustup override nightly
• Additionally add a nightly-only.yml that contains full-data (and other stuff?)
  • We could also just include this in build-test.yml and make the specific task conditional on cron.

[sffc]

Another way to configure it would be a webhook that gets triggered on a periodic basis or whenever there's a new Rust Nightly. There are lots of ways we can set up a webhook.

[Manishearth]

Hitting sagiegurari/cargo-make#722 when I try to do an override situation

[robertbastian]

Actually why nightly? Seems like we should be testing on beta, which prepares us for the next stable release, but isn't as unstable as nightly.

[Manishearth]

@robertbastian immediate notification of breakages so we have time to report stuff upstream and get it fixed if we need without asking for uplifts.

[Manishearth]

I want to be using the most unstable thing here.

Also people will use icu4x from nightly and I want it to work as much as possible. It should, nightly is reasonably stable now, but there are still occasional moments especially since we do some weird stuff.

[robertbastian]

Just as a data point, 39 out of 78 nightly CI runs have failed. Given those numbers the bot might be a bit spammy.

[Manishearth]

@robertbastian well, most of that was before we had the notifier, so we had a breakage stay broken for quite a while. We've only had two total breakages in those 78 runs, just that one was out of our hands to fix and we had to wait a while, and the other went unnoticed for a while. 2 out of 78 isn't bad.

I'm also fine with the bot being that spammy, I want to fix things when they break. The cost is just a message in Slack.

Basically we have two kinds of breakages: ones we can fix and ones we can't. For the ones we can fix hopefully they get fixed before the next time the bot complains (but otherwise they'll be fixed at most a couple complaints later), it's really the other ones that are the problem but rustc tends to take less than a week to fix those and I'm fine with that.