Outdated nightlies on Travis-CI

[SimonSapin]

@brson, is Cloudfront (that we seem to be using for HTTPS) doing any caching? I’m somehow getting an outdated Cargo nigthly (2014-08-19 instead of 2014-08-20) on Travis-CI https://travis-ci.org/servo/rust-url/builds/33162181

The other suspect is Travis, if they’re doing caching themselves.

[sarahhodne]

Henrik from Travis CI here. We're not doing any kind of caching on this URL, my initial guess would be that something is going wrong somewhere in the CDN.

[brson]

My understanding is that cloudfront syncs every 24 hours. Unless I figure out how to make it sync with s3 more quickly I intend to just drop cloudfront. I'd love somebody to tell me which knob to frob.

[brson]

Cloudfront gives you 1000 object invalidations a month for free, so I'll look into having the bots do that automatically.

[brson]

@alexcrichton modified buildbot to invalidate cloudflare on every s3 upload of the binaries (though not docs I think). This should make the CDN sync nightlies immediately.

[brson]

@alexcrichton would you call this done?

[alexcrichton]

I haven't yet confirmed this, but yeah I'll close and we can reopen if it looks to be a problem still. I think it's done at least!

[SimonSapin]

This doesn’t seem fixed, sorry :/ I just updated my nigthly from https://static.rust-lang.org/dist/rust-nightly-x86_64-unknown-linux-gnu.tar.gz and got c669411 2014-09-23, which according to http://buildbot.rust-lang.org/builders/nightly-linux is two days behind.

[SimonSapin]

Updating again this afternoon I got 2014-09-24, which is strange. I expected either 2014-09-23 again or 2014-09-25.

[alexcrichton]

Just did a bunch of testing, yes this isn't fixed. Took me longer than I should have to find this, but --cf-invalidate is only valid for the sync command, not the put command (which we're using). Yay docs!

[alexcrichton]

I believe this could be fixed with a solution to #14431. We could use the --cf-invalidate flag with a sync to update the dist folder wholesale

[alexcrichton]

I have set up a script to perform an invalidation every morning at 7am UTC-5 (I have no idea what time zone that is!). This isn't exactly guaranteed to produce optimum results, but it looks like our builders normally finish by ~4am UTC-5, so hopefully it'll be "good enough" for most.

Put another way, I would say that this is patched over enough for now that I wouldn't close it, but I would consider it "mostly fixed". If you have an issues, please let me know!

[SimonSapin]

(East coast US is in UTC-5 when daylight saving time does not apply. 1, 2.)

I just updated to adb44f5 2014-10-12 which is one day behind, but it’s still ~11pm UTC-5 right now so I’ll try again later today.

[SimonSapin]

I’ve been constantly getting up-to-date nightlies lately. It think this can be closed.

[alexcrichton]

Ok, thanks @SimonSapin for watching this!

[shadowmint]

Sorry to say, this isn't licked quite yet.

doug@Tio /c/projects/cargo
$ ls -l
total 4683
drwxr-xr-x 6 doug Administrators    4096 Dec  2 10:14 cargo-nightly-x86_64-pc-windows-gnu
-rw-r--r-- 1 doug Administrators 4790495 Dec  5 22:42 cargo-nightly-x86_64-pc-windows-gnu.tar.gz

doug@Tio /c/projects/cargo
$ ./cargo-nightly-x86_64-pc-windows-gnu/bin/cargo.exe --version
cargo 0.0.1-pre-nightly (da789a6 2014-11-30 08:14:16 +0000)

ie. Cargo for windows is currently lagging 6 days. The buildbot status is fine:
http://buildbot.rust-lang.org/builders/nightly-cargo-win-64-64

Or at least it looks fine.

Anyway, this is tangibly causing failure on windows machines, see: http://www.reddit.com/r/rust/comments/2oblow/compiling_libtime_on_windows/

Possibly also related: time-rs/time#28

Do you want to open a new bug about this, or leave it here?

[SimonSapin]

@shadowmint Do you get different version when downloading from https://static.rust-lang.org/ v.s. https://static-rust-lang-org.s3.amazonaws.com/ ? (The former is a caching CDN in front of the latter, this issue was about invalidating that cache.)

[shadowmint]

@SimonSapin The files in both versions seem identical (and old). Perhaps the issue is indeed s3 being updated, rather than the CDN?

[SimonSapin]

I just downloaded nigthly 8bca470 2014-12-08, even though http://buildbot.rust-lang.org/builders/nightly-linux shows c38e73f as the latest :(

Same thing on Travis-CI: https://travis-ci.org/servo/rust-url/builds/43648307

Ping @brson

[netvl]

Same thing with mac version. However, according to the buildbot it wasn't even uploaded due to failed synced dist builds check: it says that there is no linux build, while it is really there - last linux build was fully successful.

[netvl]

It seems that nightlies went through. Just have updated rustc successfully:

rustc 0.13.0-nightly (193390d0e 2014-12-11 22:56:54 +0000)

[brson]

Nightlies were not being produced for several days because we accidentally snapshotted a build of rust that was incompatible with the mac nightly builders. Nightlies do not update now until all platforms pass, which is why the linux builders appeared to produce nightlies but they weren't available.