You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now, the full benchmarks get generated each time we push to master. This is inconvenient, because the CI takes a long time to complete. It is also useless, because we already have testing that should catch errors.
What we'd really like is to only run the benchmark jobs (and update gh-pages with the data) once per day. In theory, we could use a service like nighli.es to achieve this, but I don't think it is possible to know from the CI script that we're being launched by the service. Hence, we wouldn't know when to actually run the benchmarks.
The text was updated successfully, but these errors were encountered:
I tried using Nightli.es, but there is no way to know that we're being run as a nightly build. So I've set up a basic Heroku app which performs the following command every night:
This uses the Travis CI API, as documented here, to start a build. I tried several different ways to make the script aware that it's being run in a nightly build, but the only thing that worked was to set METABENCH_NIGHTLY_BUILD=true explicitly. It's hacky, but it works and I documented the hack properly so it does not bite us in the future. If nightli.es gets the ability to set an environment variable, we'll be able to easily change to it and get rid of the Heroku app.
So the benchmarks should start getting updated tonight. The build will fail for Clang 3.8 because of travis-ci/apt-source-safelist#255, but everything should work just fine otherwise.
Right now, the full benchmarks get generated each time we push to master. This is inconvenient, because the CI takes a long time to complete. It is also useless, because we already have testing that should catch errors.
What we'd really like is to only run the benchmark jobs (and update gh-pages with the data) once per day. In theory, we could use a service like nighli.es to achieve this, but I don't think it is possible to know from the CI script that we're being launched by the service. Hence, we wouldn't know when to actually run the benchmarks.
The text was updated successfully, but these errors were encountered: