Build determinism in CI

[kylefleming]

Motivation

Cargo build caches build artifacts so that successive invocations won't rebuild the parts of the code that haven't changed. Currently our project is experiencing spurious rebuilds even when nothing has changed. This leads to slower development cycles.

On top of the spurious rebuilds, the build is also experience build non-determinism. Crate feature selection doesn't appear to be consistent across multiple successive builds under certain circumstances (e.g. when building more than a single package at a time). This is a problem because it means that we might be encountering bugs related to incorrect feature configuration. This also has the potential to mask bugs for the same reason.

In this PR

This PR addresses several of the issues causing spurious rebuilds and build non-determinism:

• mc-util-build-enclave emits rerun-if-changed directives based on files in the filesystem. However, currently the files aren't sorted, leading Cargo to think that the build inputs have changed, thus invoking a rebuild. This PR adds sorting to the file iteration so that they're always printed in the same order.
• Cargo's new resolver feature allows isolating feature unification between build dependencies, target dependencies, and dev dependencies. However, there is a bug within Cargo stemming from overeager caching of feature configuration, leading to incorrect feature selection under certain circumstances. See MC-1731 and Non-deterministic builds with new resolver rust-lang/cargo#8549 for more detail. This PR adds the suggested workaround to our CI builds, which is to add --target when invoking cargo.
• Additionally, there is a similar issue when running cargo test with the default settings, where feature unification appears to be non-deterministic. A workaround for this is applied to CI, which is to add --tests when calling cargo test. Typically, when calling cargo test with the default settings, cargo will build the project in a number of different ways (see https://doc.rust-lang.org/cargo/commands/cargo-test.html), and then test the tests. By specifying --tests, Cargo is instructed to build only the tests, and then run them. This prevents feature configuration from other types of builds from being impacted by this bug and affecting the feature selection of the test build.

Future Work

• Once Non-deterministic builds with new resolver rust-lang/cargo#8549 is fixed, then we should be able to remove --target and --tests from the CircleCI configuration. MC-1731 has been created to track the status of this bug.

[eranrund]

Nice!