Optimize incremental builds #5969
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
contificate
approved these changes
Sep 4, 2024
lindig
approved these changes
Sep 4, 2024
|
@edwintorok There is a conflict now after merging another PR. Could you update this and then merge? |
Tell dune which directories don't contain dune files, so it can avoid scanning them. They can still be used as dependencies though. Unit test input data is particularly large for example. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
The 'source_tree test_data' dependency also applied to the test executable. However this is only a runtime dependency, so split the rule and express the more granular dependency. Now 'dune build @check' creates a _build with 9185 files. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
We'll run the longer version in quicktests, using the QCHECK_LONG_FACTOR. Now the test takes ~3s to run instead of ~20s. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Alcotest.check prints the message to stdout and flushes the formatter. This is called in a loop a million times, so this generates a lot of output, and is very slow. Use an empty message instead, which has an optimization in alcotest to avoid the formatter flushing, and set ~pos instead so that we get a unique error on failure. The test now takes 1.2s instead of 15s. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
The proxy and time limited read tests took ~10s, they now take ~2s. A longer test will be run in quicktest with QCHECK_LONG_FACTOR. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Use ~pos:__POS__ "" instead of __LOC__ in Alcotest.check. This only prints the position on failure, previously __LOC__ was printed on success too. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Port this commit to concur-rpc-test.sh: 62ff5e7 ("[maintenance]: reduce basic-rpc-test time") Reduces time from ~33s to ~10s. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Instead of comparing byte by byte use the more efficient comparison function that cstruct provides. This speeds up the test (from ~40s to ~33s). Signed-off-by: Edwin Török <edwin.torok@cloud.com>
edwintorok
force-pushed
the
private/edvint/buildopt
branch
from
cdf9d34
to
30f24ac
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This doesn't yet result in an improvement in koji build times, but it reduces incremental build times, by:
If you change source files that are in the dependency tree of these tests then a
dune runtestwill be quicker too.An uncached
dune runtestfrom a clean build takes ~1m33s now (a Koji build takes ~5m, so we spend more time unpacking, installing dependencies and packing than building).We could further reduce build times by reducing the amount of files that we package (most libraries are only used inside XAPI, etc.) but that will be the subject of a future PR.