Introduce partition digests to avoid recomputing all rules for every build

[kodek16]

This is an optimization that may be useful for users when they work on an application that has a lot of source dependencies, and recompile it after every change. Intuitively, in this case there is no need to rebuild the whole dependency graph all the time, since dependencies sources are not likely to change. The PR introduces partitions and partition digests to facilitate faster rebuilds after changes limited in scope.

A partition is usually a dune project, but for some targets that cannot be easily mapped to a project, separate target-specific partitions are created.

Partition digest should be used to determine whether rules for partition should be recomputed. If there were no changes in both partition files (in case of a single-target partition, there are none) and dependencies digests, then it's possible to skip rule generation phase altogether.

[kodek16]

Feature-wise this should be fairly complete and ready for review. Right now partition cache is enabled for watch mode, and can be enabled for normal builds with --use-partitions flag.

Some rough benchmark data from the testing environment I've been using:

Hardware I used included an 8C/16T CPU. The workspace contained sources of Core, Async, Jenga, and their dependencies (~7200 rules as per dune rule counter). The testing executable depended on all of them, its code was a single print statement. I've been changing the print statement code to see the difference between rule generation time and the executable build time.

Without partition cache:
First build: 68.18s
Next builds (executable code changed, compiles): 1.43s
Next builds (executable code changed, errors): 1.24s
Next builds (no changes): 1.21s

With partition cache:
First build: 69.56s
Second build (required to complete partition cache): 2.57s
Next builds (executable code changed, compiles): 0.49s
Next builds (executable code changed, errors): 0.27s
Next builds (no changes): 0.24s

[kodek16]

Closing this as a more comprehensive and robust solution is proposed in #1280.