zig build system: cache build artifacts and detect cache invalidation

[andrewrk]

migrated from #204

• in zig itself, cache objects by creating a sha256 hash of all the inputs that goes into it, such as the list of source files used with their mtimes, the compile variable values set, the modification time of the zig binary itself as well as the recursive modification time of the dynamic libraries the zig binary depends on. Verify that builtin.o, compiler_rt.o, test_runner.o, and all objects made with zig only have to be created when zig is recompiled or when their sources are changed.
  • @embedFile should add the target to the hash
  • c_import should detect when a file is looked at by libclang and add that to the hash
  • various command line arguments may need to go into the hash
• in zig itself, cache executables built with a similar strategy. Additionally hash the list of dynamically linked libraries
• in zig itself, keep cached objects in ~/.zig/cache/. Let's have a file ~/.zig/config which has an option where you can set the maximum size of the LRU cache, with some nice default like 100.
• cache build.zig -> ./build executable using this strategy
• for compiling c objects, use the feature of c compilers that computes dependencies to get a list of files that are dependencies, and then avoid unnecessarily recompiling C objects. Also put into the hash environment variables, other build options, as well as the modification date of the detected compiler, and its recursive dynamic libraries. You should be able to modify the build.zig file and then, provided that the same options were chosen for compiling a given C object, the step does not needlessly be repeated.

[dumblob]

Just a note to the ~/.zig/ directory. I would strongly recommend following XDG [1] [2] variables and only if they're not set create and use this directory. On my system it looks e.g. like this:

$ env | grep XDG
XDG_PUBLICSHARE_DIR=/home/my_user/
XDG_CONFIG_HOME=/home/my_user/.config/
XDG_DOCUMENTS_DIR=/home/my_user/doc/
XDG_VTNR=1
XDG_DOWNLOAD_DIR=/home/my_user/
XDG_SESSION_ID=c1
XDG_VIDEOS_DIR=/home/my_user/video/
XDG_TEMPLATES_DIR=/home/my_user/
XDG_MUSIC_DIR=/home/my_user/audio/
XDG_DATA_DIRS=/usr/local/share/:/usr/share/:/home/my_user/.local/share/
XDG_PICTURES_DIR=/home/my_user/pics/
XDG_SEAT=seat0
XDG_CACHE_HOME=/home/my_user/.cache/
XDG_RUNTIME_DIR=/run/user/1000
XDG_DATA_HOME=/home/my_user/.local/share/
XDG_CONFIG_DIRS=/etc/xdg/
XDG_DESKTOP_DIR=/home/my_user/

[andrewrk]

Good call, using the XDG directories.

[andrewrk]

See the updated cache explanation here: #298 (comment)

[Earnestly]

Just to add a quick note here, zig doesn't quite follow XDG Base Directory Specification (XBDS) and the commenter who dumped a load of non-XBDS environments probably didn't realise it.

Instead of checking the XDG_DATA_HOME environment, zig hardcodes HOME/.local/share. This is the required fall back if XDG_DATA_HOME is not set (and not an absolute path).

Lots of information about the XBDS here: https://wiki.archlinux.org/index.php/XDG_Base_Directory