SCons : Add scu_limit argument

[lawnjelly]

"scu_limit" allows specifying the maximum number of includes in a single SCU file (translation unit). A lower limit (e.g. 8) uses less RAM during compilation, but may be slower to compile.

Explanation

The max number of includes in a SCU file (which is a single translation unit used by a single compiler instance) determines very roughly the maximum RAM used by the compiler instances. RAM is important to not let get out of hand, particularly in release builds (which use a lot more RAM), so the defaults are for 1024 maximum includes in dev builds (to maximize compilation speed) and 8 in release (to get some acceleration, but limit RAM use to approx 1Gb per compilation instance).

The total RAM used by SCons approximates to:
number_of_cores * RAM_per_core
So a more powerful machine with more RAM and cores may not always be able to safely increase scu_limit if it has a corresponding large number of cores (e.g. 64Gb machine with 32 cores = 2Gb per core, same as 8Gb with 4 cores).

Although these defaults are fine in most cases, there are a couple of situations where it is nice to be able to manually fine tune the RAM usage:

• On computers with large amounts of RAM, for faster compilation in release
• On CI runs, where cores and total RAM may be limited

Particularly the CI runs RAM may be more limited, so it may be sensible to set scu_limit for DEV builds to a value closer to the release default (8 or 16 or so, rather than the default 1024). This may require some testing to get the best value for scu_limit to use on a particular system.

Notes

• As always, happy to change the name of the parameter etc if we can think of something better.

[akien-mga]

Thanks!