[FEA] Improve build time

[wphicks]

Creating this issue to help track recent discussion and work on improving cuML's build time. This is really a subset of #3026, but there's been enough conversation specifically about build time that it seemed like it warranted its own space.

There are at least 3 parts to this as I see it right now:

1. Ninja: With PR [REVIEW] Use cmake --build in build.sh to facilitate switching build tools #3487, it is now relatively easy to switch from make to ninja locally, but additional work would need to be done to take advantage of that in CI.
2. ccache: Using ccache locally has been straightforward for awhile, but again, this would be more difficult to introduce in CI.
3. Reducing unnecessary includes: Some header files are being unnecessarily or redundantly included, substantially increasing parsing time during builds.
4. Eliminating redundant compilations: Due to extensive use of templates, we are recompiling certain sections of code many times. For instance, MetricProcessor<float> gets recompiled 69 times, according to my most recent analysis. Using extern template declarations and other techniques to reduce redundant compilations may substantially decrease build time even from a "cold start" (no caching) and regardless of build tool.

Please feel free to add any additional areas for potential improvement to this list. This issue should also be used to compile profiling data and analysis to guide work on build time down the line.

[dantegd]

Both items 1 and 2 are being looked by ops RAPIDS wide by @dillon-cullinan if I'm not mistaken, just leaving this for reference

[github-actions]

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

[wphicks]

I'm going to open a new issue for this since the problems that seem to be affecting us the most have changed a bit since we started to investigate this.