Consider breaking up big Nexus 🔨

[smklein]

As @jgallagher helpfully showed during our weekly Rust study, cargo build --timings generates a nice report of build times.

From this report, there are some particularly bad offenders - among them, Nexus take 60+ seconds to compile.

By breaking Nexus into smaller crates, we could:

1. Get better incremental compilation results - @cbiffle mentioned this was handy for Hubris folks
2. Get better visibility in "what takes so long build". This could help us make further optimizations, and keep developer productivity high.

[smklein]

Using cargo rustc -p omicron-nexus --lib -- -Zself-profile, I generated an mm_profdata file which I passed to crox, using a --minimum-duration of 50000.

This generated a chrome_profiler.json file I've attached (Edit. I didn't attach it. Github is picky. Happy to help anyone reading along regenerate this though) - it can be loaded in chrome with chrome://tracing.

Lots of interesting stuff here:

• Analysis + type checking takes ~20 seconds
• Generating crate metadata takes ~10 seconds, most of that going to exported symbols (!)
• Codegen takes another ~20 seconds

[smklein]

This is a TL;DR of what it looks like to build the Nexus library on my machine, using the mold linker:

[ryaeng]

Sean, this looks interesting. My initial thoughts include breaking Nexus up into app, db, external_api, etc. Is that how you would approach this?

Also, I saw that monomorphization seems to be taking up a large chunk of time. Would we benefit from writing some of the larger functions from this...

fn add<T>(a: T, b: T) -> T where T: Add, { a + b }

to this...

fn add_usize(a: usize, b: usize) -> usize { a + b }

I don't have the vocabulary to be able to distinguish these two functions by name. If you could help me with the words used to differentiate these two, that would be most helpful. Thanks.