Reuse LTO products for incremental builds when deps are unchanged

[fenollp]

Incremental builds with LTO (thin or fat) always take the same amount of time (~10s on my project) when dependencies are unchanged. It seems to take as long when adding a dependency.

Is it not possible to cache (at least part) of the LTO computations?

Note I'm using cross:

# cross version
cargo 1.44.0-nightly (8751eb301 2020-04-21)

Note also this bug I've encountered WRT LTO: cross-rs/cross#416

#71248 is the most recent issue I could find that seems related.

[pnkfelix]

To be clear: You're describing a scenario where your current crate is changed, but the dependencies are unchanged, right?

[fenollp]

@pnkfelix Yep: changing some of the crate's code while deps are unchanged (LTO setting unchanged) takes a while to compile (incrementally).

[bjorn3]

For fat LTO caching is not possible. Fat LTO basically works by merging bitcode for all codegen units into a single llvm module and then optimizing. For ThinLTO afaik we already reuses all optimized codegen units which LLVM tells us can be reused:

rust/compiler/rustc_codegen_llvm/src/back/lto.rs

Lines 549 to 558 in 33a2c24

	// If a module exists in both the current and the previous session,
	// and has the same LTO cache key in both sessions, then we can re-use it
	if prev_key_map.keys.get(module_name) == curr_key_map.keys.get(module_name) {
	let work_product = green_modules[module_name].clone();
	copy_jobs.push(work_product);
	info!(" - {}: re-used", module_name);
	assert!(cgcx.incr_comp_session_dir.is_some());
	cgcx.cgu_reuse_tracker.set_actual_reuse(module_name, CguReuse::PostLto);
	continue;
	}

[fenollp]

Thanks for your clear and documented answer!

Fat LTO basically works by merging bitcode for all codegen units into a single llvm module and then optimizing.

I see how caching is not possible. Then how about changing how FatLTO is done? Instead of merging+optimizing all in one step, shouldn't we be doing this with various subsets of the crates tree? I'm thinking walking the dependency tree from the bottom up, this way parts of the tree that weren't touched can be identified and reused.

[bjorn3]

Then how about changing how FatLTO is done? Instead of merging+optimizing all in one step, shouldn't we be doing this with various subsets of the crates tree?

What is the benefit of that over ThinLTO?

this way parts of the tree that weren't touched can be identified and reused.

You can still have optimizations that are affected by parts of the call graph that did change. That is the point of LTO.