Caching layer for the update pipeline

[roji]

Our perf focus in EF Core 7.0 was on the update pipeline, and specifically reducing database roundtrips and emitting better-performing SQL (#26797 (comment)). Specifically, we did not focus on runtime performance (i.e. reducing allocations and CPU time in the code itself).

While there are definitely opportunities for runtime optimizations in the update pipeline, it may be better to look into caching layer above the entire pipeline; SaveChanges involves a lot of work in order to produce the actual SQL commands, and it would be better to avoid that entirely, so that it's only performed the first time a particular kind of SaveChanges is performed. This would be similar to how the query pipeline works: the compilation process is heavy and not hyper-optimized, but it only happens once for a given query tree, and the results are cached for subsequent use.

The problem is of course the construction of a reliable, correct cache key encompassing the entire SaveChanges. If not feasible, we may be able to solve this problem for a common subset of cases (e.g. single-row SaveChanges only).

[AndriySvyryd]

Also, due to batching the generated SQL space can grow virtually unconstrained. To avoid polluting the cache we could consider doing some of these:

• Don't cache commands that can grow (like insert)
• Extract a high-level pattern that doesn't change between the command and cache that instead
• Only put the command in the cache after the 2nd time that a given cache key was created