DependencyIDs create a lot of long, duplicate strings

[panopticoncentral]

I don't have exact numbers, but looking at the GC heap of a large CPS solution, there are a lot of duplicate strings showing up that are DependencyIDs. The problem is that DependencyIDs tend to be very long and are generated on the fly, so common child IDs end up wasting a lot of space. (In a solution with 520 projects, one common dependency ID is duplicated 4k times and wastes nearly 1Mb all on its own.)

We should find a way to intern these so that we're not duplicating them all over the place.

[davkean]

Do we need to create these ids at all? @drewnoakes Do you know what we use them for?

[panopticoncentral]

A fair question... The big waste is the fact that Dependency objects use the IDs to connect parent/child Dependencies. If there's no reason to have things be disconnected, direct references would be much more efficient.

[drewnoakes]

It may be possible to avoid creating them altogether for graph nodes (which allow composite IDs). I took a stab at that a long while back but didn't quite get it working. I'm more familiar with the code now so may have better results. I can take a look, probably in the next sprint.

[davkean]

I would imagine that's where the majory of duplicated strings would be created, but I do wonder how they are being created if they are created on demand.

[drewnoakes]

This situation has been improved significantly in #6155. Pending task is to inspect new heaps and see whether this is still an issue.

[drewnoakes]

The big waste is the fact that Dependency objects use the IDs to connect parent/child Dependencies.

Since #6155 this is no longer the case.

Analysis of recent dumps shows that dependency IDs are no longer contributing in a significant fashion to heap consumption.

[panopticoncentral]

So awesome! Thanks so much for your hard work on this!

[drewnoakes]

#6276 further reduces the size of the dependency IDs by removing the TFM prefix from all of them.

[drewnoakes]

#6282 eliminates composite dependency IDs altogether.