Compute referencedBy when its needed

[sheetalkamat]

This should fix #1520 with unnecessary calculation and storage for "buildInfoProgram" where we dont even need referencedBy files

In the incremental builds:

Metric	Before	After	Improvement
Memory allocations	149.7M	71.1M	-52.51%
Memory usage	9.48MB	8.39MB	-11.59%
BuildInfo read time	9.46s	0.95s	-89.98%
Changes compute time	6.27s	3.79s	-39.52%
Total time	48.13s	36.98s	-23.17%

Fixes #1520
Fixes #1541

[Copilot]

Pull Request Overview

This PR refactors the reference map implementation to compute referencedBy relationships only when needed, replacing the existing SyncManyToManySet with a custom referenceMap type that uses lazy initialization.

• Introduces a new referenceMap type with lazy computation of reverse references
• Replaces direct SyncManyToManySet usage with wrapper methods
• Removes the now-unused SyncManyToManySet collection type

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
internal/incremental/referencemap.go	New file implementing the referenceMap type with lazy referencedBy computation
internal/incremental/snapshot.go	Changes field type from SyncManyToManySet to referenceMap
internal/incremental/snapshottobuildinfo.go	Updates method calls to use new referenceMap API
internal/incremental/programtosnapshot.go	Updates method calls to use new referenceMap API
internal/incremental/buildinfotosnapshot.go	Updates method calls to use new referenceMap API
internal/incremental/affectedfileshandler.go	Updates method calls and removes helper method that's no longer needed
internal/collections/syncmanytomanyset.go	Removes the entire file as it's no longer used

[joshcartme]

This is huge and has made our cold and warm significantly faster. Thank you!