feat(v2): metadata string interning

[kolesnikovae]

The change implements string interning in the metadata model (API) and in the index persistence layer. It introduces an option to store sharable shard-level assets in the index.

This change allows the index snapshot size to decrease by 5-10 times. In an environment with 1-2K services and the default segmentation and compaction parameters, we can now expect approximately 650-800KB of data to be added to the index daily per shard. In a deployment with 128 shards (roughly corresponding to 1GB/s of ingested traffic), the snapshot size will accumulate to 2.5-3GB of data (at rest).

Future work should focus on index snapshot compaction. Currently, the snapshot contains all partitions, including ones that are effectively immutable (due to ingestion window restrictions). We should exploit this fact. This is the reason why I changed the way partitioning is handled and removed the configuration option that controls the partition duration: it will be much more difficult to handle if partition are overlapping. Instead, these should be "immutable" – once created, a partition includes all the entries with corresponding timestamps, regardless of the current partition duration.

In the screenshot, you can see metrics from the dev environment with 12 shards; the deployment is 3 days old:

[simonswine]

LGTM

While I have not fully tested this change, I went through all the changes.

I think your assumptions about the immutability of the partition duration is a worth while one. Thanks for refactoring this and this seems to be a great improvement to the snapshotting.