Periodic reset of spanmetricsprocessor metric maps

[albertteoh]

Is your feature request related to a problem? Please describe.
The maps storing the metrics data in the spanmetricsprocessor will grow indefinitely, hence leading to growing memory usage, and felt more with high cardinality dimensions in metrics. It's worth noting though, that with low-cardinality dimensions, this growth should follow a logarithmic pattern.

See: #2012 (comment)

Describe the solution you'd like
Would like to understand what is the idiomatic way of solving this problem of preventing ever-growing metric dimension sets and continuing disjoint time series data.

Describe alternatives you've considered
A periodic reset of these maps was tested but led to metric values jumping around and metrics themselves appearing and disappearing in Prometheus.

An alternative approach to resetting all keys in the map is to adopt take an LRU cache approach and setting a limit on the number of keys and evict the oldest keys. However, this still doesn't solve the problem of joining the timeseries if these evicted keys are populated with data points again; though the benefit of this approach is to only expire keys where necessary.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• processor/spanmetrics: @albertteoh

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[albertteoh]

This was done in #6503.