Parameterize memory size and some key improvements

[hanahmily]

These changes introduce several parameters to control memory usage. I assigned some defaults to them, and now the initial memory is around 2GB 🎉

There are a series of improvements:

• Remove run.Chan, which could cause the data race. We don't have to close a channel more than notifying the receivers. @lujiajing1126 Please don't try to use it.
• Introduce a self-spinning lock to optimize the life-cycle of the block.
• Fix a tiny bug about parsing the lsm index's key. You never encounter errors in testing the measure package.

Signed-off-by: Gao Hongtao hanahmily@gmail.com

[codecov-commenter]

Codecov Report

Merging #153 (3a65acd) into main (4391412) will decrease coverage by 0.10%.
The diff coverage is 67.04%.

@@            Coverage Diff             @@
##             main     #153      +/-   ##
==========================================
- Coverage   54.11%   54.00%   -0.11%     
==========================================
  Files          89       89              
  Lines        7904     7997      +93     
==========================================
+ Hits         4277     4319      +42     
- Misses       3233     3275      +42     
- Partials      394      403       +9     

Impacted Files	Coverage Δ
banyand/tsdb/metric.go	35.29% <0.00%> (ø)
pkg/timestamp/clock.go	0.00% <ø> (ø)
banyand/tsdb/series.go	45.34% <12.50%> (-3.41%)	⬇️
banyand/tsdb/segment.go	64.91% <31.70%> (-3.15%)	⬇️
banyand/tsdb/seriesdb.go	75.93% <62.96%> (-0.51%)	⬇️
banyand/tsdb/block.go	65.05% <69.47%> (-5.58%)	⬇️
banyand/tsdb/bucket/strategy.go	73.13% <69.69%> (-11.16%)	⬇️
pkg/index/lsm/lsm.go	62.96% <80.00%> (+6.44%)	⬆️
banyand/measure/metadata.go	65.97% <100.00%> (+0.23%)	⬆️
banyand/measure/service.go	73.33% <100.00%> (+0.73%)	⬆️
... and 9 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

[lujiajing1126]

We should make default values constants or variables and add comments for them rather than directly use them everywhere.

Instead of 2<<20 or 8<<20, I prefer 1<<21 or 1<<24 which seems to be clearer.

[lujiajing1126]

• Remove run.Chan, which could cause the data race. We don't have to close a channel more than notifying the receivers. @lujiajing1126 Please don't try to use it.

Not quite follow this argument. I remember I'm not using this toolkit.

[hanahmily]

We should make default values constants or variables and add comments for them rather than directly use them everywhere.

Make sense, I will tweak them.

Instead of 2<<20 or 8<<20, I prefer 1<<21 or 1<<24 which seems to be clearer.

8<<20 indicates the literal(8) and the unit(1<<20=1024*1024=1MB), which is more precise and straightforward than 1<<24.

[hanahmily]

Fixes apache/skywalking#9383

[lujiajing1126]

Since we removed Close() here, is it possible we still have many goroutines waiting to write to the channel?

[hanahmily]

We can use close to notify the receivers, not the senders. The senders will be waiting to write. That is the purpose of this process. They won't block the GC from collecting them.

[lujiajing1126]

We can use close to notify the receivers, not the senders. The senders will be waiting to write. That is the purpose of this process.

Yes, we can use some stop channel to notify the receivers, but what about the senders?

They won't block the GC from collecting them.

From this issue, golang/go#19702, Go may not be clever enough now to garbage collect blocking goroutines.

[wu-sheng]

Automatically closing always has this kind of bugs in GC phases.
Be careful. We don't want a leak 😄

[lujiajing1126]

LGTM. We can create an issue to track goroutine leak issue later.