Use jemalloc in rocksdb

[akhi3030]

We’re using jemalloc in Rust code so it makes sense to do the same in
RocksDB.

Fixes: #6290

[mina86]

This should be enabled only if we enable jemalloc feature on neard

[akhi3030]

This should be enabled only if we enable jemalloc feature on neard

Isn't it already using it? https://github.com/near/nearcore/blob/master/neard/Cargo.toml#L28

[nikurt]

Do we understand the performance impact of this change?

[akhi3030]

Do we understand the performance impact of this change?

Nope, not yet. Do we have some standard set of tests that we can use?

[nikurt]

Do we have some standard set of tests that we can use?

That's what I did last time:
https://near.zulipchat.com/#narrow/stream/297873-pagoda.2Fnode/topic/Performance.20impact.20of.20logging/near/283804107

[mina86]

This should be enabled only if we enable jemalloc feature on neard

Isn't it already using it? https://github.com/near/nearcore/blob/master/neard/Cargo.toml#L28

I mean this feature https://github.com/near/nearcore/blob/master/neard/Cargo.toml#L47 should enable jemalloc for RocksDB.

[matklad]

Do we have some standard set of tests that we can use?

@akhi3030 another approach came up recently here: #7494 (comment)

[matklad]

LGTM, and I am inclined to merge this without blocking on up-front benchmark:

• the status quo is that we don't know which allocator rocksdb is useing, 'cause it links the one dynamically from the computer neard is running on
• any significant perf degradation should be visible in our continous estimation
• we won't immediately notice change in memory usage, but I think it unlikely to worsen, and we can always revert if we observe something iffy on the betanet.

[akhi3030]

Makes sense. I'll merge.

[mina86]

I still think this isn’t correct. We have a cargo feature in neard/Cargo.toml to enable jemalloc. IMO RocksDB jemalloc should be conditionally enabled if that feature is enabled.

[akhi3030]

Can you point me to the relevant bit of code please?

[mina86]

https://github.com/near/nearcore/blob/master/neard/Cargo.toml#L47

PS. An alternative is to get rid of the feature and always use jemalloc I suppose. Have we actually ever performed any benchmarks to see if jemalloc is faster for us?

[matklad]

Ah, good catch, thanks mina86! Yeah, I think we should just enable jemalloc by default: it's not even the question of "faster", without jemalloc we don't know which allocator we are using.

I don't remeber us performing benchmark for nearcore, but, eg, for rust-analyzer and rust-lang, we did observe jemalloc being measurably faster, and, in ra, we did spend some cycles chasing down issue which boiled down to "the user's allocator is not good".