feat(mmap_allocator): change the mmap allocator inner allcator from GlobalAllocator to JEAllocator

[BohuTANG]

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

This is the part-1:

• change the mmap allocator inner allcator from GlobalAllocator to JEAllocator
• for JEAllocator: linux use jemalloc, not linux will fallback to std allocator

Part-2 will investigate try to change the GlobalAllocator from jemalloc to std.

Closes #issue

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment

Name	Status	Preview	Comments	Updated
databend	⬜️ Ignored (Inspect)			Feb 11, 2023 at 5:53AM (UTC)

[github-actions]

💡 Click to Expand Hits

Benchmark for hits with c5.4xlarge

Name	Min	Max	Median	Mean	StdDev	ReadRow	ReadByte
Q1	0.01 ms	0.01 ms	0.01 ms	0.00 ms	0.00 ms	1	1
Q2	0.13 ms	0.21 ms	0.16 ms	0.16 ms	0.02 ms	92157620	184315240
Q3	0.19 ms	0.28 ms	0.23 ms	0.23 ms	0.03 ms	99997397	399989588
Q4	0.20 ms	0.25 ms	0.24 ms	0.24 ms	0.02 ms	99997397	799979176
Q5	0.56 ms	0.81 ms	0.57 ms	0.60 ms	0.07 ms	99997397	799979176
Q6	0.83 ms	0.85 ms	0.84 ms	0.84 ms	0.01 ms	99997397	1539621305
Q7	0.12 ms	0.17 ms	0.15 ms	0.15 ms	0.01 ms	99997397	399989588
Q8	0.12 ms	0.21 ms	0.15 ms	0.15 ms	0.03 ms	92157620	184315240
Q9	0.82 ms	0.86 ms	0.84 ms	0.84 ms	0.01 ms	99997397	1199968764
Q10	0.99 ms	1.06 ms	1.01 ms	1.02 ms	0.02 ms	99997397	1599958352
Q11	0.51 ms	0.53 ms	0.52 ms	0.52 ms	0.01 ms	99997397	1623245000
Q12	0.49 ms	0.59 ms	0.50 ms	0.51 ms	0.03 ms	99997397	1823239794
Q13	0.93 ms	0.95 ms	0.95 ms	0.94 ms	0.01 ms	99997397	1539621305
Q14	1.44 ms	1.49 ms	1.46 ms	1.46 ms	0.02 ms	99997397	2339600481
Q15	1.09 ms	1.13 ms	1.10 ms	1.10 ms	0.01 ms	99997397	1739616099
Q16	0.73 ms	0.78 ms	0.74 ms	0.75 ms	0.01 ms	99997397	799979176
Q17	2.68 ms	2.77 ms	2.70 ms	2.71 ms	0.03 ms	99997397	2339600481
Q18	1.68 ms	1.74 ms	1.70 ms	1.70 ms	0.02 ms	99997397	2339600481
Q19	4.53 ms	4.72 ms	4.58 ms	4.60 ms	0.07 ms	99997397	3139579657
Q20	0.03 ms	0.07 ms	0.03 ms	0.03 ms	0.01 ms	508079	4064632
Q21	1.95 ms	2.02 ms	1.99 ms	1.99 ms	0.02 ms	99997397	9838815562
Q22	2.40 ms	2.47 ms	2.42 ms	2.43 ms	0.02 ms	99997397	11378436867
Q23	4.42 ms	4.53 ms	4.45 ms	4.46 ms	0.03 ms	99997397	22465772877
Q24	15.82 ms	16.21 ms	16.08 ms	16.08 ms	0.10 ms	99997397	81370516203
Q25	0.86 ms	0.96 ms	0.87 ms	0.89 ms	0.03 ms	99997397	2339600481
Q26	0.52 ms	0.58 ms	0.52 ms	0.53 ms	0.02 ms	99997397	1539621305
Q27	0.88 ms	0.95 ms	0.90 ms	0.90 ms	0.02 ms	99997397	2339600481
Q28	2.23 ms	2.31 ms	2.27 ms	2.27 ms	0.03 ms	99997397	10238805150
Q29	16.11 ms	16.54 ms	16.25 ms	16.26 ms	0.12 ms	99997397	7462438629
Q30	1.31 ms	1.88 ms	1.33 ms	1.40 ms	0.18 ms	99997397	199994794
Q31	1.12 ms	1.24 ms	1.14 ms	1.15 ms	0.03 ms	99997397	2539595275
Q32	1.84 ms	1.93 ms	1.87 ms	1.87 ms	0.02 ms	99997397	3139579657
Q33	6.53 ms	6.70 ms	6.61 ms	6.62 ms	0.05 ms	99997397	1599958352
Q34	4.32 ms	4.42 ms	4.36 ms	4.36 ms	0.03 ms	99997397	9838815562
Q35	4.34 ms	4.44 ms	4.37 ms	4.38 ms	0.03 ms	99997397	9838815562
Q36	0.65 ms	0.75 ms	0.67 ms	0.68 ms	0.03 ms	99997397	399989588
Q37	0.21 ms	0.31 ms	0.24 ms	0.25 ms	0.03 ms	5130220	521644613
Q38	0.20 ms	0.32 ms	0.30 ms	0.29 ms	0.03 ms	5130220	656495496
Q39	0.17 ms	0.31 ms	0.28 ms	0.27 ms	0.04 ms	5130220	531905053
Q40	0.46 ms	0.53 ms	0.51 ms	0.50 ms	0.03 ms	5130220	961147275
Q41	0.10 ms	0.27 ms	0.26 ms	0.19 ms	0.07 ms	5130220	143646160
Q42	0.09 ms	0.26 ms	0.20 ms	0.16 ms	0.08 ms	5130220	123125280
Q43	0.07 ms	0.13 ms	0.08 ms	0.09 ms	0.02 ms	5130220	102604400

[sundy-li]

LGTM, the perf results should have comparison results to show the improvements. Better static HTML in perf.databend.rs.

[BohuTANG]

LGTM, the perf results should have comparison results to show the improvements. Better static HTML in perf.databend.rs.

I will test it by hand.
@everpcpc is working on it.

[sundy-li]

Can we disable MemoryTrack in query config?

[BohuTANG]

Can we disable MemoryTrack in query config?

No, we need MemoryTrack, and it looks like it's not a performance issue from your test.
It can be a compile option if we want to measure the best performance.

[BohuTANG]

jemalloc(main branch) is the best performance, let's merge this PR make the jemalloc always the hashtable inner allocator.