Performance for moving Immix

[qinsoon]

I compared 6 builds: [immix, nonmoving immix, Julia stock GC] x [current, base]

Current: after #170 and mmtk/julia#64. It moves much more objects than the base build.

Base: before we introduce conservative stack scanning and more pins.
Binding: cab4348 (https://github.com/qinsoon/mmtk-julia/commits/perf-test-base-1009/),
Julia: 88fea475d9639820488f6dd50fcb08c60a011899(https://github.com/qinsoon/julia/commits/perf-test-base-1009/)

The results are as below

Observations:

• We see obvious slowdown in GC time for moving Immix. We really should look into the reasons, and optimize it.
• We see slowdown in total time for both moving and non moving Immix. So extra pinning causes slowdown.
• We only see one benchmark that got improved by moving more objects. We should check if our moving criteria is too strict, and we do not actually trigger defrag promptly (maybe related to Immix performs faster if defrag is enabled for every GC? mmtk-core#1203).
• We are slower than Julia's stock GC in some benchmarks, mostly due to crazy GC time. One of the possible reasons is realloc, where we keep reallocating new objects rather than resizingexisting allocation (e.g. for the append benchmark).

Raw numbers

                                                    benchmark                                                              build  total time  total time error     gc time  gc time error
0                        multithreaded-big_arrays-issue-52937            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    1.001991          0.007631    1.068493       0.019017
1                        multithreaded-big_arrays-issue-52937  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    0.993651          0.003302    1.082192       0.023291
2                        multithreaded-big_arrays-issue-52937       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    1.079056          0.012843    1.054795       0.035577
3                        multithreaded-big_arrays-issue-52937                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    1.085304          0.020927    1.404110       0.141032
4                        multithreaded-big_arrays-issue-52937                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.015456    1.000000       0.081794
5                        multithreaded-big_arrays-issue-52937                     julia-stock(1.5x minheap,.julia_gc_threads-12)    1.008655          0.013697    1.013699       0.038033
6                           multithreaded-big_arrays-objarray            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    1.072366          0.009304    0.342968       0.002288
7                           multithreaded-big_arrays-objarray  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    0.682634          0.000489    0.380691       0.009505
8                           multithreaded-big_arrays-objarray       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    0.703751          0.001212    0.369157       0.005171
9                           multithreaded-big_arrays-objarray                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    0.832624          0.003399    0.343443       0.003856
10                          multithreaded-big_arrays-objarray                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.001396    1.000000       0.012857
11                          multithreaded-big_arrays-objarray                     julia-stock(1.5x minheap,.julia_gc_threads-12)    0.999293          0.005103    1.005089       0.011448
12                   multithreaded-binary_tree-tree_immutable            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    0.722035          0.015641    0.978037       0.074309
13                   multithreaded-binary_tree-tree_immutable  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    0.750636          0.011127    1.002157       0.055980
14                   multithreaded-binary_tree-tree_immutable       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    0.768645          0.011116    1.013629       0.055668
15                   multithreaded-binary_tree-tree_immutable                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    0.759675          0.009574    1.081871       0.070892
16                   multithreaded-binary_tree-tree_immutable                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.002101    1.000000       0.015747
17                   multithreaded-binary_tree-tree_immutable                     julia-stock(1.5x minheap,.julia_gc_threads-12)    0.993132          0.005533    0.983136       0.019657
18                     multithreaded-binary_tree-tree_mutable            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    0.569816          0.006219    0.674479       0.032193
19                     multithreaded-binary_tree-tree_mutable  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    0.595737          0.007340    0.690163       0.004300
20                     multithreaded-binary_tree-tree_mutable       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    0.608779          0.004258    0.692883       0.002972
21                     multithreaded-binary_tree-tree_mutable                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    0.596285          0.004716    0.731233       0.016462
22                     multithreaded-binary_tree-tree_mutable                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.001682    1.000000       0.017062
23                     multithreaded-binary_tree-tree_mutable                     julia-stock(1.5x minheap,.julia_gc_threads-12)    1.000135          0.007058    1.007389       0.019913
24        multithreaded-mergesort_parallel-mergesort_parallel            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    1.112921          0.003371    1.230769       0.056606
25        multithreaded-mergesort_parallel-mergesort_parallel  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    1.126781          0.006204    1.237762       0.038831
26        multithreaded-mergesort_parallel-mergesort_parallel       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    1.186690          0.001550    1.174825       0.023779
27        multithreaded-mergesort_parallel-mergesort_parallel                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    1.179336          0.002612    1.517483       0.058247
28        multithreaded-mergesort_parallel-mergesort_parallel                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.001600    1.000000       0.023779
29        multithreaded-mergesort_parallel-mergesort_parallel                     julia-stock(1.5x minheap,.julia_gc_threads-12)    1.003276          0.001639    1.020979       0.019416
30  multithreaded-mm_divide_and_conquer-mm_divide_and_conquer            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    0.991135          0.011078    0.794872       0.038993
31  multithreaded-mm_divide_and_conquer-mm_divide_and_conquer  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    0.985494          0.013852    0.789744       0.026637
32  multithreaded-mm_divide_and_conquer-mm_divide_and_conquer       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    0.993110          0.010989    0.800000       0.052314
33  multithreaded-mm_divide_and_conquer-mm_divide_and_conquer                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    0.999637          0.005533    0.964103       0.053274
34  multithreaded-mm_divide_and_conquer-mm_divide_and_conquer                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.004385    1.000000       0.045025
35  multithreaded-mm_divide_and_conquer-mm_divide_and_conquer                     julia-stock(1.5x minheap,.julia_gc_threads-12)    0.997784          0.001010    0.984615       0.066020
36                                 serial-TimeZones-TimeZones            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    1.323013          0.012830    1.835706       0.297662
37                                 serial-TimeZones-TimeZones  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    1.321206          0.002393    1.695509       0.102406
38                                 serial-TimeZones-TimeZones       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    1.368336          0.021388    2.165389       0.512314
39                                 serial-TimeZones-TimeZones                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    1.324820          0.013444    2.180723       0.351313
40                                 serial-TimeZones-TimeZones                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.004352    1.000000       0.010313
41                                 serial-TimeZones-TimeZones                     julia-stock(1.5x minheap,.julia_gc_threads-12)    1.035383          0.003297    1.003286       0.014742
42                                       serial-append-append            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    4.642200          0.318347  366.222857      30.620607
43                                       serial-append-append  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    3.212709          0.141530  228.062857      13.646882
44                                       serial-append-append       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    3.180316          0.052266  224.165714       5.340917
45                                       serial-append-append                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    3.145799          0.199476  221.177143      19.502934
46                                       serial-append-append                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.011107    1.000000       0.025085
47                                       serial-append-append                     julia-stock(1.5x minheap,.julia_gc_threads-12)    1.002832          0.004615    1.005714       0.063462
48                                serial-big_arrays-many_refs            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    0.529565          0.015935    0.429090       0.032785
49                                serial-big_arrays-many_refs  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    0.558101          0.019353    0.456170       0.038744
50                                serial-big_arrays-many_refs       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    0.565647          0.020376    0.443886       0.040647
51                                serial-big_arrays-many_refs                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    0.559748          0.010221    0.472083       0.019432
52                                serial-big_arrays-many_refs                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.003035    1.000000       0.004683
53                                serial-big_arrays-many_refs                     julia-stock(1.5x minheap,.julia_gc_threads-12)    1.004527          0.006987    0.967616       0.011457
54                               serial-big_arrays-single_ref            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    1.722498          0.072464    4.275109       0.298309
55                               serial-big_arrays-single_ref  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    1.695061          0.079693    4.161572       0.333850
56                               serial-big_arrays-single_ref       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    1.736870          0.153585    4.331878       0.643673
57                               serial-big_arrays-single_ref                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)   13.202247          0.650385   52.240175       2.705963
58                               serial-big_arrays-single_ref                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.008079    1.000000       0.031572
59                               serial-big_arrays-single_ref                     julia-stock(1.5x minheap,.julia_gc_threads-12)    1.004181          0.003735    1.012009       0.015603
60                                      serial-bigint-pollard            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    0.930587          0.011243    1.337443       0.036526
61                                      serial-bigint-pollard  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    0.911637          0.023913    1.250487       0.063993
62                                      serial-bigint-pollard       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    0.925232          0.017051    1.271901       0.039781
63                                      serial-bigint-pollard                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    1.032338          0.035329    1.613887       0.083474
64                                      serial-bigint-pollard                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.013683    1.000000       0.011537
65                                      serial-bigint-pollard                     julia-stock(1.5x minheap,.julia_gc_threads-12)    1.008445          0.018473    1.000000       0.008734
66                                         serial-linked-list            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    0.838250          0.001832    1.364341       0.040266
67                                         serial-linked-list  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    0.905960          0.002479    1.325581       0.040266
68                                         serial-linked-list       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    0.953902          0.003614    1.379845       0.072987
69                                         serial-linked-list                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    0.888258          0.002348    1.713178       0.052720
70                                         serial-linked-list                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.002701    1.000000       0.040266
71                                         serial-linked-list                     julia-stock(1.5x minheap,.julia_gc_threads-12)    1.015710          0.006138    1.054264       0.040266
72                                         serial-linked-tree            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    0.975565          0.013412    1.165517       0.035822
73                                         serial-linked-tree  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    0.986056          0.011435    1.206897       0.067698
74                                         serial-linked-tree       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    1.003044          0.007220    1.165517       0.046903
75                                         serial-linked-tree                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    0.980974          0.016630    1.496552       0.038296
76                                         serial-linked-tree                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.008773    1.000000       0.074159
77                                         serial-linked-tree                     julia-stock(1.5x minheap,.julia_gc_threads-12)    0.995787          0.012941    1.013793       0.048818
78                                     serial-strings-strings            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    0.912167          0.012381    1.301471       0.040830
79                                     serial-strings-strings  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    0.910061          0.010883    1.316176       0.020415
80                                     serial-strings-strings       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    1.020044          0.016129    1.235294       0.025003
81                                     serial-strings-strings                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    0.995634          0.009911    1.669118       0.061245
82                                     serial-strings-strings                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.012804    1.000000       0.067709
83                                     serial-strings-strings                     julia-stock(1.5x minheap,.julia_gc_threads-12)    0.994557          0.011056    1.029412       0.032279
84                                       slow-bigint-pidigits            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    5.606848          0.292921   33.578456       1.970703
85                                       slow-bigint-pidigits  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    4.995648          0.016821   29.515247       0.107972
86                                       slow-bigint-pidigits       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    5.245866          0.117416   31.165166       0.784853
87                                       slow-bigint-pidigits                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    5.664028          0.224184   33.833949       1.584757
88                                       slow-bigint-pidigits                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.009137    1.000000       0.017394
89                                       slow-bigint-pidigits                     julia-stock(1.5x minheap,.julia_gc_threads-12)    0.999433          0.014638    1.022791       0.018414
90                                       slow-rb_tree-rb_tree            julia-mmtk-immix-base(1.5x minheap,.mmtk_gc_threads-12)    0.597253          0.010999    1.061980       0.002898
91                                       slow-rb_tree-rb_tree  julia-mmtk-immix-nonmoving-base(1.5x minheap,.mmtk_gc_threads-12)    0.598262          0.005335    1.072169       0.009907
92                                       slow-rb_tree-rb_tree       julia-mmtk-immix-nonmoving(1.5x minheap,.mmtk_gc_threads-12)    0.606486          0.013802    1.059018       0.002955
93                                       slow-rb_tree-rb_tree                 julia-mmtk-immix(1.5x minheap,.mmtk_gc_threads-12)    0.596166          0.005793    1.068115       0.005425
94                                       slow-rb_tree-rb_tree                julia-stock-base(1.5x minheap,.julia_gc_threads-12)    1.000000          0.002973    1.000000       0.007947
95                                       slow-rb_tree-rb_tree                     julia-stock(1.5x minheap,.julia_gc_threads-12)    1.019828          0.028913    0.998590       0.004189

[qinsoon]

The ~10% mutator time slowdown in multithreaded-big_arrays-issue-52937 is caused by the ccall for pinning in https://github.com/mmtk/julia/blob/9709741a5a30fd500c8134bec09ed32560f20992/base/mpfr.jl#L125.

[qinsoon]

The ~10% mutator time slowdown in multithreaded-big_arrays-issue-52937 is caused by the ccall for pinning in https://github.com/mmtk/julia/blob/9709741a5a30fd500c8134bec09ed32560f20992/base/mpfr.jl#L125.

Gabriel mentioned that there is a way to do ccall faster: https://github.com/JuliaLang/julia/blob/master/src/ccall.cpp. This treats the specific C calls as compiler intrinsics.

We may not need it for this case anyway, as upstream rewrites this type and we dont need to pin it anymore: https://github.com/JuliaLang/julia/blob/4c076c80af9d9c8439cfa20e2efd5c884d88b64d/base/mpfr.jl#L141