Closes #2156: Bigint stream benchmark

[stress-tess]

This PR (closes #2156 and closes #2165) adds a bigint stream benchmark. The bigint stream does not scale well. I couldn't find any obvious optimizations

bigint_stream after this PR:

% ./benchmarks/run_benchmarks.py bigint_stream -t 1
array size = 100,000,000
number of trials =  1
>>> arkouda bigint stream
numLocales = 1, N = 100,000,000
Average bigint stream time = 22.1851 sec
Average bigint stream rate = 0.10 GiB/sec

% ./benchmarks/run_benchmarks.py bigint_stream -t 1 --max-bits=64
array size = 100,000,000
number of trials =  1
>>> arkouda bigint stream
numLocales = 1, N = 100,000,000
Average bigint stream time = 20.6594 sec
Average bigint stream rate = 0.11 GiB/sec

bigint_stream before this PR:

% ./benchmarks/run_benchmarks.py bigint_stream -t 1
array size = 100,000,000
number of trials =  1
>>> arkouda bigint stream
numLocales = 1, N = 100,000,000
Average bigint stream time = 57.1895 sec
Average bigint stream rate = 0.04 GiB/sec

% ./benchmarks/run_benchmarks.py bigint_stream -t 1 --max-bits=64
array size = 100,000,000
number of trials =  1
>>> arkouda bigint stream
numLocales = 1, N = 100,000,000
Average bigint stream time = 58.1247 sec
Average bigint stream rate = 0.04 GiB/sec

Compared to non-bigint stream:

% ./benchmarks/run_benchmarks.py stream                    
array size = 100,000,000
number of trials =  6
>>> arkouda float64 stream
numLocales = 1, N = 100,000,000
Average time = 0.2566 sec
Average rate = 8.71 GiB/sec

bigint bitwise binops after this PR:

% ./benchmarks/run_benchmarks.py bigint_bitwise_binops -t 1
array size = 100,000,000
number of trials =  1
>>> arkouda bigint bitwise binops
numLocales = 1, N = 100,000,000
Average bigint AND time = 9.9440 sec
Average bigint AND rate = 0.30 GiB/sec

Average bigint OR time = 14.2256 sec
Average bigint OR rate = 0.21 GiB/sec

Average bigint SHIFT time = 23.6632 sec
Average bigint SHIFT rate = 0.06 GiB/sec

% ./benchmarks/run_benchmarks.py bigint_bitwise_binops -t 1 --max-bits=64
array size = 100,000,000
number of trials =  1
>>> arkouda bigint bitwise binops
numLocales = 1, N = 100,000,000
Average bigint AND time = 9.2267 sec
Average bigint AND rate = 0.32 GiB/sec

Average bigint OR time = 14.2185 sec
Average bigint OR rate = 0.21 GiB/sec

Average bigint SHIFT time = 22.9780 sec
Average bigint SHIFT rate = 0.06 GiB/sec

bigint bitwise binops before this PR:

% ./benchmarks/run_benchmarks.py bigint_bitwise_binops -t 1
array size = 100,000,000
number of trials =  1
>>> arkouda bigint bitwise binops
numLocales = 1, N = 100,000,000
Average bigint AND time = 21.7934 sec
Average bigint AND rate = 0.14 GiB/sec

Average bigint OR time = 26.4953 sec
Average bigint OR rate = 0.11 GiB/sec

Average bigint SHIFT time = 39.9405 sec
Average bigint SHIFT rate = 0.04 GiB/sec

 ./benchmarks/run_benchmarks.py bigint_bitwise_binops -t 1 --max-bits=64
array size = 100,000,000
number of trials =  1
>>> arkouda bigint bitwise binops
numLocales = 1, N = 100,000,000
Average bigint AND time = 22.2118 sec
Average bigint AND rate = 0.13 GiB/sec

Average bigint OR time = 25.9856 sec
Average bigint OR rate = 0.11 GiB/sec

Average bigint SHIFT time = 39.0686 sec
Average bigint SHIFT rate = 0.04 GiB/sec

[stress-tess]

I found some optimizations by refactoring code to try and favor inplace ops. So

// instead of
tmp = la * ra;
// do
tmp = la;
tmp *= ra;

The performance is better but still not great. I need to update doBigIntBinOpsv and the bool return methods. I also think I'm gonna go ahead and add the logical/arithmetic benchmark in this PR to get a better idea of the perf boost these changes provide. Converting to a draft until I wrap that up

[ronawho]

I also see fairly poor performance with 16-node-cs-hdr, but I see much better performance when enabling parallel array deinit (chapel-lang/chapel#21670), which we saw benefit other bigint operations:

chapel 1.29:

>>> arkouda float64 stream
Average time = 0.0597 sec
Average rate = 599.50 GiB/sec

>>> arkouda bigint stream
Average bigint stream time = 14.5178 sec
Average bigint stream rate = 2.46 GiB/sec

chapel main w/ chapel-lang/chapel#21670 (parallel deinit):

>>> arkouda float64 stream
numLocales = 16, N = 1,600,000,000
Average time = 0.0589 sec
Average rate = 607.65 GiB/sec

>>> arkouda bigint stream
Average bigint stream time = 0.8836 sec
Average bigint stream rate = 40.47 GiB/sec

So ~20x improvement from parallel deinit. Still ~15x off from int/float stream, which is a little higher than I might expect, but not terrible.

[stress-tess]

So ~20x improvement from parallel deinit. Still ~15x off from int/float stream, which is a little higher than I might expect, but not terrible.

This is great news!!! So the bigint code will be much more performant in 1.30! Thanks for looking into this @ronawho!

[ronawho]

So the bigint code will be much more performant in 1.30!

Yeah, I think we'll highly highly recommend 1.30 for anybody using bigints when it's released (given the performance improvements, bug fixes, and implementation cleanup)

[stress-tess]

This code should be ready to review. I will put up my most updated perf comparison either tonight or tomorrow but it's too pretty not to be outside rn

[Ethan-DeBandi99]

Nothing jumps out as an issue to me.

[joshmarshall1]

Looks good to me

[ronawho]

@pierce314159 we're seeing bigint_bitwise_binops.py timeout during nightly testing for >>. I think the problem is probably here with assigning all elements to a bigint 1 that lives on locale 0. I think you could get rid of the tmp array and just do:

forall t in tmp with (var dB = (1:bigint) << val, var local_max_size = max_size) {

[ronawho]

Whoops, I meant this as a comment for the binopVS case. For the binopVV cases you should be able to do something like:

forall (t, ri) in zip(tmp, ra) with (var local_max_size = max_size) {
  var dB = 1:bigint;
  dB <<= ri;
  t /= dB;

[ronawho]

Opened #2178