Low memory combine_rows

[RMeli]

The script combine_rows.py has a high memory consumption for large datasets (such as PDBbind18) and runs out of memory on a 64GB machine.

This PR re-implements such script with a totally different approach, pre-allocating pandas.DataFrames and populating them one row at a time. This approach consistently reduces the memory usage.

The final check is also sped up using numpy instead of Python loops.

Since the approach is completely different from the original script, I created a new script combine_rows_lowmem.py instead of modifying the original one.

Small Test

Loading only rows 1-3 and printing a 5x5 sub-matrix with the new code:

dist =
[[     nan      nan      nan      nan      nan]
 [0.008475 0.       0.016667 0.672362 0.628814]
 [0.025    0.016667 0.       0.664322 0.633333]
 [0.677387 0.672362 0.664322 0.       0.78191 ]
 [     nan      nan      nan      nan      nan]]

lsim=
[[     nan      nan      nan      nan      nan]
 [0.479689 1.       0.290152 0.364799 0.328125]
 [0.221582 0.290152 1.       0.300582 0.323887]
 [0.301352 0.364799 0.300582 1.       0.328125]
 [     nan      nan      nan      nan      nan]]

Loading only rows 1-3 and printing a 5x5 sub-matrix with combine_rows.py:

m =
[[     nan      nan      nan      nan      nan]
 [0.008475 0.       0.016667 0.672362 0.628814]
 [0.025    0.016667 0.       0.664322 0.633333]
 [0.677387 0.672362 0.664322 0.       0.78191 ]
 [     nan      nan      nan      nan      nan]]

lm =
[[     nan      nan      nan      nan      nan]
 [0.479689 1.       0.290152 0.364799 0.328125]
 [0.221582 0.290152 1.       0.300582 0.323887]
 [0.301352 0.364799 0.300582 1.       0.328125]
 [     nan      nan      nan      nan      nan]]

[dkoes]

Not sure how I missed this PR.