Adding periodic boundary condition support to tobac

[galexsky]

Extensive update of tobac capabilities and procedures by myself and @freemansw1 . Many substantial changes to code, such as:

• Inclusion of the vertical dimension for 3D feature detection and tracking, now accounting for a potentially non-uniform dz
• Modification to 3D segmentation allowing "box seeding" of user-set size in addition to default "column seeding" method
• Feature detection, segmentation, and tracking now able to address datasets with periodic boundary conditions (PBCs)
• Addition of several new utility functions and movement of some such functions from other modules to utils
• Optimization, simplification, bug fixes, and other streamlining of overall code base
• Improved code documentation

[freemansw1]

I'm back to working on this now that we're very close to v1.4.0. I've got the latest changes from RC_v1.5.0 merged in, although there will be some more work to do once #190 is merged in.

I'm not ready for a re-review as there's a lot more to do, but I will ping when I am.

[JuliaKukulies]

Great @freemansw1 ! Thanks for the notification and for all your hard work on this one. I will have a look at #190 later this week.

[freemansw1]

Okay, I've updated this code for RC_V1.5.0. One thing that I'm struggling with: I cannot get our workaround for predictive tracking working with 3D data. For now, I've switched tests to use random, but that is clearly not ideal. @snilsn any ideas? I know that this will be fixed once soft-matter/trackpy#710 is merged, but without an ETA there, it would be much better to have a workaround.

@snilsn if you're able/willing/interested in helping, I've put the "base case" in a gist here: https://gist.github.com/freemansw1/a917e0d823524c6bb44eee458e218a02 .

[snilsn]

I could get df_link_iter to produce an output by setting pos_columns to ['z', 'y', 'x'] manually, but then the prediction doesn't work. This could mean that guessing the position columns isn't working correctly for 3d inputs within df_link_iter, which would make our workaround useless. trackpy.utils.guess_pos_columns() is used for this, but seems to work fine

One idea I had was specifyng an 3d initial_guess_vels in NearestVelocityPredict, but that isn't helping.

Did you find any examples for trackpy that include tracking 3d features using predictors @freemansw1?

[snilsn]

I think there is a solution (at least theoretically). It's possible to set the values for the interpolator manually:

from scipy.interpolate import NearestNDInterpolator
import numpy as np
import pandas as pd
import trackpy

test_df = pd.DataFrame({'z': [0,1,2,3], 'y':[0,1,2,3], 
'x': [0,1,2,3], 'frame': [0,1,2,3]})

features_list = [frame for i, frame in test_df.groupby("frame", sort=True)]

initial_guess_pos = np.asarray([[0, 0, 0]])
initial_guess_vel = np.asarray([[1, 1, 1]])

pred = trackpy.predict.NearestVelocityPredict(span=1)
pred.interpolator = NearestNDInterpolator(initial_guess_pos, initial_guess_vel)

traj_iter = trackpy.link_df_iter(features_list, 
                                 search_range=0.1, 
                                 t_column='frame', 
                                 predictor=pred.predict)
track = pd.concat(traj_iter)

This code links all features into one particle despite the small search range, i. e. the prediction is happening.

The problem is, that the value for the velocity isn't adjusted automatically. This would mean that we would have to loop through all the frames in pairs, interpolate the velocity from the last frame and set a new pred.interpolator at each step, do the linking for the two frames and combine everything into one dataframe at the end.

It's probably possible, but a little ridiculous. Maybe we can iterate to something feasible from here.

[freemansw1]

I've now merged in the latest v1.5.0 changes, most importantly the utils breakout (#191). I'm in the process of thinking how best to do this. I'm becoming inclined to break out the 3D from the PBC changes, despite the pain that that will likely cause for me. As of now, this PR is extremely difficult to review due to its size and scope, with around 5K LOC changed.

I'm going to hold off asking for re-reviews until I resolve this.

[freemansw1]

It appears that, because #209 was merged and contains this commit history, GitHub has tagged this as merged as well. I'll work on a new PR for the PBC changes.

[w-k-jones]

That's a slightly annoying feature on Github! Let us know when the new PR is ready and I'll review it asap