Skip to content

Multitarget Brainstorm 20171115

Jump to bottom
Allen Lee edited this page Nov 16, 2017 · 7 revisions

Touching Animals (Non-touching animals being tackled by neighbor masking)

  1. random init on top of trx(1) and trx(2). pool all results and then look for best pair of replicates that minimize sum_over_all_other_shapes(min(dist-to-chosen-pair-element1,dist-to-chosen-pair-element2))

  2. inputting trx centroids directly into regression. i need to think on this more to understand.

  3. generalized rotational correction. this was our "heirarchical" idea from before. maybe it is similar to 2. I think the idea is to imagine animal 1 at the origin, then parameterize the relative heading+orientation of animal2's nose or centroid via eg (r,theta). The shape vector is [tgt1pt1x tgt1pt1y tgt1pt2x tgt1pt2y r theta tgt2otherptx tgt2otherpty]. Features are drawn from "within"/(shape-indexed) for target1, and "within"/(shape-indexed) for target2, plus (r,theta). Maybe features are drawn along the (r,theta) line between two targets too. Maybe the "within tgt1" features are used to regress/update tgt1 pts only; the "within tgt2" features are used to regress/update tgt2 pts only; then the relative (r,theta) DOFs are regressed/updated using ... all features? Maybe the (r,theta) pair itself should be included in the feature set (not sure how exactly).

3a. one phrasing of my mental hangup re tackling multiple-animals-as-one-big-animal in CPR is, what does "shape-indexed" mean when you have 2+ targets? With a randomly oriented single target, "shape-indexing" needs to account for the heading of the animal. If you have 2+ animals at arbitrary orientations, yet treated as a single animal, the meaning of "the line segment connecting tgt1ptX and tgt2ptY" seems to carry very little info without the context (heading and orientation of tgt2 wrt tgt1). Hence my instinct to somehow explicitly model the location/orientation of the targets and somehow restrict the regression or shape-deltas along allowed manifolds.

However maybe this is wrong/dumb. Or maybe if you have enough data maybe it just works fine.

  1. Currently we treat all animals as indistinguishable, but with Roian's data mouse1 and mouse2 are in fact the same individuals across multiple movies. Train two separate classifiers for tgt1 and tgt2 across movies.

  2. create synthetic trx at centroid of two targets, with orientation pointing from tgt1->tgt2. To apply I guess you would turn off rotational invariance, and initialize clouds using trx-specified orientation. Are the animals at arbitrary distance/range? That seems weird. This seems the same as initializing independently on top of each trx and then treating as one big animal?

Cpr requires a cloud of points over a largeish spatial extent (all around the entire target) to zero in on a shape. so the presence of neighboring targets seems like a large potential source of noise/confusion. Masking neighbors is a natural reaction to try to counteract this effect of nbors. part detection otoh is more inherently local(?) so maybe suffers less from this problem.

Clone this wiki locally