Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crop layer for automatically aligning computations #1639

Closed
wants to merge 5 commits into from

Conversation

longjon
Copy link
Contributor

@longjon longjon commented Dec 28, 2014

After #1637 and #1638 (new content is just the commit longjon@4e38c93).

Existing layers shift and warp coordinate space: translation by padding (or lack thereof), contraction by strided convolution or pooling, and expansion by strided deconvolution (#1615). Often one wants to align two blobs, e.g., to establish a correspondence between input and output, or to fuse two different paths of computation. Counting conv/deconv strides to ensure that blob coordinates have the same scale is generally straightforward. Computing the offset between two blobs that results from intermediate padding and kernel sizes is trickier.

This layer takes two bottom blobs and produces one top, which is a copy of the first bottom cropped to the size of the second so that coordinates exactly correspond, i.e., it makes sense to fuse or compare the top blob with the second bottom, regardless of whatever padding or other shenanigans took place between their computation.

This is done by computing the coordinate mapping between the two bottom blobs, as provided by #1637 and made accessible by #1638. If that mapping is a simple translation, and has the right sign to allow the first blob to be "cropped to" the second, the layer simply performs the copy. If the mapping is not an integer translation, or the translation has the wrong sign, an error is thrown, and the net may be rearranged to allow sensible fusion.

The implementation of LayerSetUp amounts to some simple graph traversal to find the path connecting the two bottoms. Currently Net does not provide great facilities for traversing the layer graph, so it's a bit cumbersome; maybe this can be improved in the future.

There is a bit of engineering involved in these three PRs, but the result is pretty convenient: what was before a tricky offline calculation becomes a trivial layer specification.

Another way to implement this, without #1638, would be to remove the graph traversal from CropLayer, giving it a simple parameter instead, and provide some other mechanism for automatically filling in that parameter.

Currently CPU and GPU (trivially) are provided, but tests and documentation are not.

longjon added a commit to longjon/caffe that referenced this pull request Dec 29, 2014
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Dec 29, 2014
Crop layer for automatically aligning computations
@longjon longjon force-pushed the crop-layer branch 2 times, most recently from 45af8f3 to c1b7bab Compare December 29, 2014 23:44
@longjon
Copy link
Contributor Author

longjon commented Dec 29, 2014

Updated: now using a CUDA kernel for the GPU implementation, with much more reasonable performance.

longjon added a commit to longjon/caffe that referenced this pull request Dec 29, 2014
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Dec 29, 2014
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Dec 30, 2014
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Dec 30, 2014
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Dec 30, 2014
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Dec 30, 2014
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Dec 31, 2014
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Dec 31, 2014
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Dec 31, 2014
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Jan 1, 2015
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Jan 2, 2015
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Jan 2, 2015
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Jan 3, 2015
Crop layer for automatically aligning computations
longjon added a commit to longjon/caffe that referenced this pull request Jan 3, 2015
Crop layer for automatically aligning computations
philkr added a commit to philkr/caffe that referenced this pull request Jan 25, 2015
Crop layer for automatically aligning computations
@shelhamer
Copy link
Member

Replaced by #1976.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants