Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Augment layers with their induced coordinate maps #1637

Closed
wants to merge 3 commits into from

Conversation

longjon
Copy link
Contributor

@longjon longjon commented Dec 28, 2014

After #1615, not because that PR is necessary for this one, but so that DeconvolutionLayer is already supported.

All layers map values from some array to another, often of a different size. For most basic convnet layers, this forward mapping naturally induces a mapping on the coordinates that index the values of the input and output arrays. Computing this map is often useful for constructing and analyzing nets. Doing so by hand is laborious and error-prone; this PR provides an automated mechanism.

There are a few basic ways this could be done:

  1. Provide an external utility (perhaps implemented in Python).
  2. Provide a mechanism within Caffe proper, but keep all information about the coordinate maps separate from the layer implementations.
  3. Augment each layer with its induced coordinate map.

Options 1 and 2 avoid touching the layer class definition, but require separated pieces of code to be kept in sync. Option 1 makes it impossible to use these coordinate maps within Caffe proper, e.g., by a future layer. Option 3 is the one implemented here.

This PR augments layers with a member function, coord_map, that keeps track of the coordinate mapping. A new type, DiagonalAffineMap, is introduced to store these mappings. (Note that they all that they all take the form Ax + b, where A is diagonal. This could be generalized to a class hierarchy in the future.) Note that the coordinate maps depend only on the layer (specification) parameters, not on bottom data (though the latter is not strictly prohibited).

For all existing layers, the induced coordinate map is one of the following:

  • not meaningful
  • the identity
  • the mapping induced by deconvolution, implemented by FilterMap
  • the mapping induced by convolution, i.e. the inverse of the above, implemented via FilterMap(...).inv().

Certain conventions need to be chosen to define the induced coordinate map. The conventions used here:

  • The coordinate map does not make sense for all dimensions; for convolution, for example, while the num dimension induces the identity map, the channel dimension does not induce a meaningful coordinate transformation. The mapping returned by coord_map therefore applies only to the coordinates for which such a mapping makes sense and is useful (i.e., the spatial dimensions). Thus all implemented maps are 2-dimensional (but the code should be easily extendable to Blobs are N-D arrays (for N not necessarily equals 4) #1486).
  • The induced mapping goes from bottom coordinates to top coordinates, in the direction of Forward. (Often one wants the reverse, easily accessible with DiagonalAffineMap::inv.)
  • When an output value depends on input values within a rectangular region (as in convolution or pooling), the corresponding input coordinates are given by the center of that rectangle. (By symmetry, this is the only sensible choice.)

Questions

  • Neither existing code nor the Google style guide give firm rules about which methods should be CamelCase and which should be underscore_case. I made some fairly arbitrary choices here; do you like them?
  • I implemented DiagonalAffineMap::identity as a static method, but FilterMap as a separate function. Should they both be separate functions?
  • What if a layer has more than one bottom, or more than one top, and the mappings between bottoms and tops are not all compatible? Currently this issue does not come up.

@shelhamer
Copy link
Member

Replaced by #1975.

@shelhamer shelhamer closed this Feb 26, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants