Supercharge Caffe's relationship with Python

[longjon]

I'm making this issue to track some ongoing improvements to pycaffe that I don't have time to PR just yet. #1020 made it possible for Caffe to call Python, in addition to the converse, and it's now becoming possible to combine Caffe C++ and Python code in essentially arbitrary ways.

Some of the coming changes are already PRs, some are already written locally, and some are not done yet. This list is to help me remember to PR everything when ready, to let you know what I have in mind, and provide a point of discussion.

I don't expect to be making more PRs before mid-November, but I do intend to do (or have done) everything on this list. There are a few other improvements that I have in mind, but that I'll leave off until they become more concrete.

• Test the Python wrapper, starting at least with the core functionality (Python and Matlab Wrapper Test Coverage #173). [Now Python testing #1473.]
• Rewrite the C++ wrapper code to use boost::python the way it was intended, removing the existing wrapper classes, and enabling some of the following changes. [Now Reform the boost::python wrapper, including layers implemented in Python #1703.]
• Allow stepping of the solver, so that arbitrary Python code can be intermingled with gradient steps (Refactor Solver to allow interactive stepping #1228).
• Make Python layer work by subclassing caffe.Layer in Python. This will make all the layer functionality (e.g., learnable parameters) exposable (in theory) from Python. (Updates Python layer for rapidly writing nets in Python #1020). [Now Reform the boost::python wrapper, including layers implemented in Python #1703.]
• Expose layer parameters (from protobuf) to Python by bridging C++ and Python protobuf messages.
• Write examples exercising the capability of pycaffe, old and new.

[kloudkl]

Still remember CXXNET? It is not losing the competition. Instead of focusing on the academic users, it has been integrated into a commercial product. One of its authors Tianqi Chen worked as an intern of Graphlab Inc last summer. The GraphLab Create Python toolkits now include a deep learning module based on CXXNET. The souce code is located in the directory toolkits/deeplearning and the documentation is here.

[BlGene]

Here are two additional ease-of-use features that I would suggest adding to this to-do list.

• loading models directly from netspec
  ns = caffe.NetSpec() ... n = caffe.Net(ns, caffe.TEST)
• Solver specification in python.