Roadmap and status

As of Dec 3, 2019

Current status:

Branches

• texture: stale, will delete after finalize_portilla_simoncelli

• finalize_portilla_simoncelli: active, fixing Portilla-Simoncelli textures (Kate)

• eigendistortions_gpu: active, working on tests and documentations (Lyndon)

• ventral_stream: need to open PR (Billy)

• texture_CNN: leave for now, generalizing PS idea for texture models (Pierre)

• MAD: pull up to master and get working (Kate/Billy)

• geodesics: will open (Pierre)

Status

Synthesis

• eigendistortions: writing docs and examples

• metamers:

  • working, somebody else should review
  • abstract out some functionality: save/load, display/animate, coarse-to-fine optimziation

• MAD: to-do

• geodesics: basics work

  • make compliant with others' API
  • clean up tutorial
  • abstract out display code, move into new folder

Models

• L/LNL: initialize with non-random filters

• Portilla-Simoncelli: Kate checking against matlab, writing tests

• steerable pyramid: basically done possible extensions: use real_fft instead of complex (default, would make more efficient)

• front-end: setup code to train against TID (human distortion visibility database); into a new learn folder

• entropy-estimation layer

• Tim's V2 derivative model

• Nikhil's V2 model

Other stuff

• prune/update tutorials

• write documentation: setup, high-level explanation of synthesis, explanation of basic API

• restructure README: move bit about docs to contributing, put key figures from synthesis methods in it.

• synthesizing in a different domain: e.g., modifying steerable pyramid coefficients and then inverting it back to the pixel domain

• clean up / complete docstrings

• relative eigendistortions: computing eigendistortions to differentiate two models (instead of single model and MSE)

• make model / synthesis super-classes

Plan

• Lyndon:
  • finishing up eigendistortions
  • write tests, docstrings, and push divisive-normalization layer
• Kate:
  • make Portilla-Simoncelli work: make sure python and matlab outputs match, add tests so they stay that way, write tutorial that includes synthesis, add more documentation, internalize
• Billy:
  • open ventral_stream PR
  • (after Kate makes PS work) get coarse-to-fine working for PS
  • write documentation for more abstract methods
  • write model / synthesis super-classes, then work with Lyndon on eigendistortions and Pierre on geodesics
  • start on MAD
• Pierre:
  • open geodesic branch
  • make L/LNL initialize with non-random filters
  • reorganize tools folder
• Nikhil: