Merge branch 'master' of https://github.com/LabForComputationalVision…

…/plenoptic

.github/workflows/ci.yml

@@ -1,6 +1,8 @@
 name: build
 on:
-  push:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 0 * * 0" # weekly
   pull_request:
     branches:
       - master
@@ -20,13 +22,32 @@ jobs:
         uses: actions/setup-python@v2
         with:
           python-version: ${{ matrix.python-version }}
+      # based on AllenNLP's setup:
+      # https://medium.com/ai2-blog/python-caching-in-github-actions-e9452698e98d
+      - name: Cache environment
+        uses: actions/cache@v2
+        with:
+          path: ${{ env.pythonLocation }}
+          key: ${{ env.pythonLocation }}-${{ hashFiles('setup.py') }}
       - name: Install dependencies
         run: |
-          pip install -e .
-          pip install pytest-cov
-          sudo apt update
-          sudo apt install ffmpeg
+          # using the --upgrade and --upgrade-strategy eager flags ensures that
+          # pip will always install the latest allowed version of all
+          # dependencies, to make sure the cache doesn't go stale
+          pip install --upgrade --upgrade-strategy eager -e .
+          pip install --upgrade --upgrade-strategy eager pytest-cov
+
+      - name: Run tests with pytest
+        if: ${{ matrix.test_script == 'display' }}
+        # we have two cores on the linux github action runners:
+        # https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners
+        run: |
+          pip install --upgrade --upgrade-strategy eager pytest-xdist
+          pytest -n 2 --cov=plenoptic tests/test_${{ matrix.test_script }}.py
       - name: Run tests with pytest
+        if: ${{ matrix.test_script != 'display' }}
+        # only test_display should parallelize across the cores, the others get
+        # slowed down by it
         run: 'pytest --cov=plenoptic tests/test_${{ matrix.test_script }}.py'
       - name: Upload to codecov
         run: 'bash <(curl -s https://codecov.io/bash)'

.github/workflows/treebeard.yml

@@ -2,9 +2,13 @@
 # Run all notebooks on every push and weekly
 name: tutorials
 on:
-  push:
+  workflow_dispatch:
   schedule:
     - cron: "0 0 * * 0" # weekly
+  pull_request:
+    branches:
+      - master
+
 jobs:
   run:
     runs-on: ubuntu-latest
@@ -19,11 +23,18 @@ jobs:
       - uses: actions/setup-python@v2
         with:
           python-version: ${{ matrix.python-version }}
+      # based on AllenNLP's setup:
+      # https://medium.com/ai2-blog/python-caching-in-github-actions-e9452698e98d
+      - name: Cache environment
+        uses: actions/cache@v2
+        with:
+          path: ${{ env.pythonLocation }}
+          key: ${{ env.pythonLocation }}-${{ hashFiles('setup.py') }}
+      - name: Setup FFmpeg
+        uses: FedericoCarboni/setup-ffmpeg@v1
       - name: Install dependencies
         run: |
-          pip install -e .
-          sudo apt update
-          sudo apt install ffmpeg
+          pip install --upgrade --upgrade-strategy eager -e .
           pip install jupyter
           pip install ipywidgets
       - uses: treebeardtech/treebeard@master

CONTRIBUTING.md

@@ -161,6 +161,83 @@ to `.github/workflows/treebeard.yml` and add the name of the new notebook
 `examples/100_awesome_tutorial.ipynb`, you would add `100_awesome_tutorial` as
 the a new item in the `notebook` list.
 
+### Test parameterizations and fixtures
+
+#### Parametrize
+
+If you have many variants on a test you wish to run, you should probably make
+use of pytests' `parametrize` mark. There are many examples throughout our
+existing tests (and see official [pytest
+docs](https://docs.pytest.org/en/stable/parametrize.html)), but the basic idea
+is that you write a function that takes an argument and then use the
+`@pytest.mark.parametrize` decorator to show pytest how to iterate over the
+arguments. For example, instead of writing:
+
+```python
+def test_basic_1():
+    assert int('3') == 3
+
+def test_basic_2():
+    assert int('5') == 5
+```
+
+You could write:
+
+```python
+@pytest.mark.parametrize('a', [3, 5])
+def test_basic(a):
+    if a == '3':
+        test_val = 3
+    elif a == '5':
+        test_val = 5
+    assert int(a) == test_val
+
+```
+
+This starts to become very helpful when you have multiple arguments you wish to
+iterate over in this manner.
+
+#### Fixtures
+
+If you are using an object that gets used in multiple tests (such as an image or
+model), you should make use of fixtures to avoid having to load or initialize
+the object multiple times. Look at `conftest.py` to see those fixtures available
+for all tests, or you can write your own (though pay attention to the
+[scope](https://docs.pytest.org/en/stable/fixture.html#scope-sharing-fixtures-across-classes-modules-packages-or-session)).
+For example, `conftest.py` contains several images that you can use for your
+tests, such as `basic_stimuli`, `curie_img`, or `color_img`. To use them, simply
+add them as arguments to your function:
+
+```python
+def test_img(curie_img):
+    img = po.load_images('data/curie.pgm')
+    assert torch.allclose(img, curie_img)
+```
+
+#### Combining the two
+
+You can combine fixtures and parameterization, which is helpful for when you
+want to test multiple models with a synthesis method, for example. This is
+slightly more complicated and relies on pytest's [indirect
+parametrization](https://docs.pytest.org/en/stable/example/parametrize.html#indirect-parametrization)
+(and requires `pytest>=5.1.2` to work properly). For example, `conftest.py` has
+a fixture, `model`, which accepts a string and returns an instantiated model on
+the right device. Use it like so:
+
+```python
+@pytest.mark.parametrize('model', ['SPyr', 'LNL'], indirect=True)
+def test_synth(curie_img, model):
+    met = po.synth.Metamer(curie_img, model)
+    met.synthesize()
+```
+
+This model will be run twice, once with the steerable pyramid model and once
+with the Linear-Nonlinear model. See the `get_model` function in `conftest.py`
+for the available strings. Note that unlike in the simple
+[parametrize](#parametrize) example, we add the `indirect=True` argument here.
+If we did not include that argument, `model` would just be the strings `'SPyr'`
+and `'LNL'`!
+
 ## Documentation
 
 ### Adding documentation

README.md

@@ -148,9 +148,16 @@ explanation of git, Github, and the associated terminology.
 
 ### ffmpeg
 
-Several methods in this package generate videos. In order to save them or
-convert them to HTML5 for viewing, you'll need
-[ffmpeg](https://ffmpeg.org/download.html) installed on your system as well.
+Several methods in this package generate videos. There are several backends
+possible for saving the animations to file, see (matplotlib
+documentation)[https://matplotlib.org/stable/api/animation_api.html#writer-classes]
+for more details. In order convert them to HTML5 for viewing (and thus, to view
+in a jupyter notebook), you'll need [ffmpeg](https://ffmpeg.org/download.html)
+installed and on your path as well.
+
+To change the backend, run `matplotlib.rcParams['animation.writer'] = writer`
+before calling any of the animate functions. If you try to set that `rcParam`
+with a random string, `matplotlib` will tell you the available choices.
 
 ## plenoptic
 

examples/02_Eigendistortions.ipynb

@@ -4,7 +4,7 @@
    "cell_type": "markdown",
    "metadata": {
     "pycharm": {
-     "name": "#%%\n"
+     "name": "#%% md\n"
     }
    },
    "source": [
@@ -1120,4 +1120,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 4
-}
+}

plenoptic/__init__.py

@@ -1,7 +1,6 @@
 from . import simulate as simul
 from . import synthesize as synth
 from . import metric
-# from . import learn as learn
 
 from .tools.conv import *
 from .tools.signal import *

plenoptic/metric/perceptual_distance.py

@@ -5,66 +5,12 @@
 
 from ..simulate.canonical_computations import Laplacian_Pyramid, Steerable_Pyramid_Freq
 from ..simulate.canonical_computations import local_gain_control, rectangular_to_polar_dict
+from ..simulate.canonical_computations.filters import circular_gaussian2d
 
 import os
-dirname = os.path.dirname(__file__)
-
-def _gaussian(window_size=11, sigma=1.5):
-    """Normalized, centered Gaussian
-
-    1d Gaussian of size `window_size`, centered half-way, with variable std
-    deviation, and sum of 1.
-
-    With default values, this is the 1d Gaussian used to generate the windows
-    for SSIM
-
-    Parameters
-    ----------
-    window_size : int, optional
-        size of the gaussian
-    sigma : float, optional
-        std dev of the gaussian
-
-    Returns
-    -------
-    window : torch.Tensor
-        1d gaussian
-
-    """
-    x = torch.arange(window_size, dtype=torch.float32)
-    mu = window_size//2
-    gauss = torch.exp(-(x-mu)**2 / (2*sigma**2))
-    return gauss
 
+dirname = os.path.dirname(__file__)
 
-def create_window(window_size=11, n_channels=1):
-    """Create 2d Gaussian window
-
-    Creates 4d tensor containing a 2d Gaussian window (with 1 batch and
-    `n_channels` channels), normalized so that each channel has a sum of 1.
-
-    With default parameters, this is the Gaussian window used to compute the
-    statistics for SSIM.
-
-    Parameters
-    ----------
-    window_size : int, optional
-        height/width of the window
-    n_channels : int, optional
-        number of channels
-
-    Returns
-    -------
-    window : torch.Tensor
-        4d tensor containing the Gaussian windows
-
-    """
-    _1D_window = _gaussian(window_size, 1.5).unsqueeze(1)
-    _2D_window = _1D_window.mm(_1D_window.t()).float().unsqueeze(0).unsqueeze(0)
-    window = _2D_window.expand(n_channels, 1, window_size, window_size).contiguous()
-    # need to keepdim to handle RGB images (broadcasting gets made when trying
-    # to divide something of shape (3, 1, h, w) by something of shape (3, 1))
-    return window / window.sum((-1, -2), keepdim=True)
 
 def _ssim_parts(img1, img2, dynamic_range):
     """Calcluates the various components used to compute SSIM
@@ -119,7 +65,9 @@ def _ssim_parts(img1, img2, dynamic_range):
                             f"{img1.shape}, {img2.shape} instead")
 
     real_size = min(11, height, width)
-    window = create_window(real_size, n_channels=n_channels).to(img1.device)
+    std = torch.tensor(1.5).to(img1.device)
+    window = circular_gaussian2d(real_size, std=std, n_channels=n_channels)
+
     # these two checks are guaranteed with our above bits, but if we add
     # ability for users to set own window, they'll be necessary
     if (window.sum((-1, -2)) > 1).any():

plenoptic/simulate/canonical_computations/__init__.py

@@ -1,5 +1,4 @@
-from .linear import Linear
-from .linear_nonlinear import Linear_Nonlinear
 from .laplacian_pyramid import Laplacian_Pyramid
 from .steerable_pyramid_freq import Steerable_Pyramid_Freq
 from .non_linearities import rectangular_to_polar_dict, local_gain_control
+from .filters import *

plenoptic/simulate/canonical_computations/filters.py

@@ -0,0 +1,87 @@
+from typing import Union, Tuple
+
+import torch
+from torch import Tensor
+
+__all__ = ["gaussian1d", "circular_gaussian2d"]
+
+
+def gaussian1d(kernel_size: int = 11, std: Union[float, Tensor] = 1.5) -> Tensor:
+    """Normalized 1D Gaussian.
+
+    1d Gaussian of size `kernel_size`, centered half-way, with variable std
+    deviation, and sum of 1.
+
+    With default values, this is the 1d Gaussian used to generate the windows
+    for SSIM
+
+    Parameters
+    ----------
+    kernel_size:
+        Size of Gaussian. Recommended to be odd so that kernel is properly centered.
+    std:
+        Standard deviation of Gaussian.
+
+    Returns
+    -------
+    filt:
+        1d Gaussian with `Size([kernel_size])`.
+    """
+    assert std > 0.0, "std must be positive"
+    if isinstance(std, float):
+        std = torch.tensor(std)
+    device = std.device
+
+    x = torch.arange(kernel_size).to(device)
+    mu = kernel_size // 2
+    gauss = torch.exp(-((x - mu) ** 2) / (2 * std ** 2))
+    filt = gauss / gauss.sum()  # normalize
+    return filt
+
+
+def circular_gaussian2d(
+    kernel_size: Union[int, Tuple[int, int]],
+    std: Union[float, Tensor],
+    n_channels: int = 1,
+) -> Tensor:
+    """Creates normalized, centered circular 2D gaussian tensor with which to convolve.
+
+    Parameters
+    ----------
+    kernel_size:
+        Filter kernel size. Recommended to be odd so that kernel is properly centered.
+    std:
+        Standard deviation of 2D circular Gaussian.
+    n_channels:
+        Number of channels with same kernel repeated along channel dim.
+
+    Returns
+    -------
+    filt:
+        Circular gaussian kernel, normalized by total pixel-sum (_not_ by 2pi*std).
+        `filt` has `Size([out_channels=n_channels, in_channels=1, height, width])`.
+    """
+    assert std > 0.0, "stdev must be positive"
+    if isinstance(std, float):
+        std = torch.tensor(std)
+
+    device = std.device
+
+    if isinstance(kernel_size, int):
+        kernel_size = (kernel_size, kernel_size)
+
+    origin = torch.tensor(((kernel_size[0] + 1) / 2.0, (kernel_size[1] + 1) / 2.0))
+    origin = origin.to(device)
+
+    shift_y = torch.arange(1, kernel_size[0] + 1, device=device) - origin[0]
+    shift_x = torch.arange(1, kernel_size[1] + 1, device=device) - origin[1]
+
+    (xramp, yramp) = torch.meshgrid(shift_y, shift_x)
+
+    log_filt = ((xramp ** 2) + (yramp ** 2)) / (-2.0 * std ** 2)
+
+    filt = torch.exp(log_filt)
+    filt = filt / filt.sum()  # normalize
+    filt = torch.stack([filt] * n_channels, dim=0).unsqueeze(1)
+
+    return filt