Plugin Docs: Python Tools & Matplotlib

docs/requirements.txt

@@ -3,6 +3,7 @@ recommonmark
 sphinx==1.7
 breathe>=4.5
 sphinxcontrib.programoutput
+sphinxcontrib.napoleon>=1.8.1
 pygments
 # generate plots
 matplotlib

docs/source/conf.py

@@ -19,8 +19,9 @@
 import os
 import subprocess
 from recommonmark.parser import CommonMarkParser
-# import sys
-# sys.path.insert(0, os.path.abspath('.'))
+import sys
+python_libs = os.path.abspath('../../lib/python')
+sys.path.insert(0, python_libs)
 
 
 # -- General configuration ------------------------------------------------
@@ -38,13 +39,17 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = ['sphinx.ext.mathjax',
+              'sphinx.ext.napoleon',
               'breathe',
               'sphinxcontrib.programoutput',
               'matplotlib.sphinxext.plot_directive']
 
 if not on_rtd:
     extensions.append('sphinx.ext.githubpages')
 
+# napoleon autodoc config
+napoleon_include_init_with_doc = True
+
 # breathe config
 breathe_projects = {'PIConGPU': '../xml'}
 breathe_default_project = 'PIConGPU'

docs/source/dev/py_postprocessing.rst

@@ -0,0 +1,69 @@
+.. _development-pytools:
+
+Python Postprocessing Tool Structure
+====================================
+
+Each plugin should implement at least the following Python classes.
+
+1. A data reader class responsible for loading the data from the simulation directory
+2. A visualizer class that outputs a matplotlib plot
+
+The repository directory for PIConGPU Python modules for plugins is ``lib/python/picongpu/plugins/``.
+
+Data Reader
+~~~~~~~~~~~
+
+The data readers should reside in the ``lib/python/picongpu/plugins/data`` directory.
+There is a base class in ``base_reader.py`` defining the interface of a reader.
+Each reader class should derive from this class and needs to implement the following interface functions:
+
+.. autoclass:: picongpu.plugins.data.base_reader.DataReader
+   :members:
+   :private-members:
+
+To shorten the import statements for the readers, please also add an entry in the ``__init__.py`` file of the ``data`` directory.
+
+Visualizer
+~~~~~~~~~~
+
+The visualizers should reside in the ``lib/python/picongpu/plugins/plot_mpl/`` directory.
+The module names should end on ``_visualizer.py`` and the class name should only be ``Visualizer``.
+
+To shorten the import statements for the visualizers, please also add an entry in the ``__init__.py`` file of the ``plot_mpl`` directory.
+
+There is a base class for visualization found in ``base_visualizer.py`` which already handles the plotting logic.
+It uses the data reader classes for accessing the data.
+After getting the data, it ensures that (for performance reasons) a matplotlib artist is created only for the first plot and later only gets updated with fresh data.
+
+.. autoclass:: picongpu.plugins.plot_mpl.base_visualizer.Visualizer
+   :members:
+   :private-members:
+
+The complete implementation logic of the ``visualize`` function is pretty simple.
+
+.. code:: python
+
+   def visualize(self, **kwargs):
+        self.data = self.data_reader.get(**kwargs)
+        if self.plt_obj is None:
+            self._create_plt_obj()
+        else:
+            self._update_plt_obj()
+
+All new plugins should derive from this class.
+
+When implementing a new visualizer you have to perform the following steps:
+
+1. Let your visualizer class inherit from the ``Visualizer`` class in ``base visualizer.py``.
+
+2. Implement the ``_create_data_reader(self, run_directory)`` function.
+This function should return a data reader object (see above) for this plugin's data.
+
+3. Implement the ``_create_plt_obj(self)`` function.
+This function needs to access the plotting data from the ``self.data`` member (this is the data structure as returned by the data readers ``.get(...)`` function, create some kind of matplotlib artist by storing it in the ``self.plt_obj`` member variable and set up other plotting details (e.g. a colorbar).
+
+4. Implement the ``_update_plt_obj(self)`` function.
+This is called only after a valid ``self.plt_obj`` was created.
+It updates the matplotlib artist with new data.
+Therefore it again needs to access the plotting data from the ``self.data`` member and call the data update API for the matplotlib artist (normally via ``.set_data(...)``.
+

docs/source/usage/plugins.rst

@@ -61,6 +61,19 @@ Examples
 * ``42,30:50:10``: at steps 30 40 42 50 84 126 168 ...
 * ``5,10``: at steps 0 5 10 15 20 25 ... (only executed once per step in overlapping intervals)
 
+Python Postprocessing
+=====================
+
+In order to further work with the data produced by a plugin during a simulation run, PIConGPU provides python tools that can be used for reading data and visualization.
+They can be found under ``lib/python/picongpu/plugins``.
+
+It is our goal to provide at least two modules for each plugin to make postprocessing as convenient as possible:
+1. a data reader (inside the ``data`` subdirectory)
+2. a matplotlib visualizer (inside the ``plot_mpl`` subdirectory)
+
+Further information on how to use these tools can be found at each plugin page.
+
+If you would like to help in developing those classes for a plugin of your choice, please read :ref:`python postprocessing <development-pytools>`.
 
 .. rubric:: Footnotes
 

docs/source/usage/plugins/energyHistogram.rst

@@ -26,7 +26,7 @@ PIConGPU command line option                description
                                             A value of ``100`` would mean an output at simulation time step *0, 100, 200, ...*.
                                             If set to a non-zero value, the energy histogram of all **electrons** is computed.
                                             By default, the value is ``0`` and no histogram for the electrons is computed.
-``--e_energy.filter``                       Use filtered particles. Available filters are set up in 
+``--e_energy.filter``                       Use filtered particles. Available filters are set up in
                                             :ref:`particleFilters.param <usage-params-core>`.
 ``--e_energyHistogram.binCount``            Specifies the number of bins used for the **electron** histogram.
                                             Default is ``1024``.
@@ -39,18 +39,18 @@ PIConGPU command line option                description
 
 .. note::
 
-   This plugin is a multi plugin. 
+   This plugin is a multi plugin.
    Command line parameter can be used multiple times to create e.g. dumps with different dumping period.
    In the case where an optional parameter with a default value is explicitly defined the parameter will be always passed to the instance of the multi plugin where the parameter is not set.
-   e.g. 
+   For example,
 
    .. code-block:: bash
 
       --e_energyHistogram.period 128 --e_energyHistogram.filter all --e_energyHistogram.maxEnergy 10
       --e_energyHistogram.period 100 --e_energyHistogram.filter all --e_energyHistogram.maxEnergy 20 --e_energyHistogram.binCount 512
 
    creates two plugins:
- 
+
    #. create an electron histogram **with 512 bins** each 128th time step.
    #. create an electron histogram **with 1024 bins** (this is the default) each 100th time step.
 
@@ -74,50 +74,73 @@ The histograms are stored in ASCII files in the ``simOutput/`` directory.
 
 The file for the electron histogram is named ``e_energyHistogram.dat`` and for all other species ``<species>_energyHistogram.dat`` likewise.
 The first line of these files does not contain histogram data and is commented-out using ``#``.
-It describes the energy binning that needed to interpret the following data. 
-It can be seen as the head of the following data table. 
-The first column is an integer value describing the simulation time step. 
-The second column counts the number of real particles below the minimum energy value used for the histogram. 
-The following columns give the real electron count of the particles in the specific bin described by the first line/header. 
+It describes the energy binning that needed to interpret the following data.
+It can be seen as the head of the following data table.
+The first column is an integer value describing the simulation time step.
+The second column counts the number of real particles below the minimum energy value used for the histogram.
+The following columns give the real electron count of the particles in the specific bin described by the first line/header.
 The second last column gives the number of real particles that have a higher energy than the maximum energy used for the histogram.
-The last column gives the total number of particles. 
+The last column gives the total number of particles.
 In total there are 4 columns more than the number of bins specified with command line arguments.
 Each row describes another simulation time step.
 
 Analysis Tools
 ^^^^^^^^^^^^^^
 
-You can quickly plot the data in Python with:
+Data Reader
+"""""""""""
+You can quickly load and interact with the data in Python with:
 
 .. code:: python
 
    from picongpu.plugins.data import EnergyHistogramData
-   import matplotlib.pyplot as plt
+
 
    # load data
-   energy_histogram = EnergyHistogramData('/home/axel/runs/lwfa_001')
-   counts, bins = energy_histogram.get('e', species_filter='all', iteration=2000)
+   eh_data = EnergyHistogramData('/home/axel/runs/lwfa_001')
+   counts, bins_keV = eh_data.get('e', species_filter='all', iteration=2000)
+
+Matplotlib Visualizer
+"""""""""""""""""""""
+
+You can quickly plot the data in Python with:
+
+.. code:: python
+
+   from picongpu.plugins.plot_mpl import EnergyHistogramMPL
+   import matplotlib.pyplot as plt
 
-   # unit conversion
-   MeV = 1.e-3  # keV to MeV
 
-   # plotting
-   plt.plot(bins * MeV, counts)
+   # create a figure and axes
+   fig, ax = plt.subplots(1, 1)
 
-   # range
-   ax = plt.gca()
-   # log scale example
-   # ax.set_yscale('log')
-   # ax.set_ylim([1.e7, 1.e12])
+   # create the visualizer
+   eh_vis = EnergyHistogramMPL('path/to/run_dir', ax)
 
-   # annotations
-   ax.set_xlabel(r'E$_\mathrm{kin}$ [MeV]')
-   ax.set_ylabel(r'count [arb.u.]')
+   eh_vis.visualize(iteration=200, species='e')
 
    plt.show()
 
+The visualizer can also be used from the command line by writing
+
+ .. code:: bash
+
+    python energy_histogram_visualizer.py
+
+with the following command line options
+
+================================     ======================================================
+Options                              Value
+================================     ======================================================
+-p                                   Path to the run directory of a simulation.
+-i                                   An iteration number
+-s (optional, defaults to 'e')       Particle species abbreviation (e.g. 'e' for electrons)
+-f (optional, defaults to 'all')     Species filter string
+================================     ======================================================
+
+
 
-Alternatively, PIConGPU comes with a command line analysis tool for the energy histograms. 
+Alternatively, PIConGPU comes with a command line analysis tool for the energy histograms.
 It is based on *gnuplot* and requires that gnuplot is available via command line.
 The tool can be found in ``src/tools/bin/`` and is called ``BinEnergyPlot.sh``.
 It accesses the gnuplot script ``BinEnergyPlot.gnuplot`` in ``src/tools/share/gnuplot/``.

docs/source/usage/plugins/phaseSpace.rst

@@ -58,48 +58,71 @@ A file is created per species, phasespace selection and time step.
 Values are given as *charge density* per phase space bin.
 In order to scale to a simpler *charge of particles* per :math:`\mathrm{d}r_i` and :math:`\mathrm{d}p_i` -bin multiply by the cell volume ``dV``.
 
-Analysis
-^^^^^^^^
+Analysis Tools
+^^^^^^^^^^^^^^
 
-The easiest way is to load the data in Python:
+Data Reader
+"""""""""""
+You can quickly load and interact with the data in Python with:
 
 .. code:: python
 
    from picongpu.plugins.data import PhaseSpaceData
-   import matplotlib.pyplot as plt
-   from matplotlib.colors import LogNorm
    import numpy as np
 
 
    # load data
-   phase_space = PhaseSpaceData('/home/axel/runs/foil_001')
-   e_ps, e_ps_meta = phase_space.get('e', species_filter='all', iteration=1000, ps='ypy')
+   ps_data = PhaseSpaceData('/home/axel/runs/lwfa_001')
+   ps, meta = ps_data.get(species='e', species_filter='all', ps='ypy', iteration=2000)
 
    # unit conversion from SI
    mu = 1.e6  # meters to microns
    e_mc_r = 1. / (9.109e-31 * 2.9979e8)  # electrons: kg * m / s to beta * gamma
 
-   # plotting
-   plt.imshow(
-       np.abs(e_ps).T * e_ps_meta.dV,
-       extent = e_ps_meta.extent * [mu, mu, e_mc_r, e_mc_r],
-       interpolation = 'nearest',
-       aspect = 'auto',
-       origin='lower',
-       norm = LogNorm()
-   )
+   Q_dr_dp = np.abs(e_ps) * e_ps_meta.dV  # C s kg^-1 m^-2
+   extent = e_ps_meta.extent * [mu, mu, e_mc_r, e_mc_r]  # spatial: microns, momentum: beta*gamma
+
+Note that the spatial extent of the output over time might change when running a moving window simulation.
+
+Matplotlib Visualizer
+"""""""""""""""""""""
 
-   # annotations
-   cbar = plt.colorbar()
-   cbar.set_label(r'$Q / \mathrm{d}r \mathrm{d}p$ [$\mathrm{C s kg^{-1} m^{-2}}$]')
+You can quickly plot the data in Python with:
+
+.. code:: python
+
+   from picongpu.plugins.plot_mpl import PhaseSpaceMPL
+   import matplotlib.pyplot as plt
 
-   ax = plt.gca()
-   ax.set_xlabel(r'${0}$'.format(e_ps_meta.r) + r' [$\mathrm{\mu m}$]')
-   ax.set_ylabel(r'$p_{0}$ [$\beta\gamma$]'.format(e_ps_meta.p))
+
+   # create a figure and axes
+   fig, ax = plt.subplots(1, 1)
+
+   # create the visualizer
+   ps_vis = PhaseSpaceMPL('path/to/run_dir', ax)
+
+   # plot
+   ps_vis.visualize(iteration=200, species='e')
 
    plt.show()
 
-Note that the spatial extent of the output over time might change when running a moving window simulation.
+The visualizer can also be used from the command line by writing
+
+ .. code:: bash
+
+    python phase_space_visualizer.py
+
+with the following command line options
+
+================================     =======================================================
+Options                              Value
+================================     =======================================================
+-p                                   Path and filename to the run directory of a simulation.
+-i                                   An iteration number
+-s (optional, defaults to 'e')       Particle species abbreviation (e.g. 'e' for electrons)
+-f (optional, defaults to 'all')     Species filter string
+-m (optional, defaults to 'ypy')     Momentum string to specify the phase space
+================================     =======================================================
 
 Out-of-Range Behavior
 ^^^^^^^^^^^^^^^^^^^^^