Merge branch 'release-0.3.0' of github.com:GreenBankObservatory/dysh …

…into issue_256

.gitignore

@@ -1,6 +1,7 @@
 # Custom
 *.fits
 .ignore/
+.virtual_documents
 docs/source/_build/
 
 # Byte-compiled / optimized / DLL files

README.md

@@ -4,7 +4,7 @@
 
 # dysh
 
-*dysh* is a Python spectral line data reduction and analysis program for singledish data with specific emphasis on data from the Green Bank Telescope.  It is currently under development in collaboration between the [Green Bank Observatory](https:/greenbankobservatory.org) and the Laboratory for Millimeter-Wave Astronomy (LMA) at [University of Maryland (UMD)](https://www.astro.umd.edu).  It is intended to be a full replacement for the GBO's current reduction package [GBTIDL](https://www.gb.nrao.edu/GBT/DA/gbtidl/users_guide/).
+*dysh* is a Python spectral line data reduction and analysis program for singledish data with specific emphasis on data from the Green Bank Telescope.  It is currently under development in collaboration between the [Green Bank Observatory](https:/greenbankobservatory.org) and the Laboratory for Millimeter-Wave Astronomy (LMA) at [University of Maryland (UMD)](https://www.astro.umd.edu).  It is intended to be an alternative to the GBO's current reduction package [GBTIDL](https://www.gb.nrao.edu/GBT/DA/gbtidl/users_guide/).
 
 ## Getting Started
 ### Installation

docs/source/Makefile

@@ -0,0 +1,23 @@
+# Minimal makefile for Sphinx documentation
+#
+
+# You can set these variables from the command line.
+SPHINXOPTS    =
+SPHINXBUILD   = python -m sphinx
+SPHINXPROJ    = dysh
+SOURCEDIR     = .
+BUILDDIR      = _build
+
+# Put it first so that "make" without argument is like "make help".
+help:
+	@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+.PHONY: help Makefile
+
+# Catch-all target: route all unknown targets to Sphinx using the new
+# "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
+%: Makefile
+	@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
+
+dev:
+	sphinx-autobuild "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS)

docs/source/examples/subbeamnod.rst

@@ -48,7 +48,7 @@ You can also print a concise (or verbose if you choose `verbose=True`) :meth:`~d
 
 .. code:: python
 
-    >>> sdfits.summary(show_index=True)
+    >>> sdfits.summary()
         SCAN     OBJECT VELOCITY        PROC  PROCSEQN RESTFREQ DOPFREQ # IF # POL # INT # FEED     AZIMUTH   ELEVATIO
     0     32  1256-0547      0.0         Nod         1     26.5    26.5    1     2    60      2  160.975324  43.884984
     1     33  1256-0547      0.0         Nod         2     26.5    26.5    1     2    60      2  161.174093  43.928449
@@ -72,11 +72,12 @@ You can also print a concise (or verbose if you choose `verbose=True`) :meth:`~d
 The SubBeamNod scans are 43, 46, and 54.  Retrieve and calibrate a SubBeamNod scan, then plot it
 
 .. note::
-    For each scan in the summary `dysh` shows the mean of the VELOCITY, RESTFREQ, DOPFREQ, AZIMUTH and ELEVATIO columns, while `GBTIDL` reports the value of the first integration for a scan. If you use `verbo
-    se=True` in `dysh` you get all the integrations.
+    For each scan in the summary `dysh` shows the mean of the VELOCITY, RESTFREQ, DOPFREQ, AZIMUTH and ELEVATIO columns, while `GBTIDL` reports the value of the first integration for a scan. If you use `verbose=True` in `dysh` you get all the integrations.
 
 .. code:: python
 
-    >>> sbn = sdfits.subbeamnod(scan=43, fdnum=1, ifnum=0, weights='tsys')
+    >>> sbn = sdfits.subbeamnod(scan=43, fdnum=1, ifnum=0)
     >>> ta = sbn.timeaverage(weights="tsys")
-    >>> ta.plot()
+    >>> ta.plot(xaxis_unit="GHz")
+
+.. figure:: img/subbeamnod.png

docs/source/examples/totalpower.rst

@@ -55,18 +55,17 @@ You can also print a concise (or verbose if you choose `verbose=True`) summary :
 Retrieve a scan, selecting and IF number and polarization
 
 .. note::
-    For each scan in the summary `dysh` shows the mean of the VELOCITY, RESTFREQ, DOPFREQ, AZIMUTH and ELEVATIO columns, while `GBTIDL` reports the value of the first integration for a scan. If you use `verbo
-    se=True` in `dysh` you get all the integrations.
+    For each scan in the summary `dysh` shows the mean of the VELOCITY, RESTFREQ, DOPFREQ, AZIMUTH and ELEVATIO columns, while `GBTIDL` reports the value of the first integration for a scan. If you use `verbose=True` in `dysh` you get all the integrations.
 
 .. code:: python
 
     >>> tpscan = sdfits.gettp(scan=152, ifnum=0, plnum=0)
 
-The `~dysh.spectra.scan.GBTTPScan` contains the individual integrations.  The system temperatures per integration are calculated from the CALON and CALOFF data
+Inside the returns `~dysh.spectra.scan.ScanBlock` is one `~dysh.spectra.scan.TPScan` that contains the individual integrations.  The system temperatures per integration are calculated from the CALON and CALOFF data
 
 .. code:: python
 
-    >>> print('%s' % (np.array2string(tpscan.tsys, precision=2)))
+    >>> print('%s' % (np.array2string(tpscan[0].tsys, precision=2)))
     [17.46 17.47 17.51 17.48 17.27 17.24 17.43 17.51 17.36 17.41 17.27 17.36
      17.28 17.31 17.15 17.   17.54 17.21 17.4  17.38 17.49 17.43 17.15 17.2
      17.45 17.15 17.31 17.31 17.1  17.48 17.29 17.24 17.52 17.31 17.19 17.1

docs/source/for_beta_testers/beta_testing.rst

@@ -14,9 +14,9 @@ If you are interested in beta testing `dysh`, please contact Pedro Salas (psalas
 Providing feedback
 ==================
 
-If you encounter a problem with `dysh`, would like to request a new feature or enhancement or would like to leave feedback, please do so using `GitHub issues <https://github.com/GreenBankObservatory/dysh/issues>`_. There are some basic instructions of how to do this :ref:`here <for_developers/github_integrations:issues>`. This requires `creating a free account <https://github.com/>`_ on GitHub if you do not have one.
+If you encounter a problem with `dysh`, would like to request a new feature or enhancement or would like to leave feedback, please do so using `GitHub issues <https://github.com/GreenBankObservatory/dysh/issues>`_. There are some basic instructions of how to do this :ref:`here <githubissues>`. This requires `creating a free account <https://github.com/>`_ on GitHub if you do not have one.
 
-If you prefer not to create a GitHub account, please provide your feedback to the `dysh-beta mailing list <https://groups.google.com/g/dysh-beta/about>`_, or send an email to dysh-beta@googlegroups.com. Additionally, we will provide a `form for collecting feedback <https://forms.gle/27tg9adfLbDnUyz37>`_.
+If you prefer not to create a GitHub account, please provide your feedback to the `dysh-beta mailing list <https://groups.google.com/g/dysh-beta/about>`_, or send an email to dysh-beta@googlegroups.com. Additionally, we will provide a `form for collecting feedback <https://forms.gle/jNaiMTcXBiHAQUim7>`_.
 
 When providing feedback, please provide
 
@@ -40,33 +40,33 @@ Here are examples of feedback on GitHub
 * reporting an issue, `Issue #88 <https://github.com/GreenBankObservatory/dysh/issues/88>`_
 * requesting a modification, `Issue #78 <https://github.com/GreenBankObservatory/dysh/issues/78>`_
 
-.. _beta-install
+.. _beta-install:
 
 Installing `dysh`
 =================
 
 Here we provide additional installation steps that include creating a virtual environment to keep `dysh` isolated from your system `Python` version.
-We provide steps for working in one of `GBO data reduction hosts <https://greenbankobservatory.org/science/gbt-observers/public-access-data-reduction/>`_ (e.g., fourier), and if you're working outside one of the GBO data reduction hosts.
+We provide steps for working in one of `GBO data reduction hosts <https://greenbankobservatory.org/science/gbt-observers/public-access-data-reduction/>`_ (e.g., fourier), and if you're working outside one of the GBO data reduction hosts. The provided command will install `dysh` and `jupyterlab`, which can be used to run the example notebooks locally.
 
 .. tab:: At GBO
 
     Create a `Python3.11` virtual environment
 
     .. code:: bash
 
-        /users/gbosdd/python/bin/python3.11 -m venv /home/scratch/$USER/dysh-0.2-env
+        /users/gbosdd/python/bin/python3.11 -m venv /home/scratch/$USER/dysh-0.3-env
 
     Activate your virtual environment
 
     .. code:: bash
 
-        source /home/scratch/$USER/dysh-0.2-env/bin/activate
+        source /home/scratch/$USER/dysh-0.3-env/bin/activate
 
     Install `dysh` and Jupyter Lab
 
     .. code:: bash
 
-        pip install jupyterlab dysh==0.2.0b
+        pip install "dysh[nb]==0.3.0b"
 
 .. tab:: Outside of GBO
 
@@ -86,6 +86,16 @@ We provide steps for working in one of `GBO data reduction hosts <https://greenb
 
     .. code:: bash
 
-        pip install jupyterlab dysh==0.2.0b
+        pip install "dysh[nb]==0.3.0b"
 
 In the future we will provide `dysh` executables at GBO.
+
+Previous beta releases
+======================
+
+Feedback on previous beta releases is also welcome. Here you can find links to previous beta release documents.
+
+0.2.0
+-----
+
+`Instructions <https://docs.google.com/document/d/1RrHaiwmrDnPbMLdNY99_hBZzyWyYKsw0UCM8FKqhIKo/edit?usp=sharing>`_ and `questionnaire <https://forms.gle/27tg9adfLbDnUyz37>`_.

docs/source/for_developers/github_integrations.rst

@@ -7,6 +7,8 @@ Actions
 
 Tests should be named `test_*.py` and located within a `tests` directory of their parent module. To define a workflow that runs the tests, create a new `.yml` file in `dysh/.github/workflows/`.
 
+.. _githubissues:
+
 Issues
 ======
 

docs/source/getting_started/tutorials/GBT/tutorial-positionswitch.rst

@@ -81,19 +81,15 @@ The following lines will let you calibrate and time average the position switche
 
 .. code:: python
 
-    >>> psscan = sdfits.getps(152, ifnum=0, plnum=0)
+    >>> psscan = sdfits.getps(scan=152, ifnum=0, plnum=0)
     >>> ta = psscan.timeaverage(weights='tsys')
 
 Plotting the calibrated data
 ----------------------------
 
 .. code:: python
 
-    >>> ta[0].plot(xaxis_unit="km/s", yaxis_unit="mK", ymin=-100, ymax=500, xmin=3000, xmax=4500)
+    >>> ta.plot(xaxis_unit="km/s", yaxis_unit="mK", ymin=-100, ymax=500, xmin=3000, xmax=4500)
 
 .. figure:: img/ps_152_zoom.png
     :alt: The spectrum plot zoomed in along both axes to frame a central emission line.
-
-.. WARNING::
-    At this point, `dysh` does not handle Doppler corrections.
-    So the frequency and velocity information will be offset for observations requesting a reference frame other than Topocentric.

docs/source/index.rst

@@ -14,6 +14,7 @@ Contents
 .. toctree::
    :maxdepth: 3
 
+   whatsnew/0.3
    background/index
    getting_started/index
    examples/index

docs/source/whatsnew/0.3.rst

@@ -0,0 +1,21 @@
+
+.. _whatsnew-0.3:
+
+***********************
+What's New in dysh 0.3?
+***********************
+
+Overview
+========
+In this release, `dysh` has the following new or updated features:
+
+* Conversion between standard velocity frames: heliocentric, barycentric, geocentric, topcentric, local standard of rest (kinematic and dynamical), and galactocentric rest frames.
+* Conversion between the radio, optical, and relativistic velocity definitions.
+* Data selection via any SDFITS column name.
+* Frequency switching calibration.
+
+Full change log
+===============
+
+To see a detailed list of all changes in version 0.3, including changes in
+API, please see the :ref:`changelog`.

docs/source/whatsnew/CHANGES.rst

@@ -0,0 +1,25 @@
+
+Version 0.3.0
+==============
+
+Functionality & Enhancements
+---------------------------------
+- Handling of Doppler frames and conventions
+    - A GBT Spectrum has a spectral axis derived from the WCS of the spectrum (which in turn is created from meta data in the SDFITS file).  The default Doppler frame for this axis is topocentric.  The spectral axis of a Spectrum can be converted to standard frames recognized by astropy: LSRK, HCRS, ICRS, GCRS, ITRS, GalactoCentric. See `Spectrum.set_frame <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.spectra.html#dysh.spectra.spectrum.Spectrum.set_frame>`_, `Spectrum.with_frame <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.spectra.html#dysh.spectra.spectrum.Spectrum.with_frame>`_, and also the `xaxis_unit` and `vel_frame` keywords to `SpecPlot.plot.  <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.plot.html#dysh.plot.specplot.SpectrumPlot.plot>`_
+    -  The Doppler conventions *radio, optical, relativistic* are recognized by `dysh`.  Users can convert a Spectrum to different conventions with `Spectrum.set_convention <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.spectra.html#dysh.spectra.spectrum.Spectrum.set_convention>`_ and `Spectrum.with_velocity_convention <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.spectra.html#dysh.spectra.spectrum.Spectrum.set_convention>`_.  See also, the `doppler_convention` keyword of  `SpecPlot.plot <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.plot.html#dysh.plot.specplot.SpectrumPlot.plot>`_
+- Data Selection
+    - The `Selection <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.util.html#dysh.util.selection.Selection>`_ class implements a very flexible way of selecting data rows from an SDFITS file using any column name.  (Column name aliases are also supported).  Multiple selection rules are logically combined to a final selection.
+    - Data selection is implemented on `GBTFITSLoad <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.fits.html#module-dysh.fits.gbtfitsload>`_ via delegation to a Selection attribute.
+- Frequency switching calibration
+    - FS data can be calibrated using `GBTFITSLoad.getfs() <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.fits.html#dysh.fits.gbtfitsload.GBTFITSLoad.getfs>`_  with the option to fold the signal and reference spectra.
+    - The Scan class for calibrating frequency switching, `FSScan <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.spectra.html#dysh.spectra.scan.FSScan>`_, has been implemented.  Users should not need to create these directly, but rather through *getfs()*.
+
+- `ScanBlock <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.spectra.html#dysh.spectra.scan.ScanBlock>`_  API change
+    - `timeaverage() <https://dysh.readthedocs.io/en/release-0.3.0/modules/dysh.spectra.html#dysh.spectra.scan.ScanBlock.timeaverage>`_ now returns a Spectrum instead of a list.  Previously the list contained the time average of each Scan within the ScanBlock.   Now the time average across all Scans in the ScanBlock is returned.
+
+Bug Fixes
+-----------
+-  `SubBeamNod error when using cycle method <https://github.com/GreenBankObservatory/dysh/issues/207>`_
+-  `Spectrum arithmetic operations not working <https://github.com/GreenBankObservatory/dysh/issues/208>`_
+-  `SDFITS summary() reports wrong number of integrations <https://github.com/GreenBankObservatory/dysh/issues/211>`_
+- `Certain old GBTIDL files could not be read by dysh <https://github.com/GreenBankObservatory/dysh/issues/216>`_

docs/source/whatsnew/changelog.rst

@@ -0,0 +1,7 @@
+
+.. _changelog:
+
+Full Changelog
+===============
+
+.. include:: CHANGES.rst

docs/source/whatsnew/index.rst

@@ -0,0 +1,16 @@
+
+.. _dysh_whats_new_index:
+
+**********
+What's New
+**********
+
+Examples in these documents are frozen in time to respect the status of the
+API at the time of the release they are describing. Please refer to the
+main, up-to-date documentation if you run into any issues with the
+functionality highlighted in these pages.
+
+.. toctree::
+   :maxdepth: 1
+
+* What's new in Dysh 0.3?

src/dysh/__init__.py

@@ -1,6 +1,6 @@
 """Top-level package for dysh."""
 
-__version__ = "0.2.1"
+__version__ = "0.3.0b"
 
 all = ["version"]
 

src/dysh/fits/gbtfitsload.py

@@ -560,7 +560,7 @@ def _create_index_if_needed(self):
                 i = i + 1
 
     def info(self):
-        """Return information on the HDUs contained in this object. See :meth:`~astropy.HDUList/info()"""
+        """Return information on the HDUs contained in this object. See :meth:`~astropy.HDUList/info()`"""
         for s in self._sdf:
             s.info()
 
@@ -596,12 +596,10 @@ def getfs(
             Average the scans in polarization.
             The default is False.
         weights : str or None, optional
-            How to weight the spectral data when averaging.  `tsys` means use system
+            How to weight the spectral data when averaging.  'tsys' means use system
             temperature weighting (see e.g., :meth:`~spectra.scan.FSScan.timeaverage`);
             None means uniform weighting.
-            The default is "tsys".
-        fold: boolean, optional
-            The default is True
+            The default is 'tsys'.
         bintable : int, optional
             Limit to the input binary table index. The default is None which means use all binary tables.
         observer_location : `~astropy.coordinates.EarthLocation`
@@ -610,7 +608,7 @@ def getfs(
             observation DATE-OBS or MJD-OBS in
             the SDFITS header.  The default is the location of the GBT.
         **kwargs : dict
-            Optional additional selection (only?) keyword arguments, typically
+            Optional additional selection keyword arguments, typically
             given as key=value, though a dictionary works too.
             e.g., `ifnum=1, plnum=[2,3]` etc.
 
@@ -722,14 +720,14 @@ def getps(self, calibrate=True, timeaverage=True, polaverage=False, weights="tsy
         polaverage : boolean, optional
             Average the scans in polarization. The default is False.
         weights : str or None, optional
-            How to weight the spectral data when averaging.  `tsys` means use system
+            How to weight the spectral data when averaging.  'tsys' means use system
             temperature weighting (see e.g., :meth:`~spectra.scan.PSScan.timeaverage`);
-            None means uniform weighting. The default is "tsys".
+            None means uniform weighting. The default is 'tsys'.
         bintable : int, optional
             Limit to the input binary table index. The default is None which means use all binary tables.
             (This keyword should eventually go away)
         **kwargs : dict
-            Optional additional selection (only?) keyword arguments, typically
+            Optional additional selection keyword arguments, typically
             given as key=value, though a dictionary works too.
             e.g., `ifnum=1, plnum=[2,3]` etc.
 
@@ -890,7 +888,7 @@ def gettp(
         bintable : int, optional
             Limit to the input binary table index. The default is None which means use all binary tables.
         **kwargs : dict
-            Optional additional selection (only?) keyword arguments, typically
+            Optional additional selection  keyword arguments, typically
             given as key=value, though a dictionary works too.
             e.g., `ifnum=1, plnum=[2,3]` etc.
 
@@ -999,7 +997,7 @@ def subbeamnod(
         bintable : int, optional
             Limit to the input binary table index. The default is None which means use all binary tables.
         **kwargs : dict
-            Optional additional selection (only?) keyword arguments, typically
+            Optional additional selection keyword arguments, typically
             given as key=value, though a dictionary works too.
             e.g., `ifnum=1, plnum=[2,3]` etc.
 

src/dysh/spectra/scan.py

@@ -523,19 +523,17 @@ def timeaverage(self, weights="tsys"):
         return self._timeaveraged
 
 
+#        @todo   'scans' should become 'scan'
 class PSScan(ScanMixin):
     """GBT specific version of Position Switch Scan. A position switch scan object has
     one IF, one feed, and one or more polarizations.
 
     Parameters
     ----------
-
-    gbtfits : `~fit.gbtfitsload.GBFITSLoad`
+    gbtfits : `~fits.gbtfitsload.GBFITSLoad`
         input GBFITSLoad object
     scans : dict
-        dictionary with keys 'ON' and 'OFF' containing unique list of ON (signal) and OFF (reference) scan numbers
-        NOTE: there should be one ON and one OFF, a pair
-        @todo   'scans' should become 'scan'
+        dictionary with keys 'ON' and 'OFF' containing unique list of ON (signal) and OFF (reference) scan numbers NOTE: there should be one ON and one OFF, a pair
     scanrows : dict
         dictionary with keys 'ON' and 'OFF' containing the list of rows in `sdfits` corresponding to ON (signal) and OFF (reference) integrations
     calrows : dict
@@ -609,6 +607,7 @@ def scans(self):
         """The dictionary of the ON and OFF scan numbers in the PSScan.
 
         Returns
+        -------
         scans : dict
             The scan number dictionary
         """