Release pyDARNio v1.3 (#72)

# Scope 

Support of Borealis v0.7 files and `snd` files, removal of `deepdish` dependency.

## Changes

* `deepdish` library no longer maintained, introducing breaking changes with new numpy version. This dependency has been removed.
* `snd` files for sounding data are now supported.
* Borealis v0.7 files are now supported.
* Bumps Python version requirement to Python 3.8+

----------------------
Co-authored-by: Carley <60905856+carleyjmartin@users.noreply.github.com>
Co-authored-by: egthomas <evan.g.thomas@dartmouth.edu>
Co-authored-by: Theodore Kolkman <tjk584@usask.ca>

.gitignore

@@ -2,6 +2,7 @@
 __pycache__/
 *.py[cod]
 *$py.class
+*.DS_Store
 
 # C extensions
 *.so

.readthedocs.yml

@@ -0,0 +1,14 @@
+# Read the Docs configuration file for MkDocs projects
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+# Set the version of Python and other tools you might need
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.12"
+
+mkdocs:
+  configuration: mkdocs.yml

.zenodo.json

@@ -1,5 +1,5 @@
 {
-    "creators":[
+    "creators": [
         {
             "name": "SuperDARN Data Visualization Working Group"
         },
@@ -8,47 +8,44 @@
             "name": "Rohel, R.A.",
             "orcid": "0000-0003-2208-1553"
         },
-        {
-            "affiliation": "The University Centre in Svalbard",
-            "name": "Bland, E.C.",
-            "orcid": "0000-0002-0252-0400"
-        },
         {
             "affiliation": "University of Saskatchewan",
             "name": "Martin, C.J.",
             "orcid": "0000-0002-8278-9783"
         },
         {
-            "affiliation": "John Hopkins University",
-            "name": "Chartier, A.",
-            "orcid": "0000-0002-4215-031X"
+            "affiliation": "Dartmouth College",
+            "name": "Thomas, E.G.",
+            "orcid": "0000-0001-8036-8793"
         },
         {
             "affiliation": "University of Saskatchewan",
-            "name": "Krieger, K.J."
+            "name": "Kolkman, T.J."
         },
         {
             "affiliation": "University of Saskatchewan",
-            "name": "Billett, D.D.",
-            "orcid": "0000-0002-8905-8609"
+            "name": "Lozinsky, A.R.",
+            "orcid": "0000-0002-2786-8120"
         },
         {
-            "affiliation": "US Naval Research Laboratory",
-            "name": "Burrell, A.G.",
-            "orcid": "0000-0001-8875-9326"
+            "affiliation": "University of Saskatchewan",
+            "name": "Billett, D.D.",
+            "orcid": "0000-0002-8905-8609"
         },
         {
-            "affiliation": "University of Saskatchewan",
-            "name": "Detwiller, M.H."
+            "affiliation": "The University Centre in Svalbard",
+            "name": "Bland, E.C.",
+            "orcid": "0000-0002-0252-0400"
         },
         {
-            "affiliation": "University of Saskatchewan",
-            "name": "Kotyk, K."
+            "affiliation": "John Hopkins University",
+            "name": "Chartier, A.",
+            "orcid": "0000-0002-4215-031X"
         },
         {
-            "affiliation": "University of Saskatchewan",
-            "name": "Schmidt, M.T.",
-            "orcid": "0000-0002-3265-977X"
+            "affiliation": "US Naval Research Laboratory",
+            "name": "Burrell, A.G.",
+            "orcid": "0000-0001-8875-9326"
         },
         {
             "affiliation": "Virginia Tech",

README.md

@@ -1,23 +1,18 @@
 ![pyDARNio](https://raw.githubusercontent.com/SuperDARN/pyDARNio/master/docs/imgs/pydarnio_logo.png)
 
 [![License: LGPL v3](https://img.shields.io/badge/License-LGPLv3-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0) 
-[![Python 3.6](https://img.shields.io/badge/python-3.6-blue.svg)](https://www.python.org/downloads/release/python-360/) 
+[![Python 3.8](https://img.shields.io/badge/python-3.8-blue.svg)](https://www.python.org/downloads/release/python-360/) 
 ![GitHub release (latest by date)](https://img.shields.io/github/v/release/superdarn/pyDARNio)
 [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.4009470.svg)](https://doi.org/10.5281/zenodo.4009470)
 
 Python data IO library for the Super Dual Auroral Radar Network (SuperDARN).
 
 ## Changelog
 
-## Version 1.2.1 - Patch Release!
+## Version 1.3 - Release!
 
-This patch release includes a fix for the depreciation of a Numpy method.
-Below is more information for the most recent minor release.
-
-The most recent minor release included:
-- Handling of extra fields in fitacf files coming in the next RST release, including the name of the ACF fitting algorithm used (`algorithm`), the value of TDIFF used to calculate the elevation angles (`tdiff`), and more descriptive field names for the elevation angle fields in FitACF 3.0 (`elv_error` & `elv_fitted`)
-- More flexibility around optional fields for all file types
-- Updates and speed improvements for Borealis file io
+This release includes changes to support Borealis v0.7 files,
+``snd`` files, and removes the deprecated ``deepdish`` dependency.
 
 ## Documentation
 

docs/index.md

@@ -8,12 +8,13 @@ Currently the library support reading for DMAP files:
 - FITACF
 - GRID
 - MAP
+- SND
 
 As well as Borealis HDF5 files:
 
-- IQDAT
-- BeamIQ
-- RAWACF
+- antennas_iq
+- bfiq
+- rawacf
 
 There is also utilities for converting Borealis files to SuperDARN DMAP file structures. 
 
@@ -29,9 +30,7 @@ If you have any questions or concerns please submit an **Issue** on the SuperDAR
   - [Citing](user/citing.md)
   - Tutorials 
     - IO 
-        - [Read SuperDARN files](user/SDarnRead.md)
-        - [Write SuperDARN files](user/SDarnWrite.md)
-        - [Read/Write Borealis files](user/BorealisIO.md)
+      - [SuperDARN (DMAP) files](user/SDarn.md)
+      - [Borealis files](user/BorealisIO.md)
     - Convert
-      - [Borealis to SuperDARN](user/Borealis2SuperDARN.md)
-    - [Radar and Hardware Information](user/hardware.md)
+      - [Borealis to SuperDARN (DMAP)](user/Borealis2SuperDARN.md)

docs/user/Borealis2SuperDARN.md

@@ -45,8 +45,6 @@ Other information can be gathered from the converter if desired, for example:
 
 ```python
 borealis_array_data = converter.arrays
-
 sdarn_dictionary = converter.sdarn_dict # python dictionary of the SDarn standard fields.
-
 dmap_records = converter.sdarn_dmap_records
 ```

docs/user/SDarnRead.md → docs/user/SDarn.md

@@ -1,4 +1,4 @@
-# Reading in DMap structured SuperDARN data files
+# DMap structured SuperDARN data file I/O
 ---
 
 Data Map (DMap) is a binary self-describing format that was developed by Rob Barnes. 
@@ -10,6 +10,7 @@ Types of files used by SuperDARN which are usually accessed in DMap format are:
 - FITACF
 - GRID/GRD
 - MAP
+- SND
 
 This tutorial will focus on reading in DMap structured files using pyDARNio, including how to read compressed files and access common data fields.
 
@@ -20,48 +21,48 @@ The basic code to read in a DMap structured file is as follows:
 import pydarnio
 
 file = "path/to/file"
-SDarn_read = pydarnio.SDarnRead(file)
+reader = pydarnio.SDarnRead(file)
 ```
-which puts the file contents into a Python class called `SDarn_read`. 
+which puts the file contents into a Python object called `reader`. 
 
 Now you need to tell it what kind of file it is. For instance, if the file you were reading in is a FITACF file, you would write something like:
 ```python
-fitacf_data = SDarn_read.read_fitacf()
+fitacf_data = reader.read_fitacf()
 ```
-where the named variable `fitacf_data` is a python dictionary list containing all the data in the file. If you were reading a different kind of file, you would need to use the methods `read_iqdat`, `read_rawacf`, `read_grid` or `read_map` for their respective filetypes.
+where the named variable `fitacf_data` is a python dictionary list containing all the data in the file. If you were reading a different kind of file, you would need to use the methods `read_iqdat`, `read_rawacf`, `read_grid`, `read_map`, or `read_snd` for their respective filetypes.
 
-## Reading a compressed file
+### Reading a compressed file
 
 To read a compressed file like **bz2** (commonly used for SuperDARN data products), you will need to use [bz2 library](https://docs.python.org/3/library/bz2.html). 
 The `SDarnRead` class allows the user to provide the file data as a stream of data which is what the **bz2** returns when it reads a compressed file: 
 ```python
 import bz2
 import pydarnio
 
-fitacf_file = path/to/file.bz2
+fitacf_file = "path/to/file.bz2"
 with bz2.open(fitacf_file) as fp:
       fitacf_stream = fp.read()
 
 reader = pydarnio.SDarnRead(fitacf_stream, True)
 records = reader.read_fitacf()
 ```
 
-## Accessing data fields
+### Accessing data fields
 To see the names of the variables you've loaded in and now have access to, try using the `keys()` method:
 ```python
-print(fitacf_data[0].keys())
+print(records[0].keys())
 ```
 which will tell you all the variables in the first [0th] record.
 
 Let's say you loaded in a MAP file, and wanted to grab the cross polar-cap potentials for each record:
 ```python
 file = "20150302.n.map"
-SDarn_read = pydarnio.SDarnRead(file)
-map_data = SDarn_read.read_map()
+reader = pydarnio.SDarnRead(file)
+map_data = reader.read_map()
 
 cpcps=[i['pot.drop'] for i in map_data]
 ```
-## Other Examples
+### Other Examples
 
 Other examples of using pyDARNio with file reading is for reading in multiple 2-hour files, sorting them, and concatenating the data together.
 For example, you may do something like this, using the **glob** library:
@@ -73,6 +74,7 @@ import pydarnio
 from glob import glob
 
 fitacf_files = glob('path/to/fitacf/files/<date>*<radar>*.fitacf.bz2')
+data = []
 
 # assuming they are named via date and time
 fitacf_files.sort()
@@ -87,6 +89,27 @@ for fitacf_file in fitacf_files:
 print("Reading complete...")
 ```
 
-## DMapRead
+### DMapRead
 
 In pyDARNio, there also exists a class called `DMapRead`, which you can use in place of SDarnRead to read in any generic DMap structured file. However, pyDARNio won't test its integrity as it doesn't know what file it's supposed to be. If you're reading a SuperDARN file from one of the official data mirrors, then it is best to use SDarnRead in general.
+
+## Writing with SDarnWrite
+
+Very similarly to reading, DMap files can be written to file with a few simple commands.
+
+The basic code to write out a DMap structured file is as follows:
+```python
+import pydarnio
+
+data = ... # First you must get the records from a file, using SDarnRead()
+outfile = "path/to/file"
+writer = pydarnio.SDarnWrite(data, outfile)
+```
+which conducts some basic data integrity checks and stores in the object `writer`. 
+
+Now you need to tell it what kind of file it is. For instance, if the file you were reading in is a FITACF file, you would write something like:
+```python
+writer.write_fitacf()
+```
+If you were writing a different kind of file, you would need to use the methods `write_iqdat`, `write_rawacf`, `write_grid`, `write_map` or `write_snd` for their respective filetypes.
+These method calls will also check to ensure that the `data` you pass in is valid for the given file type.

docs/user/citing.md

@@ -59,14 +59,10 @@ pyDARNio uses other python libraries and the SuperDARN community's help.
 
 ### Python Library References 
 
-#### Matplotlib
-John D. Hunter. Matplotlib: A 2D Graphics Environment, Computing in Science & Engineering, 9, 90-95 (2007), [DOI:10.5281/zenodo.3264781](https://zenodo.org/record/3264781)
-
 #### Numpy 
 Stéfan van der Walt, S. Chris Colbert and Gaël Varoquaux. The NumPy Array: A Structure for Efficient Numerical Computation, Computing in Science & Engineering, 13, 22-30 (2011), [DOI:10.1109/MCSE.2011.37](https://ieeexplore.ieee.org/document/5725236)
 
 #### H5PY
-
 A. Collette, Python and HDF5, O'Reilly (2014) (http://shop.oreilly.com/product/0636920030249.do)
 
 

docs/user/install.md

@@ -4,15 +4,15 @@ author(s) Marina Schmidt-->
 ---
 
 [![License: LGPL v3](https://img.shields.io/badge/License-LGPLv3-blue.svg)](https://www.gnu.org/licenses/lgpl-3.0) 
-[![Python 3.6](https://img.shields.io/badge/python-3.6-blue.svg)](https://www.python.org/downloads/release/python-360/) 
+[![Python 3.7](https://img.shields.io/badge/python-3.6-blue.svg)](https://www.python.org/downloads/release/python-360/) 
 ![GitHub release (latest by date)](https://img.shields.io/github/v/release/superdarn/pyDARNio)
 [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.4009470.svg)](https://doi.org/10.5281/zenodo.4009470)
 
 
 
 ## Prerequisites
 
-**python 3.6+**
+**python 3.7+**
 
 | Ubuntu      | OpenSuse       | Fedora        | OSX           |
 | ----------- | -------------- | ------------- | ------------- |
@@ -34,7 +34,6 @@ pyDARNio's setup will download the following dependencies:
 - [Git](https://git-scm.com/) (For developers)
 - [pip3](https://help.dreamhost.com/hc/en-us/articles/115000699011-Using-pip3-to-install-Python3-modules)
 - [NumPy](https://numpy.org/)
-- [deepdish](https://deepdish.readthedocs.io/en/latest/api_io.html)
 - [pathlib2](https://docs.python.org/dev/library/pathlib.html)
 - [PyYAML](https://pyyaml.org/wiki/PyYAMLDocumentation)
 - [h5py](https://www.h5py.org/)
@@ -57,7 +56,7 @@ Instructions can be found here [virtual environment](https://packaging.python.or
 
 !!! Note
     If you have multiple versions of python 3 on your machine, you can access a specific version by: `python<version number>`. 
-    For example, if you want to install python 3.6 virtual environment: `python3.6 -m pip install --user virtualenv`.
+    For example, if you want to install python 3.7 virtual environment: `python3.7 -m pip install --user virtualenv`.
 
 ### Anaconda Virtual Environment
 Instructions can be found here [conda environment](https://uoa-eresearch.github.io/eresearch-cookbook/recipe/2014/11/20/conda/) and installing [anaconda](https://docs.anaconda.com/anaconda/install/)
@@ -96,7 +95,7 @@ To install:
 `$ pip3 install . --user`
 
 !!! Note
-    If `pip --version` is pointing to python 3.6+ then you can use `pip install . --user` instead. 
+    If `pip --version` is pointing to python 3.7+ then you can use `pip install . --user` instead. 
 
 !!! Warning
     Do not install pyDARNio with `python setup.py install` as this may install other libraries pyDARNio requires in a local directory causing issues down the road.

mkdocs.yml

@@ -14,9 +14,8 @@ nav:
     - Citing: user/citing.md
     - Tutorials:
         - IO:
-            - Read SuperDARN files: user/SDarnRead.md 
-            - Write SuperDARN files: user/SDarnWrite.md 
-            - Read/Write Borealis files: user/BorealisIO.md 
+            - SuperDARN (DMAP) files: user/SDarn.md
+            - Borealis files: user/BorealisIO.md
         - Convert: 
-            - Borealis to SuperDARN: user/Borealis2SuperDARN.md 
+            - Borealis to SuperDARN (DMAP): user/Borealis2SuperDARN.md
         - Radar and Hardware Information: user/hardware.md