Merge pull request #42 from ortk95/main

Improve documentation & add `--kw` command line flag as optional alternative for `--step_kwargs`

README.md

@@ -66,10 +66,11 @@ download_urls('https://naif.jpl.nasa.gov/pub/naif/generic_kernels/spk/satellites
 The [`miri_pipeline.py`](https://github.com/ortk95/jwst-pipelines/blob/main/miri_pipeline.py) script is used to reduce and process MIRI MRS observations of solar system targets. The full custom pipeline includes:
 1. The [standard JWST reduction pipeline](https://github.com/spacetelescope/jwst/) is used to reduce `stage0` uncalibrated data into `stage3` calibrated data cubes. Reduced cubes are produced for each individual dither, and with dithers combined. Background subtraction and residual fringe correction can also be applied to the data in this step.
 2. Each reduced cube is navigated, and backplanes are created to provide useful coordinates (e.g. latitude, longitude, emission angle, RA, Dec etc.) for each pixel.
-3. Saturated parts of the cubes are identified and desaturated where possible using data reduced using fewer groups.
-4. Flat field effects (e.g. striping and swirling patterns) in the data can be corrected using synthetic flat fields generated from dithered observations. To use these, see the [flat field](#miri-flat-fields) section below.
-5. Extreme outlier pixels are identified and flagged and removed from the data. Note that for small objects or point source observations, this `despike` step may be overaggressive and remove real data.
-6. Quick look [plots and animations](#examples) are generated for each cube.
+3. Darkening caused by the telescope's PSF is approximately corrected by convolving a forward model of the observation with a Gaussian PSF.
+4. Saturated parts of the cubes are identified and desaturated where possible using data reduced using fewer groups.
+5. Flat field effects (e.g. striping and swirling patterns) in the data can be corrected using synthetic flat fields generated from dithered observations. To use these, see the [flat field](#miri-flat-fields) section below.
+6. Extreme outlier pixels are identified and flagged and removed from the data. Note that for small objects or point source observations, this `despike` step may be overaggressive and remove real data.
+7. Quick look [plots and animations](#examples) are generated for each cube.
 
 This pipeline is described in Fletcher et al. (2023).
 

miri_pipeline.py

@@ -141,18 +141,22 @@
     export CRDS_PATH="/data/nemesis/jwst/crds_cache"
     export CRDS_SERVER_URL="https://jwst-crds.stsci.edu"
 
-    # Optionally redirect the verbose `reduction` step output to the file `pipeline.log`
+    # Optionally redirect the verbose stage1-3 logging to the file `pipeline.log`
     if [ -f "/data/nemesis/jwst/scripts/oliver/pipelines/stpipe-log.cfg" ]; then
         cp -f "/data/nemesis/jwst/scripts/oliver/pipelines/stpipe-log.cfg" .
         echo "Copied stpipe-log.cfg to current working directory"
     fi
 
     # Run the pipeline
     python3 /data/nemesis/jwst/scripts/oliver/pipelines/miri_pipeline.py /data/uranus/lon1 --parallel
-    
+    exit_code=$?
+
     # Change permissions on modified files so that other users can use them
     chmod -R --quiet ugo+rw /data/uranus/lon1
     chmod -R --quiet ugo+rw $CRDS_PATH
+
+    # Exit script with the same exit code as the pipeline
+    exit $exit_code 
 """
 STEP_DESCRIPTIONS = """
 - `remove_groups`: Remove groups from the data (for use in desaturating the data) [optional].
@@ -161,6 +165,7 @@
 - `defringe`: Run the JWST reduction pipeline residual fringe step [optional].
 - `stage3`: Run the standard JWST reduction pipeline stage 3.
 - `navigate`: Navigate reduced files.
+- `psf`: Correct PSF effects using a forward model convolved with telescope's PSF.
 - `desaturate`: Desaturate data using cubes with fewer groups [optional].
 - `flat`: Correct flat effects using synthetic flat field cubes.
 - `despike`: Clean cubes by removing extreme outlier pixels.

nirspec_pipeline.py

@@ -133,18 +133,22 @@
     export CRDS_PATH="/data/nemesis/jwst/crds_cache"
     export CRDS_SERVER_URL="https://jwst-crds.stsci.edu"
 
-    # Optionally redirect the verbose `reduction` step output to the file `pipeline.log`
+    # Optionally redirect the verbose stage1-3 logging to the file `pipeline.log`
     if [ -f "/data/nemesis/jwst/scripts/oliver/pipelines/stpipe-log.cfg" ]; then
         cp -f "/data/nemesis/jwst/scripts/oliver/pipelines/stpipe-log.cfg" .
         echo "Copied stpipe-log.cfg to current working directory"
     fi
 
     # Run the pipeline
     python3 /data/nemesis/jwst/scripts/oliver/pipelines/nirspec_pipeline.py /data/uranus/lon1 --parallel
-    
+    exit_code=$?
+
     # Change permissions on modified files so that other users can use them
     chmod -R --quiet ugo+rw /data/uranus/lon1
     chmod -R --quiet ugo+rw $CRDS_PATH
+
+    # Exit script with the same exit code as the pipeline
+    exit $exit_code 
 """
 STEP_DESCRIPTIONS = """
 - `remove_groups`: Remove groups from the data (for use in desaturating the data) [optional].

pipeline.py

@@ -1050,6 +1050,7 @@ def get_pipeline_argument_parser(
         '--step_kwargs',
         '--step-kwargs',
         '--kwargs',
+        '--kw',
         type=str,
         help="""JSON string containing keyword arguments to pass to individual pipeline
             steps. For example,