HSTCAL 1.3.0rc1 Release Notes

HSTCAL Version 1.3.0rc1

The following updates have been included with this release.

ACS

• CALACS updates to version 9.2.0 with this release.

• New BLEVCORR algorithm that uses new OSCNTAB, which now correctly calculates bias level for AMP B and D in subarrays.

• New SINKCORR step to apply new SNKCFILE in order to flag sink pixels. See ACS ISR 2017-01.

• New pixel based PCTECORR algorithm implemented (CTE_NAME= 'PixelCTE 2017').

  • The code defaults to this new algorithm, however, the old one can still be executed using the command line option --ctegen <1|2>. Note: The correct accompanying PCTETAB file must also be used.
  • The user can now override the PCTETAB file used (normally specified in the image header) with the command line option --pctetab <file path>.
  • The number of threads used for computing the PCTECORR can now be specified with the command line option --nthreads <N>. Note: OPENMP is required for N > 1.

• Fixed uninitialized variable in prinfo.c.

• Corrected --version reporting and added -r to report version string that was previously reported by --version.

• Affects old data.

• Users should re-retrieve affected datasets from the archive.

• Updated reference files delivered by science team.

Notes

• Due to the quick succession of this release from 1.2.0, please refer to the following release notes for other recent updates.

HSTCAL 1.2.0 Release Notes

HSTCAL Version 1.2.0

The following updates, specified in separate sections for each instrument, have been included with this release.

HSTCAL package updates

• waf installer upgraded to 1.9.9 to support Python 3.6.
• Added --O3 and --release-with-symbols options in build configuration.
• Enhanced platform check by installer.
• HSTCAL error codes were consolidated into a single global include file.
• Bug fixes for PtrRegister struct and its associated functions.

ACS

• Fixed uninitialized variable in prinfo.c.
• Corrected --version reporting and added -r to report version string that was previously reported by --version.

STIS

• calstis version 3.4.1

• fixed a bug that garbled some FITS HISTORY keywords

• fixed a bug in the bar finding algorithm

• fixed a bug in processing highres echelle data in which the error and dq arrays were incorrectly calculated

  No keyword changes.

  The only science changes will be to longslit spectroscopic data and high resolution echelle data.

• Affects old data

• Users should re-retrieve affected datasets from the archive

• updated reference files delivered by science team

WFC3

• calwf3 updates to version 3.4.1 with this release

Photometry Keyword Updates

• There are default description strings in calwf3 which are used when the keyword is being added to a header where it doesn't already exist. This update makes the text consistent among all UVIS headers, including the global header keywords that calwf3 adds for user convenience that are not populated in the raw headers during conversion. See issue #67 for more details. The table below details the changes to the photometry header keywords:

Detector	Extension	Keyword	Previous	Updated
UVIS	0,1,4	PHOTCORR	populate photometric header keywords	NO CHANGE
UVIS	0,1,4	FLUXCORR	convert to absolute flux units	Scale UVIS2 to match UVIS1 using PHTRATIO
UVIS	0	PHOTFLAM	inverse sensitivity	Inverse sensitivity, ergs/cm2/A/e-
UVIS	1,4	PHOTFLAM	Inverse sensitivity, ergs/cm2/A/e-	NO CHANGE
UVIS	0	PHOTZPT	zero point	ST magnitude zero point
UVIS	1, 4	PHOTZPT	zero point	ST magnitude zero point
UVIS	0	PHTFLAM1	photometry scaling for chip 1	Ch1 Inv Sens, use PHOTFLAM for UV filters
UVIS	1,4	PHTFLAM1	PHOTFLAM on chip1 when scales are not uniform	Ch1 Inv Sens, use PHOTFLAM for UV filters
UVIS	0	PHTFLAM2	photometry scaling for chip 2	Ch2 Inv Sens, use PHOTFLAM if FLUXCORR=COMPLETE
UVIS	1,4	PHTFLAM2	PHOTFLAM on chip2 when scales are not uniform	Ch2 Inv Sens, use PHOTFLAM if FLUXCORR=COMPLETE
UVIS	0	PHTRATIO	photometry scaling for chip2 to chip 1	PHTFLAM2/PHTFLAM1 ratio
UVIS	1,4	PHTRATIO	Ratio of PHTFLAM1/PHTFLAM2	PHTFLAM2/PHTFLAM1 ratio
UVIS	1,4	PHOTMODE	observation con	Observing configuration
UVIS	1,4	PHOTFNU	inverse sensitivity, Jy*Sec/electron	Inverse sensitivity, Jy*sec/e-
UVIS	1,4	PHOTPLAM	pivot wavelength	NO CHANGE
UVIS	1,4	PHOTBW	RMS bandwidth	NO CHANGE

• Affects old data, all UVIS data will be affected since the keywords are commonly populated

• Users should re-retrieve affected datasets from the archive for updated header keywords

• Affects full-frame and subarray UVIS datasets for all filters

• Code cleanup specific to issue #55 and issue #63; these do not change any data results

Updated reference files which will be part of the reprocessing

This software update itself does not affect data values, just header keyword descriptions. However, the team will be delivering updated reference files at the same time which will be used during reprocessing with calwf3 version 3.4.1. The following reference files will be delivered and used by the archive for reprocessing after this software change has been installed.

Keyword	Description	Applied Date Range for the new file (USEAFTER)	selection keywords	Content update or format change?
IMPHTTAB	photometry table used by calwf3	All UVIS data	DETECTOR = UVIS, PHOTCORR = PERFORM	Update
BIASFILE	bias reference file	All UVIS data	DETECTOR = UVIS, BIASCORR = PERFORM	Update
FLSHFILE	post-flash reference file	July 2012 - present	DETECTOR = UVIS	Update

Reprocessing will begin after the delivery of the reference files and those will affect header keyword values and data for the following:

• IMPHTTAB: only affects UVIS data which has PHOTCORR = PERFORM, this excludes calibration files like darks and biases
• BIASFILE: only affects UVIS data which is corrected for bias, including subarrays, except for charge injected data, data which has been read out using a single amplifier, or data from modes which are no longer supported. The science team also plans to begin delivering one bias per year.
• FLSHFILE: applies to all data taken with post-flash

HSTCAL 1.2.0rc3 Release Notes

HSTCAL Version 1.2.0rc3

The following updates have been included with this release.

HSTCAL package updates

• Fix bugs in PtrRegister struct and its associated functions.

The following issues were raised with the previous release candidate that this new candidate addresses:

#131
#133
#135
#143

NOTE: These changes are of INACTIVE code and therefore have zero runtime impact for the code as is, i.e. this effectively makes 1.2.0rc2 and 1.2.0rc3 the same.

c.f. HSTCAL 1.2.0rc2 Release Notes

HSTCAL 1.2.0rc2 Release Notes

HSTCAL Version 1.2.0rc2

The following updates, specified in separate sections for each instrument, have been included with this release.

HSTCAL package updates

• waf installer upgraded to 1.9.9 to support Python 3.6.
• Added --O3 and --release-with-symbols options in build configuration.
• Enhanced platform check by installer.
• HSTCAL error codes were consolidated into a single global include file.

ACS

• Fixed uninitialized variable in prinfo.c.
• Corrected --version reporting and added -r to report version string that was previously reported by --version.

STIS

• calstis version 3.4.1

• fixed a bug that garbled some FITS HISTORY keywords

• fixed a bug in the bar finding algorithm

• fixed a bug in processing highres echelle data in which the error and dq arrays were incorrectly calculated

  No keyword changes.

  The only science changes will be to longslit spectroscopic data and high resolution echelle data.

• Affects old data

• Users should re-retrieve affected datasets from the archive

• updated reference files delivered by science team

WFC3

• calwf3 updates to version 3.4.1 with this release

Photometry Keyword Updates

• There are default description strings in calwf3 which are used when the keyword is being added to a header where it doesn't already exist. This update makes the text consistent among all UVIS headers, including the global header keywords that calwf3 adds for user convenience that are not populated in the raw headers during conversion. See issue #67 for more details. The table below details the changes to the photometry header keywords:

Detector	Extension	Keyword	Previous	Updated
UVIS	0,1,4	PHOTCORR	populate photometric header keywords	NO CHANGE
UVIS	0,1,4	FLUXCORR	convert to absolute flux units	Scale UVIS2 to match UVIS1 using PHTRATIO
UVIS	0	PHOTFLAM	inverse sensitivity	Inverse sensitivity, ergs/cm2/A/e-
UVIS	1,4	PHOTFLAM	Inverse sensitivity, ergs/cm2/A/e-	NO CHANGE
UVIS	0	PHOTZPT	zero point	ST magnitude zero point
UVIS	1, 4	PHOTZPT	zero point	ST magnitude zero point
UVIS	0	PHTFLAM1	photometry scaling for chip 1	Ch1 Inv Sens, use PHOTFLAM for UV filters
UVIS	1,4	PHTFLAM1	PHOTFLAM on chip1 when scales are not uniform	Ch1 Inv Sens, use PHOTFLAM for UV filters
UVIS	0	PHTFLAM2	photometry scaling for chip 2	Ch2 Inv Sens, use PHOTFLAM if FLUXCORR=COMPLETE
UVIS	1,4	PHTFLAM2	PHOTFLAM on chip2 when scales are not uniform	Ch2 Inv Sens, use PHOTFLAM if FLUXCORR=COMPLETE
UVIS	0	PHTRATIO	photometry scaling for chip2 to chip 1	PHTFLAM2/PHTFLAM1 ratio
UVIS	1,4	PHTRATIO	Ratio of PHTFLAM1/PHTFLAM2	PHTFLAM2/PHTFLAM1 ratio
UVIS	1,4	PHOTMODE	observation con	Observing configuration
UVIS	1,4	PHOTFNU	inverse sensitivity, Jy*Sec/electron	Inverse sensitivity, Jy*sec/e-
UVIS	1,4	PHOTPLAM	pivot wavelength	NO CHANGE
UVIS	1,4	PHOTBW	RMS bandwidth	NO CHANGE

• Affects old data, all UVIS data will be affected since the keywords are commonly populated

• Users should re-retrieve affected datasets from the archive for updated header keywords

• Affects full-frame and subarray UVIS datasets for all filters

• Code cleanup specific to issue #55 and issue #63; these do not change any data results

Updated reference files which will be part of the reprocessing

This software update itself does not affect data values, just header keyword descriptions. However, the team will be delivering updated reference files at the same time which will be used during reprocessing with calwf3 version 3.4.1. The following reference files will be delivered and used by the archive for reprocessing after this software change has been installed.

Keyword	Description	Applied Date Range for the new file (USEAFTER)	selection keywords	Content update or format change?
IMPHTTAB	photometry table used by calwf3	All UVIS data	DETECTOR = UVIS, PHOTCORR = PERFORM	Update
BIASFILE	bias reference file	All UVIS data	DETECTOR = UVIS, BIASCORR = PERFORM	Update
FLSHFILE	post-flash reference file	July 2012 - present	DETECTOR = UVIS	Update

Reprocessing will begin after the delivery of the reference files and those will affect header keyword values and data for the following:

• IMPHTTAB: only affects UVIS data which has PHOTCORR = PERFORM, this excludes calibration files like darks and biases
• BIASFILE: only affects UVIS data which is corrected for bias, including subarrays, except for charge injected data, data which has been read out using a single amplifier, or data from modes which are no longer supported. The science team also plans to begin delivering one bias per year.
• FLSHFILE: applies to all data taken with post-flash

HSTCAL 1.1.1 Release Notes

Note: This is the same as 1.1.1rc1 pre-release.

The following changes are quick-fix to the HST calibration software for HST Data Processing for build 2016.2:

ACS (increases to version 8.3.5)

• Fixed a bug where BLEVCORR cannot remove virtual overscan of polarizer and ramp subarrays that are now using the new subarray readout format.

HSTCAL 1.1.1RC1 Release Notes

The following changes are quick-fix to the HST calibration software for HST Data Processing for build 2016.2:

ACS (increases to version 8.3.5)

• Fixed a bug where BLEVCORR cannot remove virtual overscan of polarizer and ramp subarrays that are now using the new subarray readout format.

HSTCAL 1.1.0 Release Notes

The following changes were made to the HST calibration software sent to HST Data Processing for build 2016.2:

General

• waf upgraded to 1.9.4, which enables building HSTCAL in Python 3, as well as Python 2.

WFC3 (increases to version 3.4)

• CTE subarrays are now enabled for subarrays which include physical overscan pixels.
• Sink Pixel flagging for all subarray images has been enabled.

HSTCAL v1.0.1 Release Notes

This release is to accommodate CALACS processing for new ACS subarrays added by FSW change done in May 2016. Particularly, BLEVCORR is modified to trim virtual overscans off the new subarrays (old subarrays do not have virtual overscans). No changes are made for CALWF3 and CALSTIS.

HSTCAL v1.0.0 Release Notes

The following changes were made since the last release in April 2014, broken down by sub-components:

General

• Fixed compilation warnings.

WFC3 (v3.3)

• A new processing step, FLUXCORR, was added to the UVIS pipeline, and is performed at the end of processing. It will scale the chip2 image using the new PHTFLAM1 and PHTFLAM2 values in the IMPHTAB. New flat fields for all filters, as well as a new IMPHTTAB have been delivered by the team for this step to be completely implemented.
• The CTE correction has been implemented for all full-frame UVIS data. This is done in conjunction with a full run through of the pipeline code without the CTE correction applied, such that both CTE corrected and non-CTE corrected output files are saved. This correction is for the same reasons as in ACS, but the CTE correction algorithm is different; it is applied to the raw file instead of later in the processing. Some sections of the CTE code support parallel processing with OpenMP. The default for calwf3 is to use all available processors. To restrict processing to 1 cpu use the flag -1 in the call to calwf3.e The CTE processing is controlled with the PCTECORR keyword. New CTE corrected output products will be produced at all stages which involved changes to most of the controlling routines and output trailers. See the team documentation for more complete information on the updates.
• In conjunction with the CTE correction, a standalone interface wf3cte was created to perform just the CTE correction, similar to wf32d etc.
• Sink pixel detection is now performed in the UVIS pipeline for full-frame images, using the SNKCFILE reference image, and the science data DQ mask is updated with the detections. The reference image has 2 extensions, each in the pre-overscan trimmed format. This step is performed if DQICORR is PERFORM, and is done before BLEVCORR while the science image is still untrimmed.
• Some of the new reference files required new code to read them, including the new format for the UVIS IMPHTTAB associated with the FLUXCORR step
• The default CRCORR behavior for IR SCAN data will now be set to OMIT by default so that the resulting calibrated image is last read - first read instead of the fit to the ramp.
• All IR scan related keywords formerly in the SPT file will also be present in the FLT file
• For UVIS and IR, a copy of the CSMID keyword, formerly in the SPT will also be in the FLT file, CSMID lists the channel select mechanism ID.
• bug fix: nrej initialized in wf3rej so that REJ_RATE reported consistently correct for the IR pipeline
• bug fix: a wfc3 uvis association which specifies multiple products wont finish processing and segfaults
• An assortment of memory leaks were fixed
• Explicit error added to report a non-WFC3 image used as input to the pipeline
• updated text in wf3rej to report that Astrodrizzle should be used to align images instead of PyDrizzle since that’s how it’s advertised to users
• fixed SEGFAULT error in reference file checking when iref environment variable not set by user, so can’t find file

ACS

• Added support for 2K subarrays in PCTECORR.
• acs2d.e reads calibration flags from image header instead of command line.
• Improved temporary file handling.
• Improved error message if input image does not belong to ACS.
• Added support for very long input list for ACSREJ.
• Fixed memory leaks (non-critical).