2015B ORAC-DR Release Notes

These are the 2015B release notes containing changes since the 2015A Starlink release.

General

• Some contact email addresses have been updated.
• Allow object names with apostrophes or equals signs.
• Debug mode will now print the command lines used into the log.
• Line graphics leave gaps for missing values and hence are clearer.

ACSIS

• Legacy processing of large spatial areas or many input timeseries cubes now supports chunking to avoid addressing too many voxels.
• No longer create "cube" files in the REDUCE_SCIENCE_LEGACY recipe.
• Improvement made to spike filtering limits.
• Better handling of hybrid-spectra combination when the overlap is small.
• Tile borders removed from frame cubes.
• Improved histogram-peak finding algorithm to cope with more unusual distributions.
• Fixed bugs including calculation of median Tsys for large and sparse datasets, and flatfield determination for curtailed observations.

LCOGT

• Synchronised to 2015-10-22 LCOGT's subversion repository.

UKIRT

• Extends the search for sources in chopped observations but restricted to prevent picking up the wrong beam.
• Copes with chopped data where the original beam information is lost.
• New Michelle bad-pixel mask provided from 20150301.
• Switch suffix of output file of _STANDARD_BLACKBODY_ from _dbs to _std.
• Handle missing END header in FITS airlock when extracting sources.

JSA primitives

• _COPY_JSA_HEADERS_TO_FITS_FILE_ now includes the WAVELEN, BANDWID and OBJECT headers.
• _DETERMINE_CLASSES_ copies "locked" uhdr values to temporary objects.

2015A release of ORAC-DR

2015A Release Notes

These are the 2015A release notes containing changes since the 2014A Starlink release.

General

• The documentation adheres to new Starlink documentation system, generating PDFs instead of PostScript.
• File extensions of JCMT project recpars files have been corrected from .lis to .ini.
• ORAC-DR registered with Astrophysics Source Code Library as ascl:1310.001.
• A description of the ORAC-DR system was published in Astronomy and Computing journal: Jenness & Economou, 2015, A&C, 9, 40, doi:10.1016/j.ascom.2014.10.005.

ACSIS

• Added headings to the log commentary to demarcate clearly the various stages of processing.
• New recipe parameters:
  • FRACTION_BAD specifies the maximum fraction of bad values per receptor and subband. Validate subbands independently when looking for bad receptors.
  • BASELINE_REGIONS defines a list of regions in the spectrum to be used for baseline subtraction. It may also be used to define the regions for testing baseline non-linearity if the BASELINE_LINEARITY_LINEWIDTH recipe parameter is also set to base.
  • HIGHFREQ_INTERFERNCE_MIN_SPECTRA sets the minimum number of good spectra for high-frequency interference filtering to be attempted.
• Recipes create new log files.
  • log.noisestats contains the noise statistics collated by a new primitive WRITE_NOISESTATS_LOG and calulated by FIND_MEAN_RMS and CALCULATE_MEDIAN_TSYS_.
  • log.flatstats contains the flatfield ratios and their respective errors for each receptor, created by new primitive WRITE_FLATFIELD_LOG. To support this, add an APPLY argument to REMOVE_FLATFIELD_ITERATE so that flats may be computed and logged but not applied.
• Support for added for JSA legacy (HEALPix tiling) processing.
• Various improvements to the flatfield determination, the most important being observations on the same night may be combined to give an improved flatfield.
• Primitive FIND_TRUE_PEAK now works for stares, grids, and jiggles. REDUCE_STANDARD formerly might select the wrong spectrum.
• Restructure high-frequency artefact filtering to be more robust by combining subscans, and it is faster too. Defaults to test for ringing only in H07 receptor.
• Improvements made when testing for bad receptors, including efficient concatenation of subscans.
• On the command line it is now possible supply a combination of a list and another option for bad_receptors. For instance "-calib bad_receptors=H01:H06:index" to exclude H01, H06, and any others found in index.bad_receptors_qa.
• New recipes:
  • REDUCE_SCIENCE_LEGACY for the JSA legacy processing in HEALPix tiles.
• Bugs fixed:
  • The PIXSIZE passed to UNMAKECUBE had incorrect scaling and for some not widely used spreading methods could cause the step to run for a very long time
  • Removed second call to CALCULATE_STANDARD measured over a different region giving rise to confusing duplication in log.standard.
  • Itersum method in flat field could diverge for some receptors because of the chosen collapse estimator was mean rather than sum.
  • Use HISTAT to derive median for tiled cubes >400 Mpixel.
  • Further protection against pathological data applied.

SCUBA-2

• Expanded logging: can include airmass and transmission... Map and calibration statstics logging added to several recipes.
• New recipes:
  • REDUCE_SCAN_ISOLATED_SOURCE uses the "bright compact" dimmconfig file and as such is only suitable for an isolated bright source located at the tracking position.
• New primitives:
  • CALC_ERROR_BEAM -- performs aperture photometry to measure the fraction of the total flux that lies outside a FWHM-sized aperture.
  • WRITE_CALSTATS_LOGFILE
  • WRITE_MAPSTATS_LOGFILE -- calculates and writes out a series of values derived from the reduced image, including the RMS noise, NEFD and exposure time.
• New recipe parameters:
  • NOCHUNKING: if true, requests the pipeline abort if chunking would occur. This is used to prevent chunking in JSA processing.
• Source fitting overhauled. It now returns the FWHM error and the number of components of the fit. Noisy outliers exclude to prevent spuriously high S/N ratios. Two-component fits are only allowed when some criteria, such as S/N>100, are met.
• The default dimmconfig used when running REDUCE_SCAN was changed to be the "jsa generic" dimmconfig.
• REDUCE_SCAN writes statistics to a log file.
• The positions of pointing sources are now fudged when creating JSA tiles.
• REDUCE_POINTING is now a symbolic link to REDUCE_SCAN (and the old recipe is retained as OBSOLETE_REDUCE_POINTING).
• SUN/264 extensively updated and expanded to match the current behaviour and recipes.
• Configurations for several JCMT PI and survey projects added.

PICARD

• New recipes:
  • SCUBA2_MAPSTATS generates the "mapstats" and "calstats" (for known calibrators) log files. If the input is a JSA tile, then it attempts to reproject to tangent-plane projection first.
  • JSA_CATALOGUE creates JCMT Science Archive style catalogues.
• Fix CREATE_MOMENTS_MAP for heterodyne data, adding two new recipe parameters to control the number of frequency channels and spatial pixels used for smoothing. Also added access to recipe paramters to set velocity limits and the baseline-fit order.
• SUN/265 updated to match the current recipes.

UKIRT

• WFCAM
  • A new recipe MOVING_JITTER has been added as a variant of JITTER for fast-moving targets, using a new primitive SHIFT_TO_BASE_POSITIONS.
• IFU
  • Now makes tagged group representative image.
  • Final cubes now have sky co-ordinates.
  • Make intermediate mosaics far more efficiently.
• Michelle
  • Recongnise .InBEAMA/B NDFs in the container file as raw data and access their FITS metadata.

LCOGT

• Major changes to add support for many more cameras, telescope and camera types including LCOSPECTRAL, LCOSINISTRO, LCOMEROPE and LCOFLI for Spectral Instruments, LCOGT Sinistro, Merope and LCOGT FLI Autoguider cameras respectively.
• Change LCOCC (Context``Camera) processing to use the new CALCULATE_ZEROPOINT_NULL primitive to skip too-costly zeropoint determination.
• Add a new QUICK_LOOK mode and recipe and modify several primitives to produce output files with different suffixes in QUICK_LOOK mode.
• Add support for setting data from certain proposal types to be immediately public or never public.
• Add new primitives CHECK_PROPID & CHECK_TAGID to verify proposal and TAG ids against the Science Archive.
• Switch to storing and looking for calibrations in a master calibration library. Add new primitive FLAT_QC to perform checks on evening and morning flatfields and only keep them if both good. Add new remove() method to ORAC::Index.
• Add non-valid keyword entries for SATUR_KEY & GAIN_KEY to the SExtractor config files to prevent new (V2.19.5) Extractor reading a bad (0) value for the saturation from the headers and treating every pixel as saturated.
• Add SUN/269 document describing LCOGT pipeline.

General primitives

• CREATE_GRAPHIC_FROM_FILE
  • is more efficient as it can now generate multiple resolution images in a single invocation;
  • adds axes to large representative spectra;
  • the colours have changed to a clearer black on a pale grey for spectra, and images use the cubehelix0 colour table with the same grey for bad values;
  • can scale between standard deviation limits;
  • can draw clump outlines and catalogue positions, or "No emission detected" text on the image;
  • additional metadata (such as instrument, backend, whether or not data are hybrid) are written to the graphics file

JSA primitives

• CREATE_PASTED_TILE_GRAPHIC
  • Now uses SMURF jsapaster command to combine tiles rather than KAPPA:PASTE.
• New primitives:
  • COPY_JSA_HEADERS_TO_FITS_FILE -- for copying JSA FITS headers to files lacking them.
  • CREATE_JSA_CATALOGS -- performs the actual JSA catalog creation (for the JSA_CATALOGUE recipe). Makes use of the following (also new) primitives:
    • CREATE_JSA_EXTENT_CAT
    • CREATE_JSA_PEAKS_CAT
    • CREATE_JSA_TILE_MOC
  • DETERMINE_CLASSES -- primitive extracted from previous COADD_JSA_TILES functionality.
  • REPROJECT_JSA_TILE -- attempts to reproject a JSA tile to tangent-plane projection using SMURF:JSAJOIN