Pattern Noise Removal

NAME
       cleanir.py - filter pattern noise out of NIRI and GNIRS frames

SYNOPSIS
       cleanir.py [options] infile

DESCRIPTION
       This script assumes that the NIRI & GNIRS pattern noise in a quadrant
       can be represented by a fixed pattern which is repeated over the
       entire quadrant.  The default size for this pattern is 16 pixels
       wide and 4 pixels high (which may be changed via the -x and -y
       flags).  The pattern is determined for each quadrant by taking the
       mode of the pixel value distribution at each position in the
       pattern.  Once the pattern has been determined for a quadrant
       it is replicated to cover the entire quadrant and subtracted,
       and the mean of the pattern is added back to preserve flux.
       The standard deviation of all the pixels in the quadrant is
       compared to that before the pattern subtraction, and if no
       reduction was achieved the subtraction is undone.  The pattern
       subtraction may be forced via the -f flag.  This process is
       repeated for all four quadrants and the cleaned frame is written
       to c<infile> (or the file specified with the -o flag).  The
       pattern derived for each quadrant may be saved with the -p flag.
 
       Pattern noise is often accompanied by an offset in the bias
       values between the four quadrants.  One may want to use the
       -q flag to try to remove this offset.  This attempts to match
       the iteratively determined median value of each quadrant.
       This method works best with sky subtraction (i.e. with the -s
       flag), and does not work well if there are large extended objects
       in the frame.  By default the median is determined from the
       entire frame, although the -c flag will only use a central
       portion of the image.  Note that the derived quadrant offsets
       will be applied to the output pattern file.
 
       Removing the pattern from spectroscopy is more difficult because
       of many vertical sky lines.  By default f/6 spectroscopy with the
       2-pixel or blue slits (which do not fill the detector), uses the
       empty regions at the bottom (1-272) and top (720-1024) of the
       array for measuring the pattern.  This is not possible for other
       modes of spectroscopy where the spectrum fills the detector.
       For these modes it is best to do sky subtraction before pattern
       removal.  The quickest method is to pass a sky frame (or an offset
       frame) via the -s flag.  The manual method is to generate and
       subtract the sky, determine and save the pattern via the -p flag,
       then subtract the pattern from the original image.  One may use
       the -a flag to force using all of the pixels for the pattern
       determination.  If the SKYIMAGE FITS header keyword is present
       it is assumed that the sky has already been subtracted and all
       pixels will be used for the pattern determination.

       Note that you may use glob expansion in infile, however, the
       entire string must then be quoted or any pattern matching
       characters (*,?) must be escaped with a backslash.

OPTIONS
       -a : use all pixels for pattern determination
       -b <badpixelmask> : specify a bad pixel mask (overrides DQ plane)
       -c <frac> : use central  of image for bias adjustment [1]
       -d <dir> : specify an input data directory
       -f : force cleaning of all quads even if stddev does not decrease
       -g # : graph results (0=none, 1=pattern, 2=offsets, 3=both)
       -h  <val> : ignore values above  in pattern determination
       -m : use median for quadrant normalization (instead of fitting a Gaussian)
       --nodq : ignore the DQ plane
       -o <file> : write output to <file> (instead of c<infile>)
       -p <file> : write full-frame pattern to <file>
       -q : adjust quadrant offsets
       -r : row filtering (useful for GNIRS XD spectra)
       -s <sky> : sky frame to help in pattern recognition
       -v : verbose debugging output
       -x <size> : set pattern x size in pix [16]
       -y <size> : set pattern y size in pix [4]

VERSION
       2018-Feb-07

• astropy
• NumPy
• SciPy

Download

Installation Notes

1. Remove pattern noise from an image using the default parameters:
   

   > cleanir.py N20070806S0001.fits

   Output:
   cN20070806S0001.fits - cleaned FITS image
   
   
2. Remove pattern noise from an image, saving the output to different file names:
   

   > cleanir.py -o clean.fits -p pattern.fits N20070806S0001.fits

   Output:
   clean.fits - cleaned FITS image
   pattern.fits - pattern which was subtracted from the image
   
   
3. Remove pattern noise from all quadrants even if the standard deviation does not improve,
   and normalize quadrant offsets using the central 50% of the image:
   

   > cleanir.py -f -q -c 0.5 N20070806S0001.fits

4. Remove pattern noise using an offset frame as a sky to help with pattern determination:
   

   > cleanir.py -s N20070806S0002.fits N20070806S0001.fits

5. Remove pattern noise using a bad-pixel mask to mask out extended objects:
   

   > cleanir.py -b N20070806S0001msk.pl N20070806S0001.fits

6. Remove pattern noise from GNIRS XD spectra using row filtering and fixing quadrant offsets:
   

   > cleanir.py -rq N20070806S0001

• This is BETA software and individual results may vary (side effects invariably include headaches).
• This software will eventually be included in the Gemini PYRAF package.
• Any comments or suggestions for improvements are welcome.