Pattern Noise Removal
NAME
cleanir.py - filter pattern noise out of NIRI and GNIRS frames
SYNOPSIS
cleanir.py [options] infile
DESCRIPTION
This script assumes that the NIRI & GNIRS pattern noise in a quadrant
can be represented by a fixed pattern which is repeated over the
entire quadrant. The default size for this pattern is 16 pixels
wide and 4 pixels high (which may be changed via the -x and -y
flags). The pattern is determined for each quadrant by taking the
mode of the pixel value distribution at each position in the
pattern. Once the pattern has been determined for a quadrant
it is replicated to cover the entire quadrant and subtracted,
and the mean of the pattern is added back to preserve flux.
The standard deviation of all the pixels in the quadrant is
compared to that before the pattern subtraction, and if no
reduction was achieved the subtraction is undone. The pattern
subtraction may be forced via the -f flag. This process is
repeated for all four quadrants and the cleaned frame is written
to c<infile> (or the file specified with the -o flag). The
pattern derived for each quadrant may be saved with the -p flag.
Pattern noise is often accompanied by an offset in the bias
values between the four quadrants. One may want to use the
-q flag to try to remove this offset. This attempts to match
the iteratively determined median value of each quadrant.
This method works best with sky subtraction (i.e. with the -s
flag), and does not work well if there are large extended objects
in the frame. By default the median is determined from the
entire frame, although the -c flag will only use a central
portion of the image. Note that the derived quadrant offsets
will be applied to the output pattern file.
Removing the pattern from spectroscopy is more difficult because
of many vertical sky lines. By default f/6 spectroscopy with the
2-pixel or blue slits (which do not fill the detector), uses the
empty regions at the bottom (1-272) and top (720-1024) of the
array for measuring the pattern. This is not possible for other
modes of spectroscopy where the spectrum fills the detector.
For these modes it is best to do sky subtraction before pattern
removal. The quickest method is to pass a sky frame (or an offset
frame) via the -s flag. The manual method is to generate and
subtract the sky, determine and save the pattern via the -p flag,
then subtract the pattern from the original image. One may use
the -a flag to force using all of the pixels for the pattern
determination. If the SKYIMAGE FITS header keyword is present
it is assumed that the sky has already been subtracted and all
pixels will be used for the pattern determination.
Note that you may use glob expansion in infile, however, the
entire string must then be quoted or any pattern matching
characters (*,?) must be escaped with a backslash.
OPTIONS
-a : use all pixels for pattern determination
-b <badpixelmask> : specify a bad pixel mask (overrides DQ plane)
-c <frac> : use central of image for bias adjustment [1]
-d <dir> : specify an input data directory
-f : force cleaning of all quads even if stddev does not decrease
-g # : graph results (0=none, 1=pattern, 2=offsets, 3=both)
-h <val> : ignore values above in pattern determination
-m : use median for quadrant normalization (instead of fitting a Gaussian)
--nodq : ignore the DQ plane
-o <file> : write output to <file> (instead of c<infile>)
-p <file> : write full-frame pattern to <file>
-q : adjust quadrant offsets
-r : row filtering (useful for GNIRS XD spectra)
-s <sky> : sky frame to help in pattern recognition
-v : verbose debugging output
-x <size> : set pattern x size in pix [16]
-y <size> : set pattern y size in pix [4]
VERSION
2018-Feb-07
Requirements:
Examples:
- Remove pattern noise from an image using the default parameters:
> cleanir.py N20070806S0001.fits
Output:
cN20070806S0001.fits - cleaned FITS image
- Remove pattern noise from an image, saving the output to different file names:
> cleanir.py -o clean.fits -p pattern.fits N20070806S0001.fits
Output:
clean.fits - cleaned FITS image
pattern.fits - pattern which was subtracted from the image
- Remove pattern noise from all quadrants even if the standard deviation does not improve,
and normalize quadrant offsets using the central 50% of the image:
> cleanir.py -f -q -c 0.5 N20070806S0001.fits
- Remove pattern noise using an offset frame as a sky to help with pattern determination:
> cleanir.py -s N20070806S0002.fits N20070806S0001.fits
- Remove pattern noise using a bad-pixel mask to mask out extended objects:
> cleanir.py -b N20070806S0001msk.pl N20070806S0001.fits
- Remove pattern noise from GNIRS XD spectra using row filtering and fixing quadrant offsets:
> cleanir.py -rq N20070806S0001
Disclaimer:
- This is BETA software and individual results may vary (side effects invariably include headaches).
- This software will eventually be included in the Gemini PYRAF package.
- Any comments or suggestions for improvements are welcome.
2017-Mar-11 - Andrew Stephens <astephens at gemini dot edu>