Changeset 5367

Timestamp:

Oct 18, 2005, 10:15:05 AM (21 years ago)

Author:

eugene

Message:

added flat-field norm, update names on pmAstrom functions

Location:

trunk/doc/modules

Files:

• CameraImages.tex (modified) (3 diffs)
• ChangeLogSDRS.tex (modified) (2 diffs)
• ModulesSDRS.tex (modified) (4 diffs)

trunk/doc/modules/CameraImages.tex

@@ -863 +863 @@
 entry must contain the following user-defined parameters:
 \begin{itemize}
-\item GRID_OFFSET : maximum allowed displacement in search
-\item GRID_SCALE  : grid bin size in focal-plane coordinate units
-\item MIN_ANGLE : minimum tested relative rotation 
-\item MAX_ANGLE : maximum tested relative rotation 
-\item DEL_ANGLE : relative rotation step size
+\item \code{GRID_OFFSET} : maximum allowed displacement in search
+\item \code{GRID_SCALE}  : grid bin size in focal-plane coordinate units
+\item \code{MIN_ANGLE} : minimum tested relative rotation 
+\item \code{MAX_ANGLE} : maximum tested relative rotation 
+\item \code{DEL_ANGLE} : relative rotation step size
 \end{itemize}
 The output from this process is the statistics of the best-fit
@@ -930 +930 @@
 following function performs this operation:
 \begin{prototype}
-bool pmAstromMatchedListFit (pmFPA *fpa, psArray *st1, psArray *st2, psArray *match, psMetadata *config);
+bool pmAstromFitFPA (pmFPA *fpa, psArray *st1, psArray *st2, psArray *match, psMetadata *config);
 \end{prototype}
 This function accepts the raw and reference source lists and the list
@@ -949 +949 @@
 linear terms and the higher-order terms of the polynomial fits.
 
+An alternative to fitting the rotation of the FPA relative to the
+Tangent Plane is to treat the fitted transformation as a measurement
+of the chip within the FPA.  The following function performs this
+operation in the same way as \code{pmAstromFitFPA}:
+\begin{prototype}
+bool pmAstromFitChip (pmFPA *fpa, psArray *st1, psArray *st2, psArray *match, psMetadata *config);
+\end{prototype}
+This function accepts the raw and reference source lists for a single
+chip and the list of matched entries.  It uses the matched list to
+determine a polynomial transformation between the two coordinate
+systems.  The fitting uses clipping to exclude outliers, likely
+representing poor matches.  The \code{config} element must contain the
+information \code{ASTROM_NSIGMA} (specifying the number of sigma used
+in the clipping) and \code{ASTROM_NCLIP} (specifying the number of
+clipping iterations must be performed).  The \code{config} element
+must also specify the order of the polynomial fit (keyword:
+\code{ASTROM_ORDER}).  The result of this fit is a set of
+modifications of the components of the \code{pmChip.toFPA}
+transformation.
+
 A mosaic represents a particular set of challenges when determining an
 astrometric solution.  There is substantial degeneracy between the

trunk/doc/modules/ChangeLogSDRS.tex

@@ -1 @@
-%%% $Id: ChangeLogSDRS.tex,v 1.33 2005-09-20 22:01:33 jhoblitt Exp $
+%%% $Id: ChangeLogSDRS.tex,v 1.34 2005-10-18 20:15:05 eugene Exp $
 
 \subsection{Changes from version 00 (16 August 2004) to version 01 (12 October 2004)}
@@ -142 +142 @@
 \begin{itemize}
   \item fix enum syntax
+  \item added Astrometry Fitting Support (matching / fitting routines)
+  \item added \code{pmAstromRadiusMatch}
+  \item added \code{pmAstromGridMatch}
+  \item added \code{pmAstromApplyGridMatch}
+  \item added \code{pmAstromMeasureGradients}
+  \item added \code{pmAstromFitFPA}
+  \item added \code{pmAstromFitChip}
+  \item added \code{pmAstromFitDistortion}
+  \item renamed section 7 to Detrend Creation, moved \code{pmReadoutCombine} to a subsection
+  \item added \code{pmFringeStats} (Fringe Amplitude subsection)
+  \item added \code{pmFlatNormalization} (Flat-field renorm section)
 \end{itemize}

trunk/doc/modules/ModulesSDRS.tex

@@ -1 @@
-%%% $Id: ModulesSDRS.tex,v 1.61 2005-10-18 14:32:25 eugene Exp $
+%%% $Id: ModulesSDRS.tex,v 1.62 2005-10-18 20:15:05 eugene Exp $
 \documentclass[panstarrs,spec]{panstarrs}
 
@@ -1205 +1205 @@
 \end{itemize}
 
-\subsection{Fringes}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+\section{Detrend Creation}
+
+In the detrend creation process, a collection of raw images are
+combined to produce a clean, high-quality master image for correcting
+the effect of interest.  The input images may potentially be processed
+and scaled in some way.  The resulting output images may be to be
+re-scaled to have a consistent signal for all chips in the mosaic.
+The simplest example is the construction of a bias image, in the case
+where there is signficant 2-D bias structure.  In this case, the input
+raw bias images are probably combined without any additional
+processing.  In another example, flat-field image must be
+bias-corrected and scaled to a consistent normalization before being
+combined, and the flat-field images from the different chips must be
+normalized so that each chip will be flattened consistently across the
+mosaic.  A complex example is the fringe pattern, in which the input
+images must be bias-corrected and flattened, and the resulting images
+must be scaled by the amplitude of the fringe pattern on each image,
+rather than by the average flux level.  In this section, we define the
+tools necessary to perform the detrend creation process.
+
+\subsection{Image Stacking}
+
+A basic operation in generating the master detrend images is using a
+stack of many input images of a particular type and combining them,
+with perhaps some additional scaling, in order to build up
+signal-to-noise and to reject deviant pixel.  For this, we require a
+general purpose image combination module.  We forsee this module as
+only acting upon data from the same detector, and so each input image
+will have the same noise characteristics.
+
+\begin{datatype}
+typedef struct {
+    psStats *stats;                     // Statistics to use in combining pixels
+    unsigned int maskVal,               // Mask pixels where mask & maskVal == 1
+    float fracHigh;                     // Fraction of high pixels to throw
+    float fracLow;                      // Fraction of low pixels to throw
+    int nKeep;                          // Number of pixels to be sure to keep
+} pmCombineParams;
+\end{datatype}
+
+\begin{prototype}
+psImage *
+pmReadoutCombine(psImage *output,       // Output image, or NULL
+                 const psList *inputs,  // List of input readouts
+                 pmCombineParams *params, // Combination parameters
+                 const psVector *zero,  // Offsets to apply for each image
+                 const psVector *scale, // Scales to apply for each image
+                 bool applyZeroScale,   // Are zero and scale for application, or only noise properties?
+                 float gain,            // Gain in e/ADU
+                 float readnoise        // Read noise in e
+                 );
+\end{prototype}
+
+\code{pmReadoutCombine} combines input images pixel by pixel --- for
+each pixel of the output image, a stack of contributing input pixels
+is formed and combined.  Several of its input parameters are lists or
+vectors, and if these are not all of the same length (or \code{NULL}),
+the module shall generate an error and return \code{NULL}.
+
+If the provided \code{output} is \code{NULL}, then the module shall
+allocate a new image of sufficient size for the input images.  If the
+\code{output} image is non-\code{NULL} and is not of sufficient size
+for the combined image, the module shall generate an error and return
+\code{NULL}.
+
+If the \code{inputs} is \code{NULL}, the module shall generate an
+error and return \code{NULL}.  Otherwise, the \code{inputs} shall be a
+list of \code{pmReadout}s.  The images contained within the
+\code{pmReadout}s need not all be of the same size, but the module
+shall take into account the offsets (\code{col0,row0}) from the corner
+of the detector when comparing pixels, so that it is the same
+\textit{physical} pixels that are combined.
+
+The parameters used in the combination, including how the pixels are
+to be combined, and how the rejection is performed is contained within
+the \code{params}, which may not be \code{NULL} (otherwise the module
+shall generate an error and return \code{NULL}).  We choose to use
+this structure instead of supplying the values separately in order to
+keep down the number of parameters to \code{pmReadoutCombine}; the
+\code{pmCombineParams} may be recycled for subsequent calls to
+\code{pmReadoutCombine} since the values are not dependent upon the
+choice of inputs, but merely specify how the combination is to be
+performed.
+
+The particular statistic specified by \code{stats} shall be used to
+combine each stack of pixels from the input images.  Only one of the
+statistics choices may be specified, otherwise the module shall
+generate an error and return \code{NULL}.
+
+If the \code{maskVal} is non-zero, then pixels in the \code{mask} of
+each \code{pmReadout} in the \code{inputs} which satisfy the
+\code{maskVal} shall not have the corresponding pixels placed in the
+stack for combination.
+
+After masking, but before performing the combination, the highest
+\code{fracHigh} fraction and lowest \code{fracLow} fraction of pixels
+in the stack are immediately rejected, unless this would leave less
+than \code{nKeep} pixels in the stack, in which case no immediate
+rejection is performed.
+
+If the \code{zero} vector is non-\code{NULL} and \code{applyZeroScale}
+is \code{true}, then the appropriate values shall be added to the
+\code{inputs} before rejection is performed.  If \code{zero} is
+non-\code{NULL} and \code{applyZeroScale} is false, then the values
+shall only be used in calculating the Poisson variances.
+
+If the \code{scale} vector is non-\code{NULL} and
+\code{applyZeroScale} is \code{true}, then the appropriate values
+shall multiply the \code{inputs} before rejection is performed.  If
+\code{scale} is non-\code{NULL} and \code{applyZeroScale} is false,
+then the values shall only be used in calculating the Poisson
+variances.
+
+The purpose of \code{applyZeroScale} is to allow combination of fringe
+frames, where the frames have been deliberately sky-subtracted and
+rescaled (to get the fringes amplitudes running from -1 to 1), which
+actions should not be undone when combining, but yet it is desirable
+to provide the \code{zero} and \code{scale} values so that the correct
+noise properties are used in the combination.
+
+If the \code{gain} and \code{readnoise} are positive and non-negative
+(respectively), then these shall be used to provide weights for the
+combination using Poisson statistics ($\sigma_i$ below).
+
+In summary, pixels corresponding to the same physical pixel are
+combined, having values $x_i \pm \sigma_i$.  In the case that
+\code{applyZeroScale} is \code{true}, then:
+\begin{eqnarray}
+x_i & = & s_i f_i + z_i \\
+\sigma_i & = & [g x_i + r^2]^{1/2} / g
+\end{eqnarray}
+Where $f_i$ is the value of the pixel in image $i$, $s_i$ is the scale
+applied to image $i$, $z_i$ is the zero offset applied to image $i$,
+$g$ is the gain, and $r$ is the read noise.  If scales are not
+provided, they are set to unity; if zero offsets are not provided,
+they are set to zero.
+
+If \code{applyZeroScale} is \code{false}, then the values are:
+\begin{eqnarray}
+x_i & = & f_i \\
+\sigma_i & = & [g (s_i f_i + z_i) + r^2]^{1/2} / g
+\end{eqnarray}
+where the same symbols are used as above.
+
+The \code{inputs, zero} and \code{scale} may be of U16, S32 and F32
+types, and must all be of the same type.  The \code{output} shall be
+of the same type.
+
+\subsection{Fringe Amplitude}
 
 Some images contain a signal caused by thin-film interference in the
@@ -1225 +1375 @@
 
 A simple approach to the fringe pattern is to subtract a master fringe
-frame scaled by the amplitude of the fringe pattern.  We thus need a
-method of measuring the fringe amplitude which is robust in the
-presence of stars and which is fast.  We implement a method used at
-CFHT in which the fringe pattern is mapped by a series of points pairs
-which correspond to peaks and valleys of the fringe pattern.  We
-define the following function to measure the global fringe amplitude
-of an image given a collection of fringe point pairs.
+frame scaled by the amplitude of the fringe pattern.  The amplitude of
+the fringe pattern is used both in the process of constructing the
+master image and in scaling the master image when it is applied to
+science image.  We thus need a method of measuring the fringe
+amplitude which is robust in the presence of stars and which is fast.
+We implement a method used at CFHT in which the fringe pattern is
+mapped by a series of points pairs which correspond to peaks and
+valleys of the fringe pattern.  We define the following function to
+measure the global fringe amplitude of an image given a collection of
+fringe point pairs.
 \begin{prototype}
 stats *pmFringeStats (psArray *fringePoints, psImage *image, psMetadata *config);
@@ -1256 +1409 @@
 \end{datatype}
 
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-
-\section{Calibration}
-
-The calibration module essentially consists of combining multiple
-images of a particular type in order to build up signal-to-noise.  For
-this, we require a general purpose image combination module.  We
-forsee this module as only acting upon data from the same detector,
-and so each input image will have the same noise characteristics.
-
-\begin{datatype}
-typedef struct {
-    psStats *stats;                     // Statistics to use in combining pixels
-    unsigned int maskVal,               // Mask pixels where mask & maskVal == 1
-    float fracHigh;                     // Fraction of high pixels to throw
-    float fracLow;                      // Fraction of low pixels to throw
-    int nKeep;                          // Number of pixels to be sure to keep
-} pmCombineParams;
-\end{datatype}
-
-\begin{prototype}
-psImage *
-pmReadoutCombine(psImage *output,       // Output image, or NULL
-                 const psList *inputs,  // List of input readouts
-                 pmCombineParams *params, // Combination parameters
-                 const psVector *zero,  // Offsets to apply for each image
-                 const psVector *scale, // Scales to apply for each image
-                 bool applyZeroScale,   // Are zero and scale for application, or only noise properties?
-                 float gain,            // Gain in e/ADU
-                 float readnoise        // Read noise in e
-                 );
-\end{prototype}
-
-\code{pmReadoutCombine} combines input images pixel by pixel --- for
-each pixel of the output image, a stack of contributing input pixels
-is formed and combined.  Several of its input parameters are lists or
-vectors, and if these are not all of the same length (or \code{NULL}),
-the module shall generate an error and return \code{NULL}.
-
-If the provided \code{output} is \code{NULL}, then the module shall
-allocate a new image of sufficient size for the input images.  If the
-\code{output} image is non-\code{NULL} and is not of sufficient size
-for the combined image, the module shall generate an error and return
-\code{NULL}.
-
-If the \code{inputs} is \code{NULL}, the module shall generate an
-error and return \code{NULL}.  Otherwise, the \code{inputs} shall be a
-list of \code{pmReadout}s.  The images contained within the
-\code{pmReadout}s need not all be of the same size, but the module
-shall take into account the offsets (\code{col0,row0}) from the corner
-of the detector when comparing pixels, so that it is the same
-\textit{physical} pixels that are combined.
-
-The parameters used in the combination, including how the pixels are
-to be combined, and how the rejection is performed is contained within
-the \code{params}, which may not be \code{NULL} (otherwise the module
-shall generate an error and return \code{NULL}).  We choose to use
-this structure instead of supplying the values separately in order to
-keep down the number of parameters to \code{pmReadoutCombine}; the
-\code{pmCombineParams} may be recycled for subsequent calls to
-\code{pmReadoutCombine} since the values are not dependent upon the
-choice of inputs, but merely specify how the combination is to be
-performed.
-
-The particular statistic specified by \code{stats} shall be used to
-combine each stack of pixels from the input images.  Only one of the
-statistics choices may be specified, otherwise the module shall
-generate an error and return \code{NULL}.
-
-If the \code{maskVal} is non-zero, then pixels in the \code{mask} of
-each \code{pmReadout} in the \code{inputs} which satisfy the
-\code{maskVal} shall not have the corresponding pixels placed in the
-stack for combination.
-
-After masking, but before performing the combination, the highest
-\code{fracHigh} fraction and lowest \code{fracLow} fraction of pixels
-in the stack are immediately rejected, unless this would leave less
-than \code{nKeep} pixels in the stack, in which case no immediate
-rejection is performed.
-
-If the \code{zero} vector is non-\code{NULL} and \code{applyZeroScale}
-is \code{true}, then the appropriate values shall be added to the
-\code{inputs} before rejection is performed.  If \code{zero} is
-non-\code{NULL} and \code{applyZeroScale} is false, then the values
-shall only be used in calculating the Poisson variances.
-
-If the \code{scale} vector is non-\code{NULL} and
-\code{applyZeroScale} is \code{true}, then the appropriate values
-shall multiply the \code{inputs} before rejection is performed.  If
-\code{scale} is non-\code{NULL} and \code{applyZeroScale} is false,
-then the values shall only be used in calculating the Poisson
-variances.
-
-The purpose of \code{applyZeroScale} is to allow combination of fringe
-frames, where the frames have been deliberately sky-subtracted and
-rescaled (to get the fringes amplitudes running from -1 to 1), which
-actions should not be undone when combining, but yet it is desirable
-to provide the \code{zero} and \code{scale} values so that the correct
-noise properties are used in the combination.
-
-If the \code{gain} and \code{readnoise} are positive and non-negative
-(respectively), then these shall be used to provide weights for the
-combination using Poisson statistics ($\sigma_i$ below).
-
-In summary, pixels corresponding to the same physical pixel are
-combined, having values $x_i \pm \sigma_i$.  In the case that
-\code{applyZeroScale} is \code{true}, then:
-\begin{eqnarray}
-x_i & = & s_i f_i + z_i \\
-\sigma_i & = & [g x_i + r^2]^{1/2} / g
-\end{eqnarray}
-Where $f_i$ is the value of the pixel in image $i$, $s_i$ is the scale
-applied to image $i$, $z_i$ is the zero offset applied to image $i$,
-$g$ is the gain, and $r$ is the read noise.  If scales are not
-provided, they are set to unity; if zero offsets are not provided,
-they are set to zero.
-
-If \code{applyZeroScale} is \code{false}, then the values are:
-\begin{eqnarray}
-x_i & = & f_i \\
-\sigma_i & = & [g (s_i f_i + z_i) + r^2]^{1/2} / g
-\end{eqnarray}
-where the same symbols are used as above.
-
-The \code{inputs, zero} and \code{scale} may be of U16, S32 and F32
-types, and must all be of the same type.  The \code{output} shall be
-of the same type.
+\subsection{Flat-field Re-Normalization}
+
+Consider a collection of $N_i$ flat-field images obtained with a
+mosaic camera consisting of $N_j$ chips.  Each image is exposed to an
+illumination source which should be a uniform surface
+brightness\footnote{This is likely a false assumption: the
+illumination source likely has spatial variations.  However, for the
+purposes of this discussion, it only matters that such spatial
+variations scale consistently as a function of illumination intensity.
+The spatial errors are corrected by the photometric flat-field
+correction technique (eg., Magnier \& Cuillandre 2004).}  Two factors
+determine the actual measured flux level (in Digital Numbers) on each
+of the chips in each image: the gain of each chip ($\mbox{gain}_j$)
+and the flux level from the illumination source ($\mbox{source}_i$).
+When the images are combined, the input images must be scaled so that
+the flux levels can be consistently compared.  After combining the
+collection of images, it is necessary to determine an appropriate
+re-normalization for the resulting flat-field images.  In effect, the
+individual chips must be adjusted so that the master flat-field image
+has a flux level which varies from chip to chip in proportion to the
+actual chip gain.  In this case, if a uniform illumination source
+illuminates the mosaic, the resulting flux levels will be corrected by
+the flat-field to a single, consistent flux level. 
+
+In order to determine the correct relative scaling between the
+devices, it is thus necessary to know the individual chip gains, or at
+least the gain ratios.  A typical technique scaled all chips relative
+to a reference chip, or by a statistic measured for the complete
+collection.  These techniques fail if the input collection of images
+does not always consist of the same set of chips; for the GPC on
+Pan-STARRS, we must expect that individual cells or even chips may be
+disabled on a frequent basis, so our algorithms must not be limited by
+the assumption that all chips are available in all images.  We
+therefore define the following algorithm to measure the relative chip
+gains for a collection of input flat-field images, each with a
+measured flux $\mbox{flux}_{i,j}$.  We want to solve for the chip
+gains and the source illumination fluxes which would make the best
+prediction of the measured input image fluxes:
+\[
+\mbox{flux}^{\rm pred}_{i,j} = \mbox{gain}_j \times \mbox{source}_i
+\]
+This relationship is easiest to determine if we take the logarithm of
+both sides of the equation:
+\[
+M^{\rm pred}_{i,j} = G_j + S_i
+\]
+where $M^{\rm pred}_{i,j} = \log \mbox{flux}^{\rm pred}_{i,j}$, $G_j =
+\log \mbox{gain}_j$, and $\log \mbox{source}_i = S_i$.  We can then
+write the chi-square which we want to minimize as:
+\[
+\chi^2 = \sum_{i,j} (M^{\rm obs}_{i,j} - G_j - S_i)^2
+\]
+where we ignore the weights of the different measured flux levels.
+Taking the derivatives with respect to the parameters of interest
+($G_j, S_i$), and setting them to 0, we determine the following set of
+equations which must be solved:
+\[
+G_j \times N_i = \sum_i M^{\rm obs}_{i,j} - \sum_i S_i
+\]
+\[
+S_j \times N_j = \sum_j M^{\rm obs}_{i,j} - \sum_j G_j \\
+\]
+This set of equations can be solved iteratively, starting from the
+assumption that all chip gains are 1.0, ($G_j = 0$), or by supplying
+a guess for the chip gains.  The result of this analysis is the
+measured chip gains and the measured source illumination levels for
+each of the input flat-field images.  The chip gains can then be used
+to modify the flux levels on the master flat-field images.
+
+We define the following function to perform the analysis discussed
+above:
+\begin{prototype}
+bool pmFlatNormalization (psVector *sourceFlux, psVector *chipGains, psArray *fluxLevels);
+\end{prototype}
+The input array \code{fluxLevels} consists of $N_i$ vectors, one per
+mosaic image.  Each vector consists of $N_j$ elements, each a
+measurement of the input flat-field image flux levels.  All of these
+vectors must be constructed with the same number of elements, or the
+function will return an error.  If a chip is missing from a particular
+image, that element should be set to \code{NaN}.  The vector
+\code{chipGains} supplies initial guesses for the chip gains.  If the
+vector contains the values 0.0 or \code{NaN} for any of the elements,
+the gain is set to the mean of the valid values.  If the vector length
+does not match the number of chips, an warning is raised, all chip
+gain guesses will be set to 1.0, and the vector length modified to
+match the number of chips defined by the supplied \code{fluxLevels}.
+The \code{sourceFlux} input vector must be allocated (not
+\code{NULL}), but the routine will set the vector length to the number
+of source images regardless of the initial state of the vector.  All
+vectors used by this function must be of type \code{PS_DATA_F64}.
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 
 \section{Objects on Images}