pmSubtractBias definition wrong

[Paul Price]

I got the definition for pmSubtractBias wrong on a couple of points:

• The overscans are to be derived by the function using CELL.BIASSEC (a linked

list of psRegions, accessed from the parent of the readout), not from the "bias"
parameter passed to the function (which is supposed to be the full-frame bias
image).

• The full-frame (bias and dark) subtractions should only be performed on the

region of the image specified by CELL.TRIMSEC (we don't want to subtract the
bias' overscan from the input's overscan, etc). This is simply done by:

psRegion *trimRegion = psMetadataLookupPtr(NULL, in->parent->concepts,

"CELL.TRIMSEC");

psImage *trimmed = psImageSubset(in->image, *trimRegion);

Actions on "trimmed" will then manifest in the readout.

I've corrected the SDRS, but wanted to let you know of these somewhat subtle fixes.

[Paul Price]

Actually, on playing with the phase 2 script, it would be best for all of the
operations listed as "detrend" in the Modules SDRS to only act on the region of
the readout specified by the CELL.TRIMSEC.... How about we attempt to get this
value and if unable, give a warning and then act on the entire image --- this
will give flexibility.

Functions affected:
pmSubtractBias
pmNonLinearityPolynomial
pmNonLinearityLookup
pmFlatField
pmMaskBadPixels
pmSubtractSky (deferred anyway)

[gusciora@…]

I'm not sure how to use CELL.BIASSEC. What should I do with that linked list of
psRegions? Currently, I take the overscan regions from in->bias, reduce then to
a column or vector, then subtract from the entire image. How should I use
CELL.BIASSEC.

Also, I have implemented the CELL.TRIMSEC region for the full-frame
subtractions. I assumed that this applied to the overscan subtractions. Is
that not correct?

[Paul Price]

CELL.BIASSEC will give you a linked list of psRegions. You can get subimages by
using these regions on the image with psImageSubset. Then that should be close
to where you were before.

[gusciora@…]

How do I get subImages? Do I apply the psRegions to the input image, or the
bias images?

[Paul Price]

psMetadataItem *biassecsItem = psMetadataLookup(readout->parent->concepts,

"CELL.BIASSEC");

Check that biassecsItem is not NULL, and that the type is a psList.
...;

Iterate through the list, and get subimages
psList *biassecs = biassecsItem->data.V; The list of biassecs
psListIterator *biassecsIter = psListIteratorAlloc(biassecs, PS_LIST_HEAD,

false); Iterator for list

psRegion *biassec = NULL; The region from the list
psList *biasImages = psListAlloc(NULL); The list of images
while (biassec = psListGetAndIncrement(biassecsIter)) {

psImage *bias = psImageSubset(readout->image, *biassec);
psListAdd(biasImages, PS_LIST_TAIL, bias);

}
psFree(biassecsIter);
Now you have a list of bias images

[gusciora@…]

I'm not sure if I understand this correctly: we use the CELL.BIASSEC metadata
item from the metadata of the psReadout for the main image: this points to a
linked list of psRegions which are then used on the "bias" psReadout parameter.
However, we later are required subtract that entire bias psReadout from the
image data.

It seems odd that at one point we'll be subtracting parts of the bias image from
the data image, and later, we'll subtract the entire bias image from the data
image. Is my interpretation incorrect?

[Paul Price]

I think the core operation of this function is sufficient for the moment. The
defininition is still a little unclear, since I'm working on the program that
will call it.

Let's leave it for now, and I may alter it in a branch.

[Paul Price]

I've reworked the API to something that I think will be better. Sorry to
flip-flop, but I've removed the need for you to get CELL.TRIMSEC and
CELL.BIASSEC. This makes it more like it was before, but I stuffed the overscan
stuff into a separate structure. Hopefully, not too many changes for you
besides cosmetic. I'll also be removing the need to get CELL.TRIMSEC for the
other functions.

The bias subtraction module provides a facility to correct detector
images for the electronic pedestal introduced by the readout
electronics.

Given an input image and various other parameters,
\code{pmSubtractBias} shall subtract the bias from the image:

\begin{prototype}
pmReadout *pmSubtractBias(pmReadout *in, pmOverscanOptions *overscanOpts,

psRegion imageRegion, psList *overscanRegions,
const pmReadout *bias, const pmReadout *dark);

\end{prototype}

Three types of bias correction may optionally be performed on the
input image, \code{in}. The first is the subtraction of an overscan.
Multiple overscan regions may be specified and fit as a function of
row (or column). The second is the subtraction of a full-frame bias
image. The third is the subtraction of a suitably scaled full-frame
dark image.

The input image, \code{in}, shall have the bias subtracted in-place.
The input image may be of type U16, S32, or F32. The region of the
input image that shall have the overscan or full-frame subtractions
applied is specified by \code{imageRegion}.

Overscan subtraction is performed if \code{overscanOpts} is
non-\code{NULL} (see \S\ref{sec:overscan}). \code{overscanRegions}
shall be a list of \code{psRegion}s that specify the regions that
comprise the overscans.

A \code{bias} frame shall be subtracted pixel-by-pixel from the input
image if \code{bias} is non-NULL. If \code{dark} is non-\code{NULL},
then the dark image, scaled by the ratio of dark times (from
\code{CELL.DARKTIME}) shall be subtracted pixel-by-pixel from the
input image. The full-frame subtractions (both bias and dark) should
only be performed on the image region specified by
\code{CELL.TRIMSEC}. Note that the input image, \code{in}, and the
\code{bias} and \code{dark} frames need not be the same size, but the
function shall use the offsets in the image (\code{in->x0} and
\code{in->y0}) to determine the appropriate offsets to obtain the
correct pixel on the \code{bias}. In the event that the \code{bias}
image is too small (i.e., pixels on the input image refer to pixels
outside the range of the \code{bias} image), the function shall
generate an error. Any pixels masked in the \code{bias} or
\code{dark} shall also be masked in the output. The bias and dark
images may be copied to the same type as the input image if required.

\subsubsection{Overscan subtraction}
\label{sec:overscan}

The options for performing the overscan subtraction are bundled in a
\code{pmOverscanOptions}:

\begin{datatype}
typedef struct {

Inputs
bool single; Reduce all overscan regions to a single value?
bool scanRows; Scan direction was rows? (otherwise columns)
pmFit fitType; Type of fit to overscan
unsigned int order; Order of polynomial, or number of spline pieces
psStats *stat; Statistic to use when reducing the minor

direction

Outputs
psPolynomial1D *poly; Result of polynomial fit
psSpline1D *spline; Result of spline fit

} pmOverscanOptions;
\end{datatype}

The mode in which the overscan is subtracted is specified by the
\code{single} boolean. If \code{single} is \code{true}, then the
entire overscan region is reduced to a single value using the
\code{stat}. If \code{single} is \code{false}, the overscan shall be
reduced along the dimension specified by \code{scanRows} (rows if
\code{scanRows} is true; otherwise columns).

If the overscan is not defined for each row/column,
\code{pmSubtractBias} shall generate an error if \code{fitType} is
\code{PM_FIT_NONE}; otherwise, the function shall shall generate a
warning and the undefined values shall be interpolated using the
provided functional form.

The statistic to use in combining multiple pixels in the
prescan/overscan regions is specified by \code{stat}. \code{stat} is
of type \code{psStats} instead of simply \code{psStatsOptions} so that
clipping levels may be specified, if desired. In the event that
multiple options are specified by \code{stats}, a warning shall be
generated, and the option with the highest priority shall be used,
according to the following priority order: \code{PS_STAT_SAMPLE_MEAN},
\code{PS_STAT_SAMPLE_MEDIAN}, \code{PS_STAT_CLIPPED_MEAN},
\code{PS_STAT_ROBUST_MEAN}, \code{PS_STAT_ROBUST_MEDIAN},
\code{PS_STAT_ROBUST_MODE}.

\code{fitType} is an enumerated type which specifies the type of fit
to employed on the overscan vector:
\begin{datatype}
typedef enum {

PM_FIT_NONE, /< No fit
PM_FIT_POLY_ORD, /< Fit ordinary polynomial
PM_FIT_POLY_CHEBY, /< Fit Chebyshev polynomial
PM_FIT_SPLINE /< Fit cubic splines

} pmFit;
\end{datatype}

If \code{fitType} is \code{PM_FIT_NONE}, then the overscan vector is
subtracted from the image without fitting. Otherwise, the overscan
vector is fit using the specified functional form, the fit is
subtracted from the image, and the \code{poly} or \code{spline} is
allocated and updated with the results of the fit.

The allocator for a \code{pmOverscanOptions} shall be:
\begin{prototype}
pmOverscanOptions *pmOverscanOptionsAlloc(bool single, bool scanRows,

pmFit fitType, unsigned int order,
psStats *stat);

\end{prototype}

[Paul Price]

Actually, I need to think some more about whether we should need to look up
CELL.TRIMSEC and CELL.BIASSEC.... Sorry for the flip-flop. Hang off on that for
now.

The overscan stuff should be good, though.

[eugene]

time has caught up with this bug: the code is doing the correct thing in terms of what is subtracted, the 'concepts' system is being used to handle the trimsec / biassec info. the documentation may need to be synchronized, but this is a global issue for psModules.