Changes between Version 6 and Version 7 of fullforce_intro

Timestamp:

Dec 26, 2013, 1:26:47 PM (13 years ago)

Author:

bills

Comment:

--

fullforce_intro

@@ -167 +167 @@
 
 }}}
+
+
+== Processing ==
+
+{{{
+psphotStack output cmf is run through psastro to calibrate zero point
+and to produce the 'cff' file - the list of objects which will be
+measured by psphotFullForce
+
+The cff is produced by pmSourcesWrite_CFF which is called if 
+a output file "PSPHOT.OUTPUT.CFF" is defined.
+
+This file contains a single fits binary table with 1 row for each source.
+The values in the row depend on whether or not the source is selected
+for galaxy fitting. The boolean fitGalayx is set to be true if
+
+1. The extended model fits were performed on the source and one of
+   the models was set to be the source's modelEXT
+and
+2. The model params SXX, SYY, SXY, and model magnitude are finite
+
+Here are the values for each row in the cff file
+
+ID              source->seq (IPP_IDET)  (XXX: need to make sure that these are set to the
+                            same values as the psf extension. Insure that we 
+                            write cmf first.)
+X               fitGalaxy ? modelEXT->params[PM_PAR_XPOS] : modelPSF->params[PM_PAR_XPOS]
+Y               fitGalaxy ? modelEXT->params[PM_PAR_YPOS] : modelPSF->params[PM_PAR_YPOS]
+FLUX            (fitGalaxy ? 10**-0.4*modelEXT->mag : source->psfFlux) / CELL.EXPOSURE
+AP_RADIUS       source->apRadius
+KRON_RADIUS     source->moments->Mrf*2.5
+PETRO_RADIUS    source->extpars ? source->extpars->petrosianRadius : NAN
+FIT_GALAXY      see above
+PSF_STAR        source->mode & PM_SOURCE_MODE_PSFSTAR ? true : false
+R_MAJOR         fitGalaxy ? axes.major : 0
+R_MINOR         fitGalaxy ? axes.major : 0
+THETA           fitGalaxy ? axes.theta : 0
+MODEL_TYPE      fitGalaxy ? modelEXT->type : 0
+INDEX           fitGalaxy && modelEXT->type == sersic : 0
+
+Notes:
+The flux is converted to counts per second.
+
+It would be feasible to fit multiple models for each source but this would require that psphotFullForce and
+the function that reads cff files to be enhanced to deal with multiple models.
+
+
+Full Force Pipeline
+-------------------
+The full force pipeline consists of two stages. First psphotFullForce is run on all warp skycells for each
+stack. Then the program psphotFullForceSummary combines the results to get a single set of parameters for
+each source.
+
+psphotFullForce
+---------------
+INPUTS
+------
+The cff file defines the set of sources to be processed. 
+The data is read by a function in psModules and a source is created for each row.
+A psf model of the type given by the PSFMODEL keyword is created for each source. X and Y are used to set the
+parameters. A peak is allocated at the position with detValue, rawFlux, and smoothFlux set to the value of
+the FLUX column. The flux value (in counts per second) is later multipled by the warp exposure time by the
+function psphotLoadExtSources.
+
+moments are allocated with Mx = X, My = Y, Mrf = 0.4 *KRON_RADIUS
+tmpFlag PM_SOURCE_TMPF_MOMENTS_MEASURED is set. Note that this leaves the higher orde moments NAN
+
+if FIT_GALAXY is set and the model type is a recognized galaxy model type and extended source model is created.
+The model parameters are set based on X, Y, R_MAJOR, R_MINOR, and THETA. If the model type has an 8th parameter
+it is set to 0.5/SERISC_INDEX (BUG: this will result in INF if sersic index is zero)
+
+The warp image, mask, and variance images are read. If passed as an argument the warp stage psf file is read
+as well. Currently the script does not pass the psf file because Gene told me not to. I don't recall the
+reason.
+
+Processing
+----------
+Here are the important steps in full force processing.
+
+    psphotSetMaskAndVariance (config, view, filerule);
+
+    // generate a background model (median, smoothed image)
+    if (!psphotModelBackground (config, view, filerule)) {
+        return psphotReadoutCleanup (config, view, filerule);
+    }
+    if (!psphotSubtractBackground (config, view, filerule)) {
+        return psphotReadoutCleanup (config, view, filerule);
+    }
+    if (!strcasecmp (breakPt, "BACKMDL")) {
+        return psphotReadoutCleanup (config, view, filerule);
+    }
+
+    if (!psphotLoadPSF (config, view, filerule)) {
+        // this only happens if we had a programming error in psphotLoadPSF
+        psError (PSPHOT_ERR_UNKNOWN, false, "error loading psf model");
+        return psphotReadoutCleanup (config, view, filerule);
+    }
+
+NOTE: This is where the flux is multiplied by the exposure time to convert from counts per second to counts
+    // include externally-supplied sources (loaded onto detections->newSources)
+    psphotLoadExtSources (config, view, filerule);
+
+    // merge the newly selected sources into the existing list (detections->allSources)
+    // NOTE: merge OLD and NEW
+NOTE: (we only have NEW)
+    psphotMergeSources (config, view, filerule);
+
+    // construct sources and measure moments and other basic stats (saved on detections->allSources)
+    // all sources use the auto-scaled window appropriate to a PSF, except for the saturated
+    // stars : these use a larger window (3x the basic window)
+
+From comment in psphotFullForceSourceStats
+// construct pixels and measure moments for sources. 
+// NOTE: this function differs from psphotSourceStats in the following ways:
+// 1) detections->allSources are used instead of newSources
+// 2) the sources are used directly (peaks are not assigned to sources here)
+
+// NOTE: this function is meant to be used by psphotFullForce, and the mode of the loaded
+// sources should have (MODE_EXTERNAL) set.  If so, the Mx,My values will not be recalculated
+
+
+    if (!psphotFullForceSourceStats (config, view, filerule, true)) { // pass 1
+        psError(PSPHOT_ERR_UNKNOWN, false, "failure to generate sources");
+        return psphotReadoutCleanup (config, view, filerule);
+    }
+
+    // generate a psf model for any readouts which need one
+    // psphotFullForcePSF (config, view, filerule);
+    // XXX I'm not sure we need a different algorithm here or not : we are supplying the
+    // sources; we should mark with a flag bit the ones we actually want to use as PSF
+    // stars (this means we need to supply this info in the load).
+    psphotChoosePSF (config, view, filerule, false);
+
+    // Construct an initial model for each object, set the radius to fitRadius, set circular
+    // fit mask.  NOTE: only applied to sources without guess models
+    // XXX this currently only generates a PSF model
+    psphotGuessModels (config, view, filerule);
+
+    // linear PSF fit to source peaks, subtract the models from the image (in PSF mask)
+    // XXX this will fit non-PSF sources if they are supplied
+    psphotFitSourcesLinear (config, view, filerule, false, false);
+
+    // measure kron fluxes
+    psphotKronFlux (config, view, filerule);
+
+    // measure petro fluxes
+    psphotPetroFlux (config, view, filerule);
+
+    // measure radial apertures
+    psphotRadialApertures (config, view, filerule, 0);
+
+    // measure galaxy shapes
+    psphotGalaxyShape (config, view, filerule);
+
+    // calculate source magnitudes (psf mag and ap mag)
+    psphotMagnitudes(config, view, filerule);
+
+    // replace background in residual image
+    psphotSkyReplace (config, view, filerule);
+
+Galaxy Fitting
+--------------
+Most of the functions performed by psphotFullForce are the same as standard psphot. 
+
+psphotGalaxyShape is the function that creates extended models for the objects selected for
+galaxy fitting over a range of galaxy parameters and measures the chisq for each trial.
+
+The results of this are saved in a new cmf extension table refered to as 'XGAL'.
+
+Currently the only parameters varied are the major an minor radii. The range of values is
+determined by recipe. Currently 49 combinations are tried with the range in axes of
+0.85 to 1.15 times the input values in steps of 0.05 * the input values.
+
+In processing the source's modelEXT is left equal to the model for the nominal (input values).
+We do this so that the nominal values are available in psphotFullForceSummary but this yields
+a less useful residual image. 
+
+XXX: Perhaps psphotFullForceSummary should just read the CFF file (not as sources
+but as a table) and get the nominal values from there.
+
+
+OUTPUTS
+The output of the psphotFullForce consists of a standard cmf file and optionally a residual image. 
+
+psphotFullForceSummary
+----------------------
+Once psphotFullForce has been run on all of the warps for a given skycell filter combination
+the program psphotFullForceSummary is run to combine the results.
+
+Inputs
+------
+The cmfs from the full force runs on the individual warps.
+
+Computations
+------------
+
+psf measurements
+
+petrosian radius and magnitude
+    TBD mean radius?
+
+galaxy fits
+
+For each R_MAJOR, R_MINOR combination (i) the chisq of the combintation is measured by summing
+over the measurements on the indvidual warps (k). Note that sources which are masked in an input
+are not included in these sums. This is determined by requiring that PSF_QF_PERFECT > 0.9
+
+    chisq[i] = sum(chisq[k] * nPix[k]) / sum(nPix[k])
+    flux[i]  = sum( (flux[k] / dFlux[k])**2) / sum ( 1 / dFlux[k]**2)
+    dFlux[i] = sum ( 1 / dFlux[k]**2 )
+
+Once these values are computed the trial with the lowest chisq value is chosen as the "best fit"
+and the extended source model paramters are set
+
+Outputs
+-------
+These values are written to a summary cmf in the standard format
+
+
+}}}
+