Changes between Version 3 and Version 4 of PS1_Science_Processing_Status_20090705

Timestamp:

Jul 7, 2009, 12:19:05 PM (17 years ago)

Author:

eugene

Comment:

--

PS1_Science_Processing_Status_20090705

@@ -19 +19 @@
 variance (does not exist for the raw data, of course):
 
-|| Processing Stage    || Image Data per Exposure     || Download Options  ||
-|| Raw Data            || ~1.5GB                      || Images or DB Tables Only ||
-|| Chip (Detrended)    || ~3.6GB (images + variance)  || Images or DB Tables Only ||
-|| Camera (Calibrated) || ~0.0GB (masks only)         || Detection files, logs ||
-|| Warp                || ~6.0GB (images + variance)  || ||
-|| Diff                || ~5.6GB (images + variance)  || || 
-|| Stack               || ~6.0GB per field per filter || ||
+|| Processing Stage    || Image Data per Exposure     || Data Products ||
+|| Raw Data            || ~1.5GB                      || Raw images + rawExp, rawImfile DB table entries ||
+|| Chip (Detrended)    || ~3.6GB (images + variance)  || Detrend images, mask, variance, log, DB table entries ||
+|| Camera (Calibrated) || ~0.0GB (masks only)         || Calibrated detection files, logs, DB table entries ||
+|| Warp                || ~6.0GB (images + variance)  || Warped images, mask, variance, log, DB table entries ||
+|| Diff                || ~5.6GB (images + variance)  || Difference images, mask, variance, detection tables, log, DB table entries || 
+|| Stack               || ~6.0GB per field per filter || Stack images, mask, variance, detection tables, log, DB table entries ||
 
 Except for the images and the variance images, the other data products (including the masks) are a small fraction of the image data value (< 5%). 
@@ -31 +31 @@
 The full June data set is about 420*(1.8 + 3.6 + 6 + 5.6) = 7TB. April + May will be another 5TB.
 
-Institutions may register for subsets of the data, and may request
-(for example) to receive the detection files from the detrend chip
-stage, but no pixels, and all detections and pixels for the difference
-stage.
+Institutions may register for subsets of the data, and may request (for example) to receive the detection files from the detrend chip stage, but no pixels, and all detections and pixels for the difference stage.  At the moment, choices may be controlled independently for each MD field (and filter?). 
 
+A few other notes related to the data products for different stages:
+
+* the detection tables generated at the chip stage (with names of the form o4984g0109o.72894.ch.13035.XY33.cmf) are intermediate, uncalibrated data products.  '''Are we distributing these still?''' The calibrated detections are rolled up into a single file (with many FITS extensions) with names of the form o4984g0109o.72894.cm.13035.smf.  
+
+* the detection tables from the warp stage are generated by transforming the detections from the chip stage, not by performing a full re-detection stage.  The PSF is remeasured to determine the impact of warping on the image quality.
+
+* the detection tables generated in the stack creation stage are (a) only going to limited depth and (b) only performing PSF analysis.  These tables do not represent the full analysis of the stack images, but are used at this stage to check the output PSF model.
 
 == Summary of Data ==