Changes between Version 1 and Version 2 of MD.PV1

Timestamp:

Apr 19, 2013, 2:30:41 PM (13 years ago)

Author:

Mark Huber

Comment:

--

MD.PV1

@@ -5 +5 @@
  * [wiki:MD.GR0] summary page -- summary of all reprocessing details and 2011/2012 refstacks and some deepstack samples
 
-Defined PV1 for MD based on the detrend ([wiki:GPC1_Detrend_Documentation]) and faint-end bias photometry (details somewhere..). Many of the details pulled from the [wiki:MD.GR0] page.
+Processing and (re)reprocessing is defined PV1 for MD based on the date of 2012-05-16 to include improved and new detrend types ([wiki:GPC1_Detrend_Documentation]) and corrections for the faint-end bias photometry (ipp-20120404,r33642 tag -- details related to psphot elsewhere..). While detrends have been updated since that date, the changes are minor and should not impact the MD processing. While the latest date for the new VideoMask detrend is 2012-05-03, the date of 2012-05-16 was chosen as the day after the summary of the detrend updates had been established. Many of the details here are also taken from the [wiki:MD.GR0] page, see that page for more information. In general, all the MD fields required at least partial re-reprocessing of chip to warp except MD09 for which MD.GR0 reprocessing started at the beginning of June 2012 (including the refstack and night stacks). All MD fields have refstacks created after the 2012-05-16 date. Only MD09 has a complete set of PV1 night stacks, all other fields will have a mixture of pre and post-PV1 night stacks (post-PV1 will generally be the normal nightly science processing of night stacks and not from MD.GR0).
 
 
 
 = Reprocessed Data Products = 
-Each MD field has a variety of ''issues'' and details are summarized in each MD section. Label/data_group names have also evolved and the appropriate MD section should be referred to.
+Each MD field has a variety of ''issues'' and details are summarized in each MD section of [wiki:MD.GR0] or later in the relevant refstack wiki page [wiki:MediumDeepFields]. Label/data_group names have also slowly simplified and the appropriate MD section should be referred to if those must be be used. It is recommended to use the <stage>_id to select appropriate data products and each section below will provide the number after which the PV1 definition should hold for.
+
+
+
+It is important to make clear that the x.PV1.x label/data_group/workdir for MD was a sub-re-reprocessing of exposures that were reprocessed or just processed before date X (now set at 5/16/2012). There have been various other re-reprocessing and reprocessing of exposures (for refstacks, GR0 late 2012 etc ***) that have happened after that date. It is really the <stage>_id that needs to be used to simply select with, and so there are duplicates of exposures with valid <stage>_id. 
+
+The largest <stage>_id of such an exposure set could just be used. 
+
+duplicates were allowed because
+
+1: didn't know what the exact date would be
+2: can provide an old/new valid reprocessed exposure comparison
+
+*** even reprocessing of exposures for old format V2 exist, those were ignored and re-reprocessed for V3 warps and stacks later on. These could also be included in the duplicate case and using the largest <stage>_id in the duplicate set should exclude those if interested, however, if interested in the warp and later products the V3 format should be verified.
+
+
+Re-reprocessing tried to finish out observing season for some fields since date uncertain for PV1 initially, so will be duplicate processing for the same exposure.  Either should be acceptable to use, comparison of original and newer may be a helpful 
+
 
 
 
 == Chip+Warp ==
-Fully reprocessed but not distributed due to disk space load. Can be obtained via the Postage Stamp Server.
+Fully reprocessed but not distributed due to disk space load. These can be obtained via the Postage Stamp Server.
  * MD.GR0 was a mix of synthetic/ubercal catalog calibrations except MD08, 09. 
  * for PV1 chip_id, warp_id >= 433987, 393977 respectively it was processed using the 20120524v0 ubercal catalog and with detrends
@@ -24 +41 @@
  * MD.GR0 was a mix of synthetic/ubercal catalog calibrations except MD08, 09
  * a possible guideline to follow, including nightly science data processing,  if MD exposures with a cam_id >=412069 (actually >=411444 of some V2 destined reprocessing) is used, those will be with the ipp-20120404,r33642 tag that includes the faint-end bias fix and the ubercal catalog 20120524v0 version also used for LAP.
+
+cam_id>=434615
+
+
+
 
 
@@ -38 +60 @@
  * out-of-season/anti-yearly refstacks, yearly refstack: on hold for MD.GR1 (would need to update/reprocess large numbers of exposures to do)
 
-__Deep Stacks (deeptest)__: Not a full production product yet. Test version exists for MD04.deeptest.20120727 (synthetic catalog), MD09.deeptest.20120727 (ubercal catalog 20120524v0)
+__Deep Stacks (deeptest)__: Not a full production product yet. Test version exists for MD04.deeptest.20120727 (synthetic catalog, pre-PV1 warps), MD09.deeptest.20120727 (ubercal catalog 20120524v0, PV1 warps)
 
  
-__IQ/PSF Stack__: on hold until MD.GR1 (like out-of-season, would required many updates). A sample for MD09 done but requires development/code time. Test version exists for MD02.iqtest.20120927, MD10.psfrefstack.20110814.
+__IQ/PSF Stack__: on hold until MD.GR1 (like out-of-season, would required many updates). A sample for MD09 done but requires development/code time. Test version exists for MD02.iqtest.20120927, MD10.psfrefstack.20110814, but neither will be made from PV1 warps.
 
 
@@ -47 +69 @@
 Originally a possible reprocessing product, but was decided not useful at this time (for future reprocessing, need to outline interests for these if any). Will need to finish the convolution choice (input stack or reference stack) and improvements to convolution before started. 
 
+However, diff_id>= will be PV1 from nightly science processing
 
 
@@ -169 +192 @@
 
 
-