Changes between Version 1 and Version 2 of MD.PV2

Timestamp:

Jan 30, 2014, 11:36:41 AM (13 years ago)

Author:

Mark Huber

Comment:

--

MD.PV2

@@ -6 +6 @@
  * [wiki:MD.PV1] summary page -- the data set that PV2 update reprocessing is based on
 
-(more to come)
+(much more to be added)
+
+This is a summary of the MD.PV2 update processing (as started with MD04 in [wiki:MD04.pv2_stacktest.20130903]) for all MD exposures through September 2013. Some fields are still active (MD09,10,01,02) and some soon to be so (MD03,04), and all will have final exposures at the end of the survey (~February/March 2014) so all numbers will at least by a few. 
+
+Like GR0, will have summary for each field, but unlike GR0 will try and keep analysis and issues to other pages to reduce the length and load time for the summary page. Updates will be added from active fields and such on a MD by MD basis.
+
+
+Status:
+ * prior 9/3 many issues being worked out for PV2 and MD ([wiki:MD10.refstack.20130715])
+ * 9/3 PV2 updates started for MD04 after M31 reprocessing finished
+ * 9/6 MD04 PV2 updates finished -- so can proceed rather quickly since only update processing
+ * 10/17 PV2 updates started for MD03
+ * 10/20 PV2 updates finished 
+ * (lots of update processing when ever possible)
+ * end of Nov. essentially finished -- still issues to resolve by >90% done
+ * 
+
+
+----
+
+= MD01 =
+
+PV1 camera label -- data_group: mix of processing labels/data_groups, but all of PV1 type (cam_id>=434615)
+|| label || data_group ||
+|| MD01.pv1.20130325 || MD01.pv1.20130325 ||
+|| MD01.GR0.20121122 || MD01.GR0.20121122 ||
+|| MD01.refstack.20120803 || MD01.refstack.20120803 ||
+|| MD01.nightlystack || MD01.<dateobs x--x> ||
+|| || ||
+
+There have been some test reprocessing that had to be accounted for -- czw.MDVDtest%..
+
+PV2 camera (as well as chip, warp) label '''MD01.pv2.20131031''', data_group left to what originally set as in order to be able to track backwards to the processing group.
+
+Number totals through Sept. 2013: (will be more by end of survey to add)
+ * not all exposures pass camera stage, FWHM>12 pixels or other quality issues
+{{{
+| filter  | count(cam_id) | count(warp_id) |
++---------+---------------+----------------+
+| g.00000 |           887 |            864 | 
+| i.00000 |           974 |            971 | 
+| r.00000 |           849 |            831 | 
+| y.00000 |          1018 |           1000 | 
+| z.00000 |          1264 |           1249 | 
+}}}
+
+PV2 warps will also be kept available (not cleaned up) for making deep stacks  but also will be a accessible via the PSS/PSPS interface more quickly (release name MD01.PV2).
+
+'''(still need to load into release system)'''
+
+
+----
+
+= MD02 = 
+
+PV1 camera label -- data_group: mix of processing labels/data_groups, but all of PV1 type (cam_id>=434615)
+|| label || data_group ||
+|| MD02.pv1.20130325 || MD02.pv1.20130325 ||
+|| MD02.GR0.20121210 || MD02.GR0.20121210 ||
+|| MD02.refstack.20120927 || MD02.refstack.20120927 ||
+|| MD02.nightlyscience || MD02.<dateobs x--x> ||
+|| || ||
+
+Test reprocessing for MD02 properly sorted and none clearly needed to be avoided.
+
+PV2 camera (as well as chip, warp) label '''MD02.pv2.20131101''', data_group left to what originally set as in order to be able to track backwards to the processing group.
+
+Number totals through Sept. 2013: (will be more by end of survey to add)
+ * not all exposures pass camera stage, FWHM>12 pixels or other quality issues
+{{{
+| filter  | count(cam_id) | count(warp_id) |
++---------+---------------+----------------+
+| g.00000 |           590 |            573 | 
+| i.00000 |           746 |            744 | 
+| r.00000 |           609 |            602 | 
+| y.00000 |           847 |            822 | 
+| z.00000 |           911 |            901 | 
+}}}
+
+PV2 warps will also be kept available (not cleaned up) for making deep stacks  but also will be a accessible via the PSS/PSPS interface more quickly (release name MD01.PV2).
+
+'''(still need to load into release system)'''
+
+-----
+
+= MD03 =
+
+(from [wiki:MD03.pv2_refstack.20131018#PV2updates])
+
+PV1 camera label -- data_group: mix of processing labels/data_groups, but all of PV1 type (cam_id>=434615)
+|| label || data_group ||
+|| MD03.pv1.20130326 || MD03.pv1.20130326 ||
+|| MD03.refstack.20121101 || MD03.refstack.20121101 ||
+|| MD03.GR0.20121212 || MD03.GR0.20121212 ||
+|| MD03.nightlyscience || MD03.<dateobs x--x> ||
+|| || ||
+
+Test reprocessing for MD03 properly sorted and none clearly needed to be avoided.
+
+PV2 camera (as well as chip, warp) label '''MD03.pv2.20131018''', data_group left to what originally set as in order to be able to track backwards to the processing group.
+
+Number totals through Sept. 2013: (will be more by end of survey to add)
+ * not all exposures pass camera stage, FWHM>12 pixels or other quality issues
+{{{
+| filter  | count(cam_id) | count(warp_id) |
++---------+---------------+----------------+
+| g.00000 |           733 |            720 | 
+| r.00000 |           709 |            707 | 
+| i.00000 |           871 |            869 | 
+| z.00000 |          1168 |           1151 | 
+| y.00000 |           903 |            898 | 
+}}}
+
+PV2 warps will also be kept available (not cleaned up) for making deep stacks  but also will be a accessible via the PSS/PSPS interface more quickly (release name MD03.PV2).
+
+'''(still need to load into release system)'''
+
+----
+
+
+= MD04 = 
+
+(from [wiki:MD04.pv2_stacktest.20130903#MD04])
+
+PV1 camera label -- data_group: mix of processing labels/data_groups, but all of PV1 type (cam_id>=434615)
+|| label || data_group ||
+|| MD04.pv1.20130322 || MD04.pv1.20130322 ||
+|| MD04.refstack.20121125 || MD04.refstack.20121125 ||
+|| MD04.nightlyscience  || MD04.<obsdate 20121129--20130525> 
+|| || ||
+
+There have been some test reprocessing that had to be avoided -- czw.MDVDtest%, czw.MDVD.MD04.%
+
+PV2 camera (as well as chip, warp) label '''MD04.pv2.20130903''', data_group left to what originally set as in order to be able to track backwards to the processing group.
+
+
+With ipp047 data disk down, have found ~3% of exposures missing a related imfile (~half are the burn.tbl files that can be regenerated) and indicates serious need of fixing the replication of raw data. Also found some replicated but on a host (ipp064X) that no longer exists.. it is also possible some replicated products may be on the same host if push to the stsci nodes with 3 partitions..
+
+
+Number totals through Sept. 2013: (will be more by end of survey) 
+ * not all exposures pass camera stage, FWHM>12 pixels or other quality issues
+{{{
+| filter  | count(cam_id) | count(warp_id) |
++---------+---------------+----------------+
+| g.00000 |           596 |            577 | 
+| r.00000 |           588 |            571 | 
+| i.00000 |           719 |            713 | 
+| z.00000 |           711 |            710 | 
+| y.00000 |           471 |            465 | 
+}}}
+
+PV2 warps will also be kept available (not cleaned up) for making deep stacks  but also will be a accessible via the PSS/PSPS interface more quickly (release name MD04.PV2).
+
+Loaded into release system as MD04.PV2 except for the camera quality faulted exposures.
+
+----
+
+= MD05 = 
+
+PV1 camera label -- data_group: mix of processing labels/data_groups, but all of PV1 type (cam_id>=434615). MD05 had a large mix of GR0 reprocessing done for QUB.
+|| label || data_group ||
+|| MD05.pv1.20130327 || MD05.pv1.20130327 ||
+|| MD05.refstack.20121202 || MD05.refstack.20121202 ||
+|| MD05.GR0.20121005 || MD05.GR0.20121005 ||
+|| MD05.GR0.20121030 || MD05.GR0.20121030 ||
+|| MD05.GR0.20121104 || MD05.GR0.20121104 ||
+|| MD05.nightlyscience || MD05.<dateobs x--x> ||
+|| || ||
+
+Test reprocessing for MD02 properly sorted and none clearly needed to be avoided.
+
+PV2 camera (as well as chip, warp) label '''MD05.pv2.20131022''', data_group left to what originally set as in order to be able to track backwards to the processing group.
+
+Number totals through Sept. 2013: (will be more by end of survey to add)
+ * not all exposures pass camera stage, FWHM>12 pixels or other quality issues
+{{{
+| filter  | count(cam_id) | count(warp_id) |
++---------+---------------+----------------+
+| g.00000 |           761 |            759 | 
+| i.00000 |           806 |            799 | 
+| r.00000 |           722 |            718 | 
+| y.00000 |           750 |            748 | 
+| z.00000 |           901 |            899 | 
+}}}
+
+PV2 warps will also be kept available (not cleaned up) for making deep stacks  but also will be a accessible via the PSS/PSPS interface more quickly (release name MD01.PV2).
+
+'''(still need to load into release system)'''
+
+----
+
+= MD06 = 
+
+----
+
+= MD07 = 
+
+----
+
+= MD08 = 
+
+----
+
+= MD09 = 
+
+----
+
+= MD10 = 
+
+----