Changes between Initial Version and Version 1 of MD01.refstack.20130731

Timestamp:

Aug 2, 2013, 11:00:28 AM (13 years ago)

Author:

Mark Huber

Comment:

--

MD01.refstack.20130731

@@ +1 @@
+= Summary of MD01.refstack.20130731 =
+[[PageOutline]]
+
+(back to [wiki:MediumDeepFields] summary)
+
+Similar to the revised MD01.refstack.20120803 ([wiki:MD01.refstack.20120803]) refstacks for the 2012 season, these include recent improvements for the PV2. Overall fewer inputs to the refstack were done this round and may not be as deep as last season.
+
+'''need to decide which parts of PV2 to include -- 1D conv, SAFE combining F, MATCH.REJ@4 rather than 5.. others..'''
+
+MD01 observations:
+ * starting late July 2013 with pre-observations, regular observations early Aug.
+ * 08/03/2012--02/12/2013
+ * 08/04/2011--01/15/2012
+ * 07/28/2010--02/05/2011
+ * 08/29/2009--01/26/2010
+ * (some 2008 but pre-survey)
+
+
+|| '''Filter''' || '''N total''' || '''N FWHM<8 pixels''' ||  '''N (FWHM<8 pixels for 2012''' || '''N FWHM<6 pixels''' ||  '''N (FWHM<6 pixels for 2012''' ||
+|| || || || ||
+|| g || 604 || 560 || 85 || 444 || 65 ||
+|| r || 574 || 540 || 86 || 458 || 71 ||
+|| i || 725 || 673 || 146 || 546 || 99 ||
+|| z || 943 || 900 || 281 || 774 || 237 ||
+|| y || 741 || 643 || 298 || 552 || 261 ||
+|| || || || ||
+ * 2012 wasn't a great season for MD01 gr, but better than MD09
+
+
+Status:
+ * 07/31/2013 selected exposure reprocessing chip->warp for refstack started 
+ * 08/02 exposures finished (processing issues). reftests started, both ISIS and 1D again to check rejections
+ * 08/05?? refstack finished, checked and problem skycells found/fixed if possible. staticsky single filter only finished and mappings started 
+ * 08/06?? mappings finished, field pre-obs exposures setup for diffims and finished 
+ * 08/07?? reftest+refstack to distribution
+ * 08/08?? updating wiki
+
+== MD01.V3  Processing (chip->warp + refstack) ==
+
+Exposure reprocessing was done for only a limited number (~150) of the best images from 2010-2012/2013 to provide a balance of N input warps versus best seeing, excluding exposures with large background, low zeropoints and large elongation. 
+
+Processing chip-warp and the refstack itself has the same label, data_group
+ * label: MD01.refstack.20130731 --> ''once set as active refstack for diffims, stack label will change to MD01.refstack'', chip/cam/warp label will remain the same
+ * data_group: MD01.refstack.20130731
+
+
+The ipp-20130715 tag used 
+ * [wiki:GPC1_Detrend_Documentation#Currentdetrends] -- new video darks as started with MD10.refstack.20130731
+ * dynamic masks now propagating through to warps -- ghost mask is slightly offset or missing at times -- will not be fixed for PV2
+ * 1-d convolution option for PV2 -- more rejection than expected i
+ * changes to psphot, PSF_MAJOR,MINOR root(2) different but appropriate now and 2*root(2) for FWHM; may be stability issue in PSF fitting
+ * SCL_ keyword fixed back to if rejected scaling value 0. instead of original value or 1. (as it has been before the bug)
+ * new reduction class with clear name MD_REF_STACK 
+ * SAFE config now set to false?
+
+
+Histograms of the FWHM major and minor and elongation from camera stage value (full exposure) for the sample of reprocessed exposures (red) and used for refstack.20130731 (top blue) and used for refstack.20120803 (bottom blue) in grizy order 
+
+|| [[Image(.g.expfwhm.png,300px)]] || [[Image(.r.expfwhm.png,300px)]] || [[Image(.i.expfwhm.png,300px)]] || 
+|| [[Image(.z.expfwhm.png,300px)]] || [[Image(.y.expfwhm.png,300px)]] ||   ||
+
+
+Test stack (reftest) sample also made again for skycell.055 with different number of input warps. 
+ * label: MD01.reftest.20130731
+ * data_group: 
+  * MD01.reftest.20130715.c#f<FWHM cut> where #=comparison run, <FWHM cut>=cut  <fwhm_major from camProcessedExp x10>
+  * MD01.reftest.20130715.all.<FWHM cut> where all means 05/2010+2011+2012/2013 data possible, <FWHM cut>=cut <fwhm_major from camProcessedExp x10>
+  * MD01.reftest.20130715.2012.<FWHM cut> where 2012 means 2012/2013 data only for season stack example case, <FWHM cut>=cut <fwhm_major from camProcessedExp x10>
+ * the main field center shift was through ~Dec 2009, but is a very small shift. Excluding 2009 since usually not very good, or uncharacteristically too good (avoiding problems basically)
+ * c1 cuts
+{{{
+filter: (FWHM set), elongation, zeropoint, background limits
+y: 
+z:
+i: 
+r: 
+g:
+--> refstack starts with  pixels for upper cut in grizy (may go smaller based on MEAN+1.0*STDDEV)
+}}}
+
+Summary comparison table with the approximate FWHM (major) as the PSF FWHM from a center field skycell (V3=055) after running psphot on the stacks. 
+
+{{{
+| data_group                        | filter  | stack_id | warps_in | warps_rej |  FWHM_MAJ   IQ_FW1 (pixels)
++-----------------------------------+---------+----------+----------+-----------+
+--> 2013 refstack
+
+--> reftest comparisons                                         
+
+--> previous 2012 refstack                              
+
+}}}
+
+As before, the FWHM_MAJOR with the central skycell can be variable and more so with largely different input sets. Comparison with the psf_major/minor maps are more useful between refstack versions.
+
+
+== Problems  ==
+ * as with all past refstacks, generally edge skycells have issues -- need better containment of those still. (see core coverage below). 
+ * like others, greater % of inputs rejected (20-30%) so per pixel input is less, but still of order 30 inputs per pixel
+ * some convolved stacks have poorer FWHM PSF but not significant enough to be likely improved by reprocessing using DEEP_STACK config as in earlier MD fields
+ * skycell notes: 
+  * y: s025,026,036,037,047,057 bad cell bands still.. s032 odd patch
+  * z: s012 quality fault, rerun with same MD_REF_STACK but change elong cut <1.25 from 1.3 to remove odd exposures messing up Target PSF; s046,063,065,066 odd,raised background (bad cell?)
+  * i: s065 like some other cells, some odd raised background (bad cell? glint?)
+  * r: s014,015,048,064,065,066,084 poor ghost mask/mess
+  * g: s014,015,022,023,027,032,037,045,048,058,064,065,067,085 poor ghost mask/mess; s038 edge really ratty; s044,054 poor cell arc (wonder if could/should mask cell as seen in [wiki:MD10.refstack.20120705])
+
+
+Core coverage  -- not a shifted pointing 
+{{{
+
+}}}
+ * skycells with fewer inputs (typical for edges): 
+ * skycells with <50% or more of typical input number is filter dependent - usually  
+
+
+
+
+== Refstack Summary ==
+Stack phomtometry on the refstacks done through the staticsky and skycal stages using the unconvolved images
+ * single filter mode so basic psphot with deteff measurements that has the zero-point recalculated for the test plots below
+  * label: MD01.refstack.20130731, MD01.refstack.20130731
+  * data_group (single filter):  MD01.refstack.20130731.1x20130805, MD01.refstack.20130731.1x20130805.cal20130805 
+ * multi-filter (Nx) also run
+  * label: 
+  * data_group:  (all skycells have 5 filters matched)
+  * on datastore under ps1-md-staticsky, but only the staticsky version, the skycal product still has no method for distribution
+
+
+Source photometry psf_major,_minor maps (grizy): '''all new plots here scaled by root(2) to match past scaling (but not on the plot label yet)'''
+
+|| [[Image(md01v2013x1skycal_cmf_g.txt_psfmap.png,300px)]] || [[Image(md01v2013x1skycal_cmf_r.txt_psfmap.png,300px)]] ||  [[Image(md01v2013x1skycal_cmf_i.txt_psfmap.png,300px)]] ||  [[Image(md01v2013x1skycal_cmf_z.txt_psfmap.png,300px)]] ||  [[Image(md01v2013x1skycal_cmf_y.txt_psfmap.png,300px)]] ||   
+||  ||  ||  || [[Image(md01v2013x1r130307sky_cmf_z.txt_psfmap.png,300px)]]  ||  || 
+
+
+Magnitude of sources@5-sigma maps (grizy):
+
+|| [[Image(md01v2013x1skycal_cmf_g.txt_mag5sigmap.png,300px)]] || [[Image(md01v2013x1skycal_cmf_r.txt_mag5sigmap.png,300px)]] ||  [[Image(md01v2013x1skycal_cmf_i.txt_mag5sigmap.png,300px)]] ||  [[Image(md01v2013x1skycal_cmf_z.txt_mag5sigmap.png,300px)]] ||  [[Image(md01v2013x1skycal_cmf_y.txt_mag5sigmap.png,300px)]] || 
+||  ||  ||  || [[Image(md01v2013x1r130307sky_cmf_z.txt_mag5sigmap.png,300px)]]  ||  || 
+
+
+Detection efficiency magnitude @50% recovery (500 fakes, grizy): '''0 rej for all now that the SCL_ keyword has its true value other than 1. when rejected in stacking'''
+
+|| [[Image(md01v2013x1skycal_cmf_g.txt_mag50gridmap.png,300px)]] || [[Image(md01v2013x1skycal_cmf_r.txt_mag50gridmap.png,300px)]] || [[Image(md01v2013x1skycal_cmf_i.txt_mag50gridmap.png,300px)]] || [[Image(md01v2013x1skycal_cmf_z.txt_mag50gridmap.png,300px)]] || [[Image(md01v2013x1skycal_cmf_y.txt_mag50gridmap.png,300px)]] || 
+||  ||  ||  || [[Image(md01v2013x1r130307sky_cmf_z.txt_mag50gridmap.png,300px)]] ||  || 
+
+
+----
+
+= 1D convolution mode PV2 =
+
+
+Same naming format as generally above, adding _1dg to the data_group (as will be seen on the datastore)
+ * label: MD01.reftest.20130731
+ * data_group: 
+  * MD01.reftest.20130731.c#f<FWHM cut>_1dg where #=comparison run, <FWHM cut>=cut  <fwhm_major from camProcessedExp x10>
+  * MD01.reftest.20130731.all.<FWHM cut>_1dg where all means 05/2010+2011+2012/2013 data possible, <FWHM cut>=cut <fwhm_major from camProcessedExp x10>
+  * MD01.reftest.20130731.2012.<FWHM cut>_1dg where 2012 means 2012/2013 data only for season stack example case, <FWHM cut>=cut <fwhm_major from camProcessedExp x10>
+ * c1 cuts (same as above)
+
+Full refstack:
+ * label: MD01.refstack.20130731_1dg
+ * data_group: MD01.refstack.20130731_1dg 
+
+Summary comparison table with the approximate FWHM (major) as the PSF FWHM from a center field skycell (V3=055) after running psphot on the stacks. 
+
+{{{
+| data_group                        | filter  | stack_id | warps_in | warps_rej |  FWHM_MAJ   IQ_FW1 (pixels)
++-----------------------------------+---------+----------+----------+-----------+
+
+
+}}}
+
+
+
+----