= Summary STS.refstack.20120510 = 
[[PageOutline]]

MOPS would like a refstack for the STS fields to do warp-stack diffims.

The previous STS.refstack.20110613,20110615 wasn't usable as it was destroyed by magic.

STS.20120510 was made using a similar number of inputs from the best exposures from 2011. Not pretty but hopefully sufficient.

Status:
 * 5/ : test samples for refstack finished
 * 5/ : refstack finished
 * 5/ : test sample 1
 * 5/ : test sample 2 

Details to add:
 * FWHM of test stacks (STS.reftest.20120510.c#f#) like done with MD refstacks
 * unconv vs convolved stacks and issues found
 * test diffims with recent STS data



= STS observation and tessellation summary =

rawExp.comment name details 

 * GIQ -- shorter exp_time (20s) to not saturate more sources in better seeing conditions
 * ND -- no dither so always with center and typically with 30s, occasional NDGIQ
 *  NOM -- nominal IQ?
 * 000,090,180,270 -- rotator to keep rough PA on field over season
 * PP# -- 1-7, 7 pointings
 * dither 1-4 - dither shifts (by ?) and typically with GIQ



|| [[Image(STS_i_smf_fieldcolor.png,300px)]] || 7 projections and their skycell overlap (STS.V4, 0.25"/pixel platescale). ||

Approximate pattern, E left, N up:
{{{
 1 2     
3 4 5   
 6 7
}}}


There is significant overlap of projections and skycells, may cause issue with general input into a refstack skycell with wildly different IQ.

----

= Refstack = 

Reference stack using data from 2011 only, a balance of N input warps versus best seeing, excluding exposures with large background, low zeropoints. 

Chip->Warp processing: doing selected sample to use in the refstack only
 * label STS.refstack.20120510
 * data_group STS.refstack.20120510
 * dist_group NULL 
 * workdir: neb://@HOST@.0/gpc1/STS.refstack.20120510

{{{
chiptool -definebyquery -set_label STS.mehtest -set_workdir neb://@HOST@.0/meh/STS.mehtest -set_data_group STS.mehtest -set_dist_group NULL -set_tess_id STS.V4 -set_end_stage warp -dbname gpc1 -dateobs_begin 2011-05-24 -dateobs_end 2011-05-25 -filter i% -comment STS% -obs_mode STS -simple -pretend >& teststschip.log
--> 5/24 has 84 exposures, yep so need more restrictive for test - do o5705g0373o
chiptool -definebyquery -set_label STS.mehtest -set_workdir neb://@HOST@.0/meh/STS.mehtest -set_data_group STS.mehtest -set_dist_group NULL -set_tess_id STS.V4 -set_end_stage warp -set_reduction STS_DATASET -dbname gpc1 -exp_name o5705g0373o -simple -pretend >& teststschip_o5705g0373o.log
}}}

-- test run - STS.mehtest - dont see corner unglow in camera image, dynamic range such that not see strongly?
neb://@HOST@.0/meh/STS.mehtest/o5705g0373o.340508
--> see some patterns in warp background for sample -- significant?
meh/STS.mehtest/o5705g0373o.340508/o5705g0373o.340508.wrp.408756.skycell.1.04.fits

{{{
mysql -hippdb03 -uippuser -pippuser gpc1 -B -e 'select rawExp.exp_id from chipRun join rawExp using (exp_id) join camRun using (chip_id) join camProcessedExp using (cam_id) where rawExp.obs_mode="STS" and rawExp.dateobs>"2011-05-23" and rawExp.dateobs<"2012-01" and  camProcessedExp.fwhm_major<3.9 and camProcessedExp.zpt_obs>24.5 and camProcessedExp.bg<120.0 and rawExp.comment like "%PP1%" group by rawExp.exp_id;' | grep -v "exp_id" 

perl sts.v4.refstackexp.20120510_pp1.chip > run_pp1.chip.bat
source run_pp1.chip.bat >& run_pp1.chip.log
}}}

Stack processing:
 * label: STS.refstack.20120510
 * data_group: STS.refstack.20120510
 * workdir: 
 * reduction: PS1_REFERENCE
{{{
    OUTPUT.NOCOMP       BOOL    TRUE
    OUTPUT.LOGFLUX      BOOL    FALSE
    OUTPUT.REPLICATE    BOOL    TRUE
}}}

Test stack (reftest) sample also made again for skycell.055 with different number of input warps.
 * label: STS.reftest.201204xx  (xx=17 for g, 18 for r, 19 for i, 20 for z, 21 for y)
 * data_group: 
  * STS.reftest.201204xx.c#f<FWHM cut> where #=comparison run, <FWHM cut>=cut  <fwhm_major from camProcessedExp
  * STS.reftest.201204xx.all.<FWHM cut> where all means 2009+2010+2011 data possible, <FWHM cut>=cut <fwhm_major from camProcessedExp
  * STS.reftest.201204xx.2011.<FWHM cut> where 2011 means 2011 data only, <FWHM cut>=cut <fwhm_major from camProcessedExp


|| || || || ||
|| '''version data_group''' (stack_id) || '''filter''' || '''center skycell input warps (rej)''' || '''center skycell FWHM (major)''' || 
|| || || || ||
|| new ''STS.refstack.20120510'' V4  || i || 96 (2) || 1.27" ||
|| STS.reftest.201205.c1f49 V4  () || i || 34 (1) || 1.20"  ||
|| old STS.refstack.2011 V3  || i || 78 (4) || 1.45" ||

----
= Test Diffim samples = 
Want to create a sample of diffims with detections for MOPS to look at. Will need to also keep in "full" state so they can get postage stamps quickly. Focus on projection 1.


{{{
survey.add.publish STS.mopsdiff STS.mopsdiffv2.mehtest 12 NULL
survey.show.publish 
survey.del.publish STS.mopsdiff
add.label STS.mopsdiffv2.mehtest
}}}


== Diffim sample 1 ==
Sample diffims for projection 1 faulted in publishing 



== Diffim sample 2 ==
Rerun of sample diffims for projection 1 and published under IPP-MOPS-TEST-2 


{{{
SELECT exp_id,exp_time,dateobs,zpt_obs,zpt_stdev,fwhm_major,format(fwhm_major/fwhm_minor,3) as elong,format(iq_fwhm_major/iq_fwhm_minor,3) as iq_elong,format(fwhm_major-iq_fwhm_major,2) as fwhm_iq_maj,format(camProcessedExp.bg,2) as bg,format(camProcessedExp.bg_stdev,2) as bg_stdev,format(fwhm_major_uq,2) as uq,format(fwhm_major_lq,2) as lq,n_stars,comment FROM rawExp join chipRun using (exp_id) join camRun using (chip_id) join camProcessedExp using (cam_id) join fakeRun using (cam_id) join warpRun using (fake_id) WHERE warpRun.data_group like "STS.refstack.20120510" and comment like "%PP1%" group by exp_id;
}}}

{{{
mysql -hippdb01 -uippuser -pippuser gpc1 -B -e 'select distinct exp_name,dateobs,chip_id,warp_id from warpRun join fakeRun using (fake_id) join camRun using (cam_id) join chipRun using (chip_id) join rawExp using (exp_id) where comment like "STS%PP1%" and dateobs like "2011-09-30%" and warpRun.state like "cleaned" and chipRun.state like "cleaned" group by exp_name; ' | grep -v chip_id | awk '{print "chiptool -dbname gpc1 -setimfiletoupdate -set_label STS.mopsdiffv2.mehtest -chip_id "$4"; warptool -dbname gpc1  -setskyfiletoupdate -set_label STS.mopsdiffv2.mehtest -warp_id  "$5}' > ~/updatechipwarp_sts.mopsdiffv2.mehtest_110930.bat
...

difftool -dbname gpc1 -definewarpstack -good_frac 0.2 -warp_label STS.mopsdiffv2.mehtest -stack_label STS.refstack.20120510 -set_dist_group NULL -set_label STS.mopsdiffv2.mehtest -set_workdir neb://@HOST@.0/meh/STS.mopsdiffv2.mehtest -set_data_group STS.mopsdiffv2.mehtest  -set_reduction WARPSTACK -simple -pretend
}}}


{{{
select * from diffRun where data_group like "STS.20110930";

difftool -dbname gpc1 -definewarpstack -good_frac 0.2 -warp_label STS.mopsdiffv2.mehtest -stack_label STS.refstack.20120510 -set_dist_group NULL -set_label STS.mopsdiffv2.mehtest -set_workdir neb://@HOST@.0/meh/STS.mopsdiffv2.mehtest -set_data_group STS.mopsdiffv2.mehtest  -set_reduction WARPSTACK -simple -rerun -pretend

pubtool -dbname gpc1 -definerun -label STS.mopsdiffv2.mehtest -client_id 12 -simple -pretend

}}}