Summary STS.refstack.20120510

MOPS would like a refstack for the STS fields to do warp-stack diffims.

The previous STS.refstack.20110613,20110615 wasn't usable as it was destroyed by magic.

STS.20120510 was made using a similar number of inputs from the best exposures from 2011. Not pretty but hopefully sufficient.

Status:

• 5/ : test samples for refstack finished
• 5/ : refstack finished
• 5/ : test sample 1
• 5/ : test sample 2

Details to add:

• FWHM of test stacks (STS.reftest.20120510.c#f#) like done with MD refstacks
• unconv vs convolved stacks and issues found
• test diffims with recent STS data

STS observation and tessellation summary

rawExp.comment name details

• GIQ -- shorter exp_time (20s) to not saturate more sources in better seeing conditions
• ND -- no dither so always with center and typically with 30s, occasional NDGIQ
• NOM -- nominal IQ?
• 000,090,180,270 -- rotator to keep rough PA on field over season
• PP# -- 1-7, 7 pointings
• dither 1-4 - dither shifts (by ?) and typically with GIQ

7 projections and their skycell overlap (STS.V4, 0.25"/pixel platescale).

Approximate pattern, E left, N up:

 1 2     
3 4 5   
 6 7

There is significant overlap of projections and skycells, may cause issue with general input into a refstack skycell with wildly different IQ.

---

Refstack

Reference stack using data from 2011 only, a balance of N input warps versus best seeing, excluding exposures with large background, low zeropoints.

Chip->Warp processing: doing selected sample to use in the refstack only

• label STS.refstack.20120510
• data_group STS.refstack.20120510
• dist_group NULL
• workdir: neb://@HOST@.0/gpc1/STS.refstack.20120510

chiptool -definebyquery -set_label STS.mehtest -set_workdir neb://@HOST@.0/meh/STS.mehtest -set_data_group STS.mehtest -set_dist_group NULL -set_tess_id STS.V4 -set_end_stage warp -dbname gpc1 -dateobs_begin 2011-05-24 -dateobs_end 2011-05-25 -filter i% -comment STS% -obs_mode STS -simple -pretend >& teststschip.log
--> 5/24 has 84 exposures, yep so need more restrictive for test - do o5705g0373o
chiptool -definebyquery -set_label STS.mehtest -set_workdir neb://@HOST@.0/meh/STS.mehtest -set_data_group STS.mehtest -set_dist_group NULL -set_tess_id STS.V4 -set_end_stage warp -set_reduction STS_DATASET -dbname gpc1 -exp_name o5705g0373o -simple -pretend >& teststschip_o5705g0373o.log

-- test run - STS.mehtest - dont see corner unglow in camera image, dynamic range such that not see strongly? neb://@HOST@.0/meh/STS.mehtest/o5705g0373o.340508 --> see some patterns in warp background for sample -- significant? meh/STS.mehtest/o5705g0373o.340508/o5705g0373o.340508.wrp.408756.skycell.1.04.fits

mysql -hippdb03 -uippuser -pippuser gpc1 -B -e 'select rawExp.exp_id from chipRun join rawExp using (exp_id) join camRun using (chip_id) join camProcessedExp using (cam_id) where rawExp.obs_mode="STS" and rawExp.dateobs>"2011-05-23" and rawExp.dateobs<"2012-01" and  camProcessedExp.fwhm_major<3.9 and camProcessedExp.zpt_obs>24.5 and camProcessedExp.bg<120.0 and rawExp.comment like "%PP1%" group by rawExp.exp_id;' | grep -v "exp_id" 

perl sts.v4.refstackexp.20120510_pp1.chip > run_pp1.chip.bat
source run_pp1.chip.bat >& run_pp1.chip.log

Stack processing:

• label: STS.refstack.20120510
• data_group: STS.refstack.20120510
• workdir:
• reduction: PS1_REFERENCE

      OUTPUT.NOCOMP       BOOL    TRUE
      OUTPUT.LOGFLUX      BOOL    FALSE
      OUTPUT.REPLICATE    BOOL    TRUE
  

Test stack (reftest) sample also made again for skycell.055 with different number of input warps.

• label: STS.reftest.201204xx (xx=17 for g, 18 for r, 19 for i, 20 for z, 21 for y)
• data_group:
  • STS.reftest.201204xx.c#f<FWHM cut> where #=comparison run, <FWHM cut>=cut <fwhm_major from camProcessedExp
  • STS.reftest.201204xx.all.<FWHM cut> where all means 2009+2010+2011 data possible, <FWHM cut>=cut <fwhm_major from camProcessedExp
  • STS.reftest.201204xx.2011.<FWHM cut> where 2011 means 2011 data only, <FWHM cut>=cut <fwhm_major from camProcessedExp

version data_group (stack_id)	filter	center skycell input warps (rej)	center skycell FWHM (major)
new STS.refstack.20120510 V4	i	96 (2)	1.27"
STS.reftest.201205.c1f49 V4 ()	i	34 (1)	1.20"
old STS.refstack.2011 V3	i	78 (4)	1.45"

---

Test Diffim samples

Want to create a sample of diffims with detections for MOPS to look at. Will need to also keep in "full" state so they can get postage stamps quickly. Focus on projection 1.

survey.add.publish STS.mopsdiff STS.mopsdiffv2.mehtest 12 NULL
survey.show.publish 
survey.del.publish STS.mopsdiff
add.label STS.mopsdiffv2.mehtest

Diffim sample 1

Sample diffims for projection 1 faulted in publishing

Diffim sample 2

Rerun of sample diffims for projection 1 and published under IPP-MOPS-TEST-2

SELECT exp_id,exp_time,dateobs,zpt_obs,zpt_stdev,fwhm_major,format(fwhm_major/fwhm_minor,3) as elong,format(iq_fwhm_major/iq_fwhm_minor,3) as iq_elong,format(fwhm_major-iq_fwhm_major,2) as fwhm_iq_maj,format(camProcessedExp.bg,2) as bg,format(camProcessedExp.bg_stdev,2) as bg_stdev,format(fwhm_major_uq,2) as uq,format(fwhm_major_lq,2) as lq,n_stars,comment FROM rawExp join chipRun using (exp_id) join camRun using (chip_id) join camProcessedExp using (cam_id) join fakeRun using (cam_id) join warpRun using (fake_id) WHERE warpRun.data_group like "STS.refstack.20120510" and comment like "%PP1%" group by exp_id;

mysql -hippdb01 -uippuser -pippuser gpc1 -B -e 'select distinct exp_name,dateobs,chip_id,warp_id from warpRun join fakeRun using (fake_id) join camRun using (cam_id) join chipRun using (chip_id) join rawExp using (exp_id) where comment like "STS%PP1%" and dateobs like "2011-09-30%" and warpRun.state like "cleaned" and chipRun.state like "cleaned" group by exp_name; ' | grep -v chip_id | awk '{print "chiptool -dbname gpc1 -setimfiletoupdate -set_label STS.mopsdiffv2.mehtest -chip_id "$4"; warptool -dbname gpc1  -setskyfiletoupdate -set_label STS.mopsdiffv2.mehtest -warp_id  "$5}' > ~/updatechipwarp_sts.mopsdiffv2.mehtest_110930.bat
...

difftool -dbname gpc1 -definewarpstack -good_frac 0.2 -warp_label STS.mopsdiffv2.mehtest -stack_label STS.refstack.20120510 -set_dist_group NULL -set_label STS.mopsdiffv2.mehtest -set_workdir neb://@HOST@.0/meh/STS.mopsdiffv2.mehtest -set_data_group STS.mopsdiffv2.mehtest  -set_reduction WARPSTACK -simple -pretend

select * from diffRun where data_group like "STS.20110930";

difftool -dbname gpc1 -definewarpstack -good_frac 0.2 -warp_label STS.mopsdiffv2.mehtest -stack_label STS.refstack.20120510 -set_dist_group NULL -set_label STS.mopsdiffv2.mehtest -set_workdir neb://@HOST@.0/meh/STS.mopsdiffv2.mehtest -set_data_group STS.mopsdiffv2.mehtest  -set_reduction WARPSTACK -simple -rerun -pretend

pubtool -dbname gpc1 -definerun -label STS.mopsdiffv2.mehtest -client_id 12 -simple -pretend