IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 1 and Version 2 of MOPS.dailytestset


Ignore:
Timestamp:
May 2, 2018, 8:24:22 AM (8 years ago)
Author:
Mark Huber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • MOPS.dailytestset

    v1 v2  
    1 The recent time SNAFU of silently changing a telescope header keyword for SHUTOUTC because system /etc/localtime was incorrect has highlighted the need for a regular/daily test sample of IPP processing... Ken has agreed this is necessary ASAP.
     1The recent time SNAFU of silently changing a telescope header keyword for SHUTOUTC because system /etc/localtime was incorrect ([wiki:PS1_IPP_Czarlog_20150810], [wiki:PS1_timing_problems]) has highlighted the need for a regular/daily test sample of IPP processing... Ken has agreed this is necessary ASAP.
    22
    3 MOPS is willing to do the extra work to select and analyze the test set, the IPP should be willing to also process and provid it. In the near future a set of WS diffs for QUB should also be added as it could help to highlight other problems.
     3MOPS is willing to do the extra work to select and analyze the test set, the IPP should be willing to also process and provide it. In the near future a set of WS diffs for MOPS+QUB should also be added as it could help to highlight other problems.
    44
    5 Initial/current test quad --
    6  * o6370g0351o o6370g0373o o6370g0395o o6370g0417o from 162.OSS.O.Q.w (2013-03-19)
     5Issues:
     6 * original case was to catch odd node issue, needs to touch on all nightly science nodes (if >60 then some might not be checked) -- need to check hostname for any missed
     7 * ...
    78
    8 To run -- the normal chip/cam/warp, WWdiff v12,v34
    99
    10 Initial commands --
     10Initial/current test quad -- 162.OSS.O.Q.w (2013-03-19) exposures -- o6370g0351o, o6370g0373o, o6370g0395o, o6370g0417o
     11{{{
     12+-------------+---------------------+----------+---------+-----------------------------------+------------+-----------+----------+
     13| exp_name    | dateobs             | exp_time | filter  | comment                           | posang     | radeg     | decdeg   |
     14+-------------+---------------------+----------+---------+-----------------------------------+------------+-----------+----------+
     15| o6370g0351o | 2013-03-19 11:37:26 |       45 | w.00000 | 162.OSS.O.Q.w ps1_14_2124 visit 1 | 337.868713 | 186.44580 | -2.16131 |
     16| o6370g0373o | 2013-03-19 11:59:29 |       45 | w.00000 | 162.OSS.O.Q.w ps1_14_2124 visit 2 | 337.876892 | 186.44597 | -2.16143 |
     17| o6370g0395o | 2013-03-19 12:21:39 |       45 | w.00000 | 162.OSS.O.Q.w ps1_14_2124 visit 3 | 337.877411 | 186.44591 | -2.16137 |
     18| o6370g0417o | 2013-03-19 12:43:52 |       45 | w.00000 | 162.OSS.O.Q.w ps1_14_2124 visit 4 | 337.877594 | 186.44574 | -2.16125 |
     19
     20+--------+-------------+-------+------------+---------+---------+
     21| exp_id | exp_name    | fault | fwhm_major | quality | deteff  |
     22+--------+-------------+-------+------------+---------+---------+
     23| 590994 | o6370g0351o |     0 |    5.26383 |       0 | 22.7106 |
     24| 591016 | o6370g0373o |     0 |    5.99344 |       0 | 22.4768 |
     25| 591038 | o6370g0395o |     0 |    6.14446 |       0 | 22.4466 |
     26| 591060 | o6370g0417o |     0 |    6.48741 |       0 | 22.3831 |
     27}}}
     28
     29
     30
     31Manually to run for code change test (never use a script except for generating the base commands) -- normal chip/cam/warp, WWdiff v12,v34 and WSdiff, publishing
     32 * chip-cam-warp -- include date and revision being tested
     33{{{
     34chiptool -dbname gpc1 -definebyquery -set_label MOPS.dailytestset -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.DATErN -set_data_group MOPS.dailytestset.DATErN -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_reduction SWEETSPOT -simple -exp_name o6370g0351o  -pretend
     35}}}
     36
     37 * WWdiffs to queue -- o6370g0351o-o6370g0373o, o6370g0395o-o6370g0417o (warp_id - template_warp_id)
     38{{{
     39difftool -dbname gpc1 -definewarpwarp -warp_id VISIT1/VISIT3 -template_warp_id VISIT2/VISIT4 -backwards  -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.DATErN -set_dist_group NULL -set_label MOPS.dailytestset -set_data_group MOPS.dailytestset.DATErN -set_reduction SWEETSPOT -simple -rerun -pretend
     40}}}
     41
     42 * WSdiffs to queue -- testing new w-band reference stack, currently only run for testing that
     43{{{
     44difftool -dbname gpc1 -definewarpstack -good_frac 0.05 -warp_id WARPID -stack_label PSNSC.wref.20170707 -set_dist_group NULL -set_label MOPS.dailytestset -set_data_group MOPS.WS.dailytestset.DATErN -set_workdir neb://@HOST@.0/gpc1/MOPS.WS.dailytestset.DATErN  -set_reduction SWEETSPOT_WS -simple -rerun -pretend
     45}}}
     46
     47 * checks before publishing -- skycell count, good skycell count, number of detections per skycell...
     48
     49 * publishing -- using IPP-MOPS-TEST-2, client_id 12 until time to restore the IPP-MOPS-dailytestset location...
     50{{{
     51pubtool -dbname gpc1 -definerun -set_label MOPS.dailytestset -data_group MOPS.dailytestset.DATErN -client_id 12 -simple -pretend
     52
     53////pubtool -dbname gpc1 -definerun -set_label MOPS.dailytestset -data_group MOPS.WS.dailytestset.DATErN -client_id 12 -simple -pretend
     54}}}
     55
     56 * publishing list email to Serge/MOPS -- should be
     57{{{
     58mysql -hippdb08 -uippuser -p gpc1 -B -e ' select path_base from publishRun join publishDone using (pub_id) join diffRun where publishRun.label like "MOPS.dailytestset" and stage_id=diff_id and diffRun.data_group like "MOPS.dailytestset.DATErN"; ' | grep -v path_base | awk '{print "neb-ls -p "$1"%.mops"}' | tcsh
     59}}}
     60
     61 * cleanup -- should never do for a normal test set unless setup for testing updates etc
     62
     63
     64
     65
     66
     67Initial run description (until script finished) for daily test -- normal chip/cam/warp, WWdiff v12,v34 and publishing
     68 * chip-cam-warp commands --
    1169{{{
    1270
    … …  
    1876
    1977chiptool -dbname gpc1 -definebyquery -set_label MOPS.dailytestset -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.20150815 -set_data_group MOPS.dailytestset.20150815 -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_reduction SWEETSPOT -simple -exp_name  o6370g0417o -pretend
    20 
     78}}}
     79 
     80 * WWdiffs to queue -- o6370g0351o-o6370g0373o, o6370g0395o-o6370g0417o (warp_id - template_warp_id)
     81{{{
    2182difftool -dbname gpc1 -definewarpwarp -warp_id 1615271 -template_warp_id 1615273 -backwards -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.20150815 -set_dist_group NULL -set_label MOPS.dailytestset -set_data_group MOPS.dailytestset.20150815 -set_reduction SWEETSPOT -simple -pretend
    2283
    2384difftool -dbname gpc1 -definewarpwarp -warp_id 1615268 -template_warp_id 1615270 -backwards -set_workdir neb://@HOST@.0/gpc1/MOPS.dailytestset.20150815 -set_dist_group NULL -set_label MOPS.dailytestset -set_data_group MOPS.dailytestset.20150815 -set_reduction SWEETSPOT -simple -pretend
    24 
    25 pubtool -dbname gpc1 -definerun -set_label MOPS.dailytestset -data_group MOPS.dailytestset.20150815 -client_id 16 -simple -pretend
    26 
    2785}}}
    2886
    29 Publication location on datastore -- IPP-MOPS-dailytestset
     87 * WSdiffs -- not ready, still in testing with new w-band refstack
     88{{{
     89difftool -dbname gpc1 -definewarpstack -good_frac 0.05 -warp_id WARPID -stack_label PSNSC.wref.20170707 -set_dist_group NULL -set_label MOPS.dailytestset -set_data_group MOPS.WS.dailytestset.DATE -set_workdir neb://@HOST@.0/gpc1/MOPS.WS.dailytestset.DATE  -set_reduction SWEETSPOT_WS -simple -rerun -pretend
     90}}}
    3091
    31 Cleanup -- no cleanup until setup worked out, then likely <12 hours
     92 * publication location on datastore -- IPP-MOPS-dailytestset, client_id 16 -- was lost in the move to ITC and needs to be restored, likely needs to just point to ipp113 in db
     93{{{
     94pubtool -dbname gpc1 -definerun -set_label MOPS.dailytestset -data_group MOPS.dailytestset.20150815 -client_id 16 -simple -pretend
     95}}}
     96
     97 * cleanup -- no cleanup until setup worked out, then likely <12 hours is acceptable
     98
     99
     100Summary of past runs -- ...
    32101
    33102