IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links
wiki:PS1_IPP_Czarlog_20190211

PS1 IPP Czar Logs for the week 2019.02.11 - 2019.02.17

(Up to PS1 IPP Czar Logs)

Monday : 2019.02.11

  • 08:15 EAM : without the ipp-b nodes, and ippb04 & ippb05 in repair for rsyncs, many update jobs failed due to lack of a backup target. I've put those two machines back to nebulous 'up' and reverted the failed update jobs.

JRF: Tidying up the mess I left and failed to rectify over the weekend

Some errors in chip stage for ps_ud_WEB

Exp Name    Class ID    Exp ID  Chip ID State   Label   data grp    dist grp    Fault   Revert Command
o5327g0047o XY26    165316  512230  update  ps_ud_WEB   ecliptic.rp.7hr     4   chiptool -revertprocessedimfile -fault 4 -label ps_ud_WEB -dbname gpc1
o5675g0136o XY26    328761  512289  update  ps_ud_WEB   ecliptic.rp.7hr     4   chiptool -revertprocessedimfile -fault 4 -label ps_ud_WEB -dbname gpc1
o5675g0136o XY61    328761  512289  update  ps_ud_WEB   ecliptic.rp.7hr     4   chiptool -revertprocessedimfile -fault 4 -label ps_ud_WEB -dbname gpc1
o5675g0136o XY74    328761  512289  update  ps_ud_WEB   ecliptic.rp.7hr     4   chiptool -revertprocessedimfile -fault 4 -label ps_ud_WEB -dbname gpc1
o5829g0438o XY26    399488  512334  update  ps_ud_WEB   ecliptic.rp.7hr     4   chiptool -revertprocessedimfile -fault 4 -label ps_ud_WEB -dbname gpc1
o5829g0438o XY61    399488  512334  update  ps_ud_WEB   ecliptic.rp.7hr     4   chiptool -revertprocessedimfile -fault 4 -label ps_ud_WEB -dbname gpc1
o5829g0438o XY62    399488  512334  update  ps_ud_WEB   ecliptic.rp.7hr     4   chiptool -revertprocessedimfile -fault 4 -label ps_ud_WEB -dbname gpc1

Query the first exp_id:

SELECT chipRun.exp_id,chipRun.chip_id,chipRun.label,chipRun.state, camRun.cam_id,camRun.state, fakeRun.fake_id,fakeRun.state, warpRun.warp_id,warpRun.state FROM chipRun LEFT JOIN camRun USING (chip_id) LEFT JOIN fakeRun USING (cam_id) LEFT JOIN warpRun USING (fake_id) WHERE exp_id=165316;

Gives the following:

+--------+---------+---------------------+---------+---------+-------+---------+-------+---------+---------+
| exp_id | chip_id | label               | state   | cam_id  | state | fake_id | state | warp_id | state   |
+--------+---------+---------------------+---------+---------+-------+---------+-------+---------+---------+
| 165316 |   92712 | goto_cleaned        | cleaned |   76896 | full  |   63594 | full  |   61845 | cleaned | 
| 165316 |   92712 | goto_cleaned        | cleaned |   76896 | full  |   63594 | full  |   64803 | cleaned | 
| 165316 |  113216 | cleanup_testing_old | cleaned |   96018 | full  |   82702 | full  |   81939 | cleaned | 
| 165316 |  512230 | ps_ud_WEB           | update  |  494867 | full  |  474675 | full  |  476374 | update  | 
| 165316 | 2128504 | PSNSC.wfix.20180525 | full    | 2094976 | full  | 2063980 | full  | 2070634 | full    | 
| 165316 | 2129009 | PSNSC.wfix.20180525 | new     |    NULL | NULL  |    NULL | NULL  |    NULL | NULL    | 
+--------+---------+---------------------+---------+---------+-------+---------+-------+---------+---------+

So it looks like there is a complete state for this exposure. There also appears to be an erroneous one 2129009 which has been sent after that is not needed (I'm guessing this did not process as the label was missing). Should cleanup chip_id=512230 and warp_id=476374 to get rid of the errors:

chiptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -chip_id 512230
warptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -warp_id 476374

However, notice that there is another entry that was sent for reprocessing. This is likely a mistake on my part not realising that it had been reprocessed successfully. I already set it to cleaned, but really it should have been either 'dropped' from the table as it hasn't been done and doesn't need to be, or it should just be processed as I'm unsure if a later chip_id will cause problems (Mark also said he was unsure if it would cause problems).

Looks the same for both of the other exposures, so this time I will set enable the label in pantasks, and get the other exposures to process. Then clean the remaining images:

pantasks_client
add.label PSNSC.wfix.20180525
exit

chiptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -chip_id 512289
chiptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -chip_id 512334
warptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -warp_id 476431
warptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -warp_id 476477

Errors in the warp stage for ps_ud_MOPS.2

o5327g0047o 165316  512230  494867  474675  476374  skycell.1873.066    update  ps_ud_MOPS.2    2   warp.revert.on and warp.revert.off
o5327g0047o 165316  512230  494867  474675  476374  skycell.1873.076    update  ps_ud_MOPS.2    2   warp.revert.on and warp.revert.off
o5675g0136o 328761  512289  494926  474732  476431  skycell.1874.008    update  ps_ud_MOPS.2    2   warp.revert.on and warp.revert.off
o5675g0143o 328768  512296  494933  474739  476438  skycell.1790.092    update  ps_ud_MOPS.2    2   warp.revert.on and warp.revert.off
o5675g0143o 328768  512296  494933  474739  476438  skycell.1874.008    update  ps_ud_MOPS.2    2   warp.revert.on and warp.revert.off
o5829g0438o 399488  512334  494975  474778  476477  skycell.1873.083    update  ps_ud_MOPS.2    2   warp.revert.on and warp.revert.off
o5829g0438o 399488  512334  494975  474778  476477  skycell.1873.093    update  ps_ud_MOPS.2    2   warp.revert.on and warp.revert.off
o5328g0050o 165913  512261  494898  477328  479027  skycell.1789.088    update  ps_ud_MOPS.2    2   warp.revert.on and warp.revert.off
o5328g0050o 165913  512261  494898  477328  479027  skycell.1789.089    update  ps_ud_MOPS.2    2   warp.revert.on and warp.revert.off
o5328g0050o 165913  512261  494898  477328  479027  skycell.1789.099    update  ps_ud_MOPS.2    2   warp.revert.on and warp.revert.off

The three of these exposures have the same ids as the chips we just fixed from ps_ud_WEB, so we know these have been processed now and we just need to tidy things up. Since the wap_ids were sent to clean the errors should disappear. DO THEY??! For the remaining problem, let us query:

SELECT chipRun.exp_id,chipRun.chip_id,chipRun.label,chipRun.state, camRun.cam_id,camRun.state, fakeRun.fake_id,fakeRun.state, warpRun.warp_id,warpRun.state FROM chipRun LEFT JOIN camRun USING (chip_id) LEFT JOIN fakeRun USING (cam_id) LEFT JOIN warpRun USING (fake_id) WHERE exp_id=165913;

This one has been reprocessed successfully...multiple times (silly John not realising he had already done this)!

It's just a case of sending that warp_id to be cleaned up:

warptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -warp_id 479027

For exp_id=328768 it's the same treatment, just send the old warp_id (476438) to be cleaned.

Final note! Most things appear cleaned up, however there is now some stuff that I have allowed to reprocess again but it getting stuck in the burntool step. Neb logs say that it cannot find particular nebulous files -> because they are on the ippb machines that are down? Not sure how to confirm.

Tuesday : 2019.02.12

CCL: set nodes down for Haydn: "Last week we replaced a disk in the RAID, which has completed rebuilding, but it still keeps on sending daily complaint messages. In the past, rebooting once has fix this."

neb-host --host ipp127.0 --state down --volume --note "CCL: set down for restart"

set ipp127 up after Haydn reboot it.

neb-host --host ipp127.0 --state up --volume --note "CCL: down-> up after reboot"

CCL: set nodes ippb07-ippb15 down->up, ippb16-ippb23 down->repair

neb-host --host ippb07.0 --state up --volume --note "power back, down->up"
neb-host --host ippb16.0 --state repair --volume --note "power back, down->repair"

NOTE: xattr= 0 "regular machine in ITC", 1 "backup machine", 5 "no longer available"

MEH: with PS1/2 down for a bit, ramping up update processing day+night in ~ippmops/stdscience -- will monitor for issues to determine limiting factors (currently looks like largest is raw data only on ipp067-071,100-104,123-126)

  • adding update.X label to ~ippitc/stdscience with more of a focus on older data that will likely be reprocessed in near future so that the warps don't take up as much space in the long term on nearly full ipp067-0122 (default targeting for ~ippitc/stdscience)

Wednesday : YYYY.MM.DD

Thursday : 2019.02.14

JRF: Big train of thought here based on what I have done to fix some missing PStamp stuff

All of Rob's broken things log:

o5218g0041o o5218g0057o o5327g0047o o5328g0034o o5328g0050o o5362g0570o o5362g0583o o5395g0050o o5476g0223o o5476g0245o o5476g0267o o5476g0289o o5675g0136o o5675g0143o o5829g0438o o5855g0127o o6208g0086o o6208g0102o o6208g0116o o6208g0123o o6208g0130o o6208g0137o o6208g0144o o6208g0151o o6208g0158o o6208g0165o o6208g0172o o6208g0179o o6208g0186o o6208g0193o o6208g0200o o6208g0207o o6209g0131o o6209g0138o o6209g0145o o6209g0152o o6209g0159o o6209g0166o o6209g0173o o6209g0180o o6209g0187o o6209g0194o o6209g0208o o6210g0126o o6210g0133o o6210g0140o o6210g0147o o6210g0154o o6210g0168o o6210g0175o o6210g0182o o6210g0189o o6210g0196o o6210g0210o o6210g0224o o6210g0231o o6210g0238o o6211g0148o o6211g0155o o6211g0162o o6211g0176o o6211g0197o o6211g0204o o6211g0211o o6211g0218o o6212g0144o o6212g0151o o6212g0158o o6212g0165o o6212g0172o o6212g0179o o6212g0186o o6212g0193o o6212g0200o o6212g0207o o6212g0214o o6213g0134o o6213g0156o o6215g0113o o6215g0120o o6215g0127o o6215g0134o o6215g0141o o6215g0148o o6215g0155o o6215g0162o o6215g0169o o6215g0176o o6215g0190o o6216g0129o o6216g0185o o6217g0095o o6217g0111o o6217g0139o o6356g0420o o6356g0438o o6356g0456o o6357g0380o o6401g0326o o6401g0344o o6412g0207o o6427g0160o o6427g0178o o6427g0206o o6430g0247o o6430g0265o o6478g0064o o6481g0080o o6481g0259o o6518g0383o o6571g0132o o6571g0153o o6571g0174o o6571g0195o o6592g0111o o6652g0098o o6674g0401o o6675g0565o o6698g0464o o6698g0485o o6698g0506o o6698g0527o o6708g0516o o6708g0535o o6708g0554o o6708g0573o o6709g0444o o6709g0461o o6727g0192o o7688g0672o o8018g0506o o8292g0262o o8292g0285o o8292g0308o o8292g0331o o8303g0258o o8303g0280o o8303g0302o o8303g0324o

The originals exposure names that gave either an error [25] or [26] from the PStamp server (above)

Out of these the following are now full all the way through:

SELECT exp_name, chipRun.exp_id,chipRun.chip_id,chipRun.label,chipRun.state, camRun.cam_id,camRun.state, fakeRun.fake_id,fakeRun.state, warpRun.warp_id,warpRun.state FROM rawExp LEFT JOIN chipRun USING (exp_id) LEFT JOIN camRun USING (chip_id) LEFT JOIN fakeRun USING (cam_id) LEFT JOIN warpRun USING (fake_id) WHERE exp_name IN ("o5218g0041o","o5218g0057o","o5327g0047o","o5328g0034o","o5328g0050o","o5362g0570o","o5362g0583o","o5395g0050o","o5476g0223o","o5476g0245o","o5476g0267o","o5476g0289o","o5675g0136o","o5675g0143o","o5829g0438o","o5855g0127o","o6208g0086o","o6208g0102o","o6208g0116o","o6208g0123o","o6208g0130o","o6208g0137o","o6208g0144o","o6208g0151o","o6208g0158o","o6208g0165o","o6208g0172o","o6208g0179o","o6208g0186o","o6208g0193o","o6208g0200o","o6208g0207o","o6209g0131o","o6209g0138o","o6209g0145o","o6209g0152o","o6209g0159o","o6209g0166o","o6209g0173o","o6209g0180o","o6209g0187o","o6209g0194o","o6209g0208o","o6210g0126o","o6210g0133o","o6210g0140o","o6210g0147o","o6210g0154o","o6210g0168o","o6210g0175o","o6210g0182o","o6210g0189o","o6210g0196o","o6210g0210o","o6210g0224o","o6210g0231o","o6210g0238o","o6211g0148o","o6211g0155o","o6211g0162o","o6211g0176o","o6211g0197o","o6211g0204o","o6211g0211o","o6211g0218o","o6212g0144o","o6212g0151o","o6212g0158o","o6212g0165o","o6212g0172o","o6212g0179o","o6212g0186o","o6212g0193o","o6212g0200o","o6212g0207o","o6212g0214o","o6213g0134o","o6213g0156o","o6215g0113o","o6215g0120o","o6215g0127o","o6215g0134o","o6215g0141o","o6215g0148o","o6215g0155o","o6215g0162o","o6215g0169o","o6215g0176o","o6215g0190o","o6216g0129o","o6216g0185o","o6217g0095o","o6217g0111o","o6217g0139o","o6356g0420o","o6356g0438o","o6356g0456o","o6357g0380o","o6401g0326o","o6401g0344o","o6412g0207o","o6427g0160o","o6427g0178o","o6427g0206o","o6430g0247o","o6430g0265o","o6478g0064o","o6481g0080o","o6481g0259o","o6518g0383o","o6571g0132o","o6571g0153o","o6571g0174o","o6571g0195o","o6592g0111o","o6652g0098o","o6674g0401o","o6675g0565o","o6698g0464o","o6698g0485o","o6698g0506o","o6698g0527o","o6708g0516o","o6708g0535o","o6708g0554o","o6708g0573o","o6709g0444o","o6709g0461o","o6727g0192o","o7688g0672o","o8018g0506o","o8292g0262o","o8292g0285o","o8292g0308o","o8292g0331o","o8303g0258o","o8303g0280o","o8303g0302o","o8303g0324o") AND chipRun.state="full" AND warpRun.state="full";

+-------------+--------+---------+---------------------+-------+---------+-------+---------+-------+---------+-------+
| exp_name    | exp_id | chip_id | label               | state | cam_id  | state | fake_id | state | warp_id | state |
+-------------+--------+---------+---------------------+-------+---------+-------+---------+-------+---------+-------+
| o5218g0041o | 125344 | 2128159 | PSNSC.wfix.20180525 | full  | 2094629 | full  | 2063638 | full  | 2070292 | full  | 
| o5218g0057o | 125358 | 2128160 | PSNSC.wfix.20180525 | full  | 2094630 | full  | 2063637 | full  | 2070291 | full  | 
| o5327g0047o | 165316 | 2128504 | PSNSC.wfix.20180525 | full  | 2094976 | full  | 2063980 | full  | 2070634 | full  | 
| o5328g0034o | 165897 | 2128485 | PSNSC.wfix.20180525 | full  | 2094953 | full  | 2063957 | full  | 2070611 | full  | 
| o5328g0050o | 165913 | 2128486 | PSNSC.wfix.20180525 | full  | 2094954 | full  | 2063959 | full  | 2070613 | full  | 
| o5328g0050o | 165913 | 2128508 | PSNSC.wfix.20180525 | full  | 2094979 | full  | 2063981 | full  | 2070635 | full  | 
| o5328g0050o | 165913 | 2129013 | PSNSC.wfix.20180525 | full  | 2095485 | full  | 2064491 | full  | 2071145 | full  | 
| o5362g0570o | 181829 | 2128161 | PSNSC.wfix.20180525 | full  | 2094632 | full  | 2063640 | full  | 2070294 | full  | 
| o5362g0583o | 181842 | 2128162 | PSNSC.wfix.20180525 | full  | 2094633 | full  | 2063639 | full  | 2070293 | full  | 
| o5476g0223o | 235917 | 2128164 | PSNSC.wfix.20180525 | full  | 2094635 | full  | 2063641 | full  | 2070295 | full  | 
| o5476g0245o | 235940 | 2128148 | PSNSC.wfix.20180525 | full  | 2094621 | full  | 2063630 | full  | 2070284 | full  | 
| o5476g0267o | 235962 | 2128149 | PSNSC.wfix.20180525 | full  | 2094623 | full  | 2063631 | full  | 2070285 | full  | 
| o5476g0289o | 235984 | 2128163 | PSNSC.wfix.20180525 | full  | 2094636 | full  | 2063644 | full  | 2070298 | full  | 
| o5675g0136o | 328761 | 2128505 | PSNSC.wfix.20180525 | full  | 2094977 | full  | 2063983 | full  | 2070637 | full  | 
| o5675g0136o | 328761 | 2129010 | PSNSC.wfix.20180525 | full  | 2095482 | full  | 2064489 | full  | 2071143 | full  | 
| o5675g0143o | 328768 | 2128506 | PSNSC.wfix.20180525 | full  | 2094980 | full  | 2063984 | full  | 2070638 | full  | 
| o5675g0143o | 328768 | 2129011 | PSNSC.wfix.20180525 | full  | 2095483 | full  | 2064488 | full  | 2071142 | full  | 
| o5829g0438o | 399488 | 2128507 | PSNSC.wfix.20180525 | full  | 2094978 | full  | 2063982 | full  | 2070636 | full  | 
| o5829g0438o | 399488 | 2129012 | PSNSC.wfix.20180525 | full  | 2095484 | full  | 2064490 | full  | 2071144 | full  | 
| o5855g0127o | 410356 | 2128484 | PSNSC.wfix.20180525 | full  | 2094951 | full  | 2063958 | full  | 2070612 | full  | 
+-------------+--------+---------+---------------------+-------+---------+-------+---------+-------+---------+-------+

The following are unique in the above table: o5218g0041o o5218g0057o o5327g0047o o5328g0034o o5328g0050o o5362g0570o o5362g0583o o5476g0223o o5476g0245o o5476g0267o o5476g0289o o5675g0136o o5675g0143o o5829g0438o o5855g0127o

The reason these are ok is that I have done some other reprocessing since Rob's email on the 8th Feb

Which leaves the following to fix:

SELECT exp_name, rawExp.exp_id,rawExp.filter,chipRun.data_group,chipRun.label,chipRun.chip_id,chipRun.state,chipRun.reduction camRun.cam_id,camRun.state, fakeRun.fake_id,fakeRun.state, warpRun.warp_id,warpRun.state FROM rawExp LEFT JOIN chipRun USING (exp_id) LEFT JOIN camRun USING (chip_id) LEFT JOIN fakeRun USING (cam_id) LEFT JOIN warpRun USING (fake_id) WHERE exp_name IN ("o5395g0050o","o6208g0086o","o6208g0102o","o6208g0116o","o6208g0123o","o6208g0130o","o6208g0137o","o6208g0144o","o6208g0151o","o6208g0158o","o6208g0165o","o6208g0172o","o6208g0179o","o6208g0186o","o6208g0193o","o6208g0200o","o6208g0207o","o6209g0131o","o6209g0138o","o6209g0145o","o6209g0152o","o6209g0159o","o6209g0166o","o6209g0173o","o6209g0180o","o6209g0187o","o6209g0194o","o6209g0208o","o6210g0126o","o6210g0133o","o6210g0140o","o6210g0147o","o6210g0154o","o6210g0168o","o6210g0175o","o6210g0182o","o6210g0189o","o6210g0196o","o6210g0210o","o6210g0224o","o6210g0231o","o6210g0238o","o6211g0148o","o6211g0155o","o6211g0162o","o6211g0176o","o6211g0197o","o6211g0204o","o6211g0211o","o6211g0218o","o6212g0144o","o6212g0151o","o6212g0158o","o6212g0165o","o6212g0172o","o6212g0179o","o6212g0186o","o6212g0193o","o6212g0200o","o6212g0207o","o6212g0214o","o6213g0134o","o6213g0156o","o6215g0113o","o6215g0120o","o6215g0127o","o6215g0134o","o6215g0141o","o6215g0148o","o6215g0155o","o6215g0162o","o6215g0169o","o6215g0176o","o6215g0190o","o6216g0129o","o6216g0185o","o6217g0095o","o6217g0111o","o6217g0139o","o6356g0420o","o6356g0438o","o6356g0456o","o6357g0380o","o6401g0326o","o6401g0344o","o6412g0207o","o6427g0160o","o6427g0178o","o6427g0206o","o6430g0247o","o6430g0265o","o6478g0064o","o6481g0080o","o6481g0259o","o6518g0383o","o6571g0132o","o6571g0153o","o6571g0174o","o6571g0195o","o6592g0111o","o6652g0098o","o6674g0401o","o6675g0565o","o6698g0464o","o6698g0485o","o6698g0506o","o6698g0527o","o6708g0516o","o6708g0535o","o6708g0554o","o6708g0573o","o6709g0444o","o6709g0461o","o6727g0192o","o7688g0672o","o8018g0506o","o8292g0262o","o8292g0285o","o8292g0308o","o8292g0331o","o8303g0258o","o8303g0280o","o8303g0302o","o8303g0324o") ORDER BY chipRun.data_group, exp_id;

+-------------+---------+---------+----------------------------------+--------------------+---------+---------+---------+---------+---------+-------+---------+---------+
| exp_name    | exp_id  | filter  | data_group                       | label              | chip_id | state   | cam_id  | state   | fake_id | state | warp_id | state   |
+-------------+---------+---------+----------------------------------+--------------------+---------+---------+---------+---------+---------+-------+---------+---------+
| o6652g0098o |  690914 | w.00000 | ESS.20131226                     | ps_ud_MOPS.2       |  928005 | update  |  898573 | full    |  875613 | full  |  878575 | cleaned | 
| o6727g0192o |  714050 | r.00000 | LAP.PV3.20140730.20140814        | goto_cleaned       | 1046931 | cleaned | 1022111 | full    |  999006 | full  | 1005603 | cleaned | 
| o6727g0192o |  714050 | r.00000 | LAP.PV3.20140730.20140814        | ps_ud_MOPS.2       | 1046972 | update  | 1022152 | full    |  999047 | full  | 1005644 | cleaned | 
| o6357g0380o |  587642 | g.00000 | LAP.PV3.20140730.20140905        | ps_ud_MOPS.2       | 1064321 | update  | 1032056 | full    | 1007358 | full  | 1013944 | update  | 
| o6356g0420o |  587015 | r.00000 | LAP.PV3.20140730.20140907        | ps_ud_MOPS.2       | 1066560 | update  | 1035341 | full    | 1010937 | full  | 1017534 | update  | 
| o6356g0438o |  587033 | i.00000 | LAP.PV3.20140730.20140907        | ps_ud_MOPS.2       | 1067220 | update  | 1035574 | full    | 1017621 | full  | 1024219 | cleaned | 
| o6356g0456o |  587051 | i.00000 | LAP.PV3.20140730.20140907        | ps_ud_MOPS.2       | 1067222 | update  | 1035576 | full    | 1017623 | full  | 1024221 | update  | 
| o6401g0326o |  601772 | r.00000 | LAP.PV3.20140730.20140907        | ps_ud_MOPS.2       | 1066642 | update  | 1035424 | full    | 1011013 | full  | 1017619 | update  | 
| o6401g0344o |  601790 | r.00000 | LAP.PV3.20140730.20140907        | ps_ud_MOPS.2       | 1066643 | update  | 1035425 | full    | 1011014 | full  | 1017620 | update  | 
| o6412g0207o |  607408 | r.00000 | LAP.PV3.20140730.20140907        | ps_ud_MOPS.2       | 1066828 | update  | 1035207 | full    | 1011362 | full  | 1017975 | update  | 
| o6698g0464o |  705016 | i.00000 | LAP.PV3.20140730.20140910        | ps_ud_MOPS.2       | 1078806 | update  | 1040652 | full    | 1016261 | full  | 1022862 | cleaned | 
| o6698g0485o |  705035 | i.00000 | LAP.PV3.20140730.20140910        | ps_ud_MOPS.2       | 1078809 | update  | 1040655 | full    | 1016264 | full  | 1022865 | cleaned | 
| o6698g0506o |  705057 | i.00000 | LAP.PV3.20140730.20140910        | ps_ud_MOPS.2       | 1078812 | update  | 1040658 | full    | 1016267 | full  | 1022868 | cleaned | 
| o6698g0527o |  705079 | i.00000 | LAP.PV3.20140730.20140910        | ps_ud_MOPS.2       | 1078815 | update  | 1040661 | full    | 1016270 | full  | 1022871 | update  | 
| o6708g0516o |  706839 | i.00000 | LAP.PV3.20140730.20140916        | ps_ud_MOPS.2       | 1086240 | update  | 1049677 | full    | 1025381 | full  | 1032207 | cleaned | 
| o6708g0535o |  706859 | i.00000 | LAP.PV3.20140730.20140916        | ps_ud_MOPS.2       | 1086242 | update  | 1049679 | full    | 1025383 | full  | 1032209 | cleaned | 
| o6708g0554o |  706877 | i.00000 | LAP.PV3.20140730.20140916        | ps_ud_MOPS.2       | 1086244 | update  | 1049682 | full    | 1025385 | full  | 1032211 | cleaned | 
| o6708g0573o |  706896 | i.00000 | LAP.PV3.20140730.20140916        | ps_ud_MOPS.2       | 1086246 | update  | 1049684 | full    | 1025387 | full  | 1032214 | cleaned | 
| o6709g0444o |  707443 | r.00000 | LAP.PV3.20140730.20140916        | ps_ud_MOPS.2       | 1085607 | update  | 1049094 | full    | 1024809 | full  | 1031564 | cleaned | 
| o6709g0461o |  707460 | r.00000 | LAP.PV3.20140730.20140916        | ps_ud_MOPS.2       | 1085610 | update  | 1049108 | full    | 1024821 | full  | 1031579 | cleaned | 
| o6427g0160o |  611797 | g.00000 | LAP.PV3.20140730.20140928        | ps_ud_MOPS.2       | 1099942 | update  | 1072061 | full    | 1047789 | full  | 1055086 | update  | 
| o6427g0178o |  611816 | g.00000 | LAP.PV3.20140730.20140928        | ps_ud_MOPS.2       | 1099944 | update  | 1072063 | full    | 1047791 | full  | 1055088 | update  | 
| o6427g0206o |  611845 | g.00000 | LAP.PV3.20140730.20140928        | ps_ud_MOPS.2       | 1099945 | update  | 1072064 | full    | 1047792 | full  | 1055089 | update  | 
| o6430g0247o |  613048 | i.00000 | LAP.PV3.20140730.20140929        | ps_ud_MOPS.2       | 1101267 | update  | 1070261 | full    | 1046292 | full  | 1053484 | cleaned | 
| o6430g0265o |  613073 | i.00000 | LAP.PV3.20140730.20140929        | ps_ud_MOPS.2       | 1101269 | update  | 1070263 | full    | 1046294 | full  | 1053486 | cleaned | 
| o6592g0111o |  670085 | z.00000 | LAP.PV3.20140730.20141027        | ps_ud_MOPS.2       | 1188757 | update  | 1157133 | full    | 1132784 | full  | 1139396 | update  | 
| o5395g0050o |  192649 | g.00000 | LAP.PV3.20140730.bigmem.20150119 | goto_cleaned       | 1425014 | cleaned | 1392244 | full    |    NULL | NULL  |    NULL | NULL    | 
| o6675g0565o |  698412 | z.00000 | LAP.PV3.20140730.ipp.20140909    | ps_ud_MOPS.2       | 1077612 | update  | 1063523 | full    | 1039126 | full  | 1045726 | cleaned | 
| o6674g0401o |  697666 | r.00000 | LAP.PV3.20140730.ipp.20140926    | ps_ud_MOPS.2       | 1095567 | update  | 1063850 | full    | 1039452 | full  | 1046052 | cleaned | 
| o6478g0064o |  629141 | y.00000 | LAP.PV3.20140730.ipp.20141002    | ps_ud_MOPS.2       | 1110944 | update  | 1079084 | full    | 1054557 | full  | 1061157 | update  | 
| o6481g0080o |  630036 | z.00000 | LAP.PV3.20140730.ipp.20141002    | ps_ud_MOPS.2       | 1111169 | update  | 1079750 | full    | 1055392 | full  | 1061785 | cleaned | 
| o5395g0050o |  192649 | g.00000 | LAP.ThreePi.20110809.20110827    | goto_cleaned.rerun |  277048 | cleaned |  254344 | full    |  238616 | full  |  241705 | cleaned | 
| o5395g0050o |  192649 | g.00000 | LAP.ThreePi.20110809.20110831    | goto_cleaned.rerun |  279457 | cleaned |  256752 | full    |  240876 | full  |  243966 | cleaned | 
| o5395g0050o |  192649 | g.00000 | LAP.ThreePi.20110809.20120214    | goto_cleaned.rerun |  408025 | cleaned |  385273 | full    |  366377 | full  |  368072 | cleaned | 
| o5395g0050o |  192649 | g.00000 | LAP.ThreePi.20120706.20120919    | goto_cleaned       |  597363 | cleaned |  574172 | cleaned |  553507 | full  |  555206 | cleaned | 
| o6592g0111o |  670085 | z.00000 | LAP.ThreePi.20130717.20140407    | goto_cleaned       |  966291 | cleaned |  935385 | cleaned |  912238 | full  |  915190 | cleaned | 
| o6356g0420o |  587015 | r.00000 | MOPSreq.czw.20130312             | goto_cleaned_redo  |  797515 | cleaned |  774380 | full    |  752432 | full  |  754136 | cleaned | 
| o6356g0438o |  587033 | i.00000 | MOPSreq.czw.20130312             | goto_cleaned_redo  |  797582 | cleaned |  774554 | full    |  752608 | full  |  754311 | cleaned | 
| o6356g0456o |  587051 | i.00000 | MOPSreq.czw.20130312             | goto_cleaned_redo  |  797583 | cleaned |  774557 | full    |  752613 | full  |  754313 | cleaned | 
| o6356g0420o |  587015 | r.00000 | MOPSreq.czw.20130313             | goto_cleaned       |  798890 | cleaned |  775634 | full    |  753680 | full  |  755391 | cleaned | 
| o6356g0438o |  587033 | i.00000 | MOPSreq.czw.20130313             | goto_cleaned       |  798957 | cleaned |  775704 | full    |  753762 | full  |  755473 | cleaned | 
| o6356g0456o |  587051 | i.00000 | MOPSreq.czw.20130313             | goto_cleaned       |  798958 | cleaned |  775706 | full    |  753751 | full  |  755462 | cleaned | 
| o6481g0259o |  630216 | w.00000 | OSS.20130708                     | ps_ud_MOPS.2       |  856071 | update  |  832046 | full    |  809646 | full  |  812627 | update  | 
| o6518g0383o |  642778 | w.00000 | OSS.20130814                     | ps_ud_MOPS.2       |  873301 | update  |  846259 | full    |  823811 | full  |  826792 | update  | 
| o6571g0153o |  663086 | w.00000 | OSS.20131006                     | ps_ud_MOPS.2       |  894769 | update  |  865425 | full    |  842950 | full  |  845917 | update  | 
| o6571g0174o |  663087 | w.00000 | OSS.20131006                     | ps_ud_MOPS.2       |  894791 | update  |  865447 | full    |  842971 | full  |  845938 | cleaned | 
| o6571g0132o |  663102 | w.00000 | OSS.20131006                     | ps_ud_MOPS.2       |  894748 | update  |  865408 | full    |  842929 | full  |  845896 | update  | 
| o6571g0195o |  663123 | w.00000 | OSS.20131006                     | ps_ud_MOPS.2       |  894817 | update  |  865473 | full    |  843000 | full  |  845967 | cleaned | 
| o6698g0464o |  705016 | i.00000 | OSS.20140210                     | goto_cleaned       |  946396 | cleaned |  915854 | full    |  892798 | full  |  895760 | cleaned | 
| o6698g0485o |  705035 | i.00000 | OSS.20140210                     | goto_cleaned       |  946420 | cleaned |  915875 | full    |  892824 | full  |  895786 | cleaned | 
| o6698g0506o |  705057 | i.00000 | OSS.20140210                     | goto_cleaned       |  946441 | cleaned |  915893 | full    |  892849 | full  |  895811 | cleaned | 
| o6698g0527o |  705079 | i.00000 | OSS.20140210                     | goto_cleaned       |  946462 | cleaned |  915916 | full    |  892864 | full  |  895826 | cleaned | 
| o6708g0516o |  706839 | i.00000 | OSS.20140220                     | goto_cleaned       |  949093 | cleaned |  917752 | full    |  894698 | full  |  897660 | cleaned | 
| o6708g0535o |  706859 | i.00000 | OSS.20140220                     | goto_cleaned       |  949112 | cleaned |  917771 | full    |  894717 | full  |  897679 | cleaned | 
| o6708g0554o |  706877 | i.00000 | OSS.20140220                     | goto_cleaned       |  949131 | cleaned |  917789 | full    |  894733 | full  |  897695 | cleaned | 
| o6708g0573o |  706896 | i.00000 | OSS.20140220                     | goto_cleaned       |  949150 | cleaned |  917809 | full    |  894751 | full  |  897713 | cleaned | 
| o8018g0506o | 1306007 | w.00000 | OSS.20170922                     | ps_ud_MOPS.2       | 1996895 | update  | 1963370 | full    | 1933998 | full  | 1940599 | update  | 
| o8292g0262o | 1403156 | i.00000 | OSS.20180623                     | goto_cleaned       | 2086098 | cleaned | 2052571 | full    | 2021884 | full  | 2028485 | full    | 
| o8292g0285o | 1403179 | i.00000 | OSS.20180623                     | goto_cleaned       | 2086121 | cleaned | 2052594 | full    | 2021908 | full  | 2028509 | full    | 
| o8292g0308o | 1403202 | i.00000 | OSS.20180623                     | goto_cleaned       | 2086144 | cleaned | 2052617 | full    | 2021931 | full  | 2028532 | full    | 
| o8292g0331o | 1403225 | i.00000 | OSS.20180623                     | goto_cleaned       | 2086167 | cleaned | 2052640 | full    | 2021951 | full  | 2028552 | full    | 
| o8303g0258o | 1406840 | i.00000 | OSS.20180704                     | goto_cleaned       | 2088933 | cleaned | 2055407 | full    | 2024712 | full  | 2031313 | full    | 
| o8303g0280o | 1406862 | i.00000 | OSS.20180704                     | goto_cleaned       | 2088955 | cleaned | 2055435 | full    | 2024736 | full  | 2031337 | full    | 
| o8303g0302o | 1406884 | i.00000 | OSS.20180704                     | goto_cleaned       | 2088977 | cleaned | 2055450 | full    | 2024751 | full  | 2031352 | full    | 
| o8303g0324o | 1406906 | i.00000 | OSS.20180704                     | goto_cleaned       | 2088999 | cleaned | 2055472 | full    | 2024775 | full  | 2031376 | full    | 
| o6208g0086o |  527811 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628582 | cleaned |  604414 | full    |  583500 | full  |  585199 | cleaned | 
| o6208g0102o |  527827 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628598 | cleaned |  604437 | full    |  583525 | full  |  585224 | cleaned | 
| o6208g0116o |  527841 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628612 | cleaned |  604454 | full    |  583537 | full  |  585236 | cleaned | 
| o6208g0123o |  527848 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628619 | cleaned |  604467 | full    |  583558 | full  |  585257 | cleaned | 
| o6208g0130o |  527855 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628626 | cleaned |  604464 | full    |  583548 | full  |  585247 | cleaned | 
| o6208g0137o |  527862 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628635 | cleaned |  604480 | full    |  583559 | full  |  585258 | cleaned | 
| o6208g0144o |  527869 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628750 | cleaned |  604486 | full    |  583579 | full  |  585278 | cleaned | 
| o6208g0151o |  527876 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628781 | cleaned |  604491 | full    |  583574 | full  |  585273 | cleaned | 
| o6208g0158o |  527883 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628840 | cleaned |  604500 | full    |  583586 | full  |  585285 | cleaned | 
| o6208g0165o |  527890 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628863 | cleaned |  604513 | full    |  583597 | full  |  585296 | cleaned | 
| o6208g0172o |  527897 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628870 | cleaned |  604521 | full    |  583608 | full  |  585307 | cleaned | 
| o6208g0179o |  527904 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628877 | cleaned |  604524 | full    |  583616 | full  |  585315 | cleaned | 
| o6208g0186o |  527911 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628884 | cleaned |  604534 | full    |  583620 | full  |  585319 | cleaned | 
| o6208g0193o |  527918 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628891 | cleaned |  604536 | full    |  583623 | full  |  585322 | cleaned | 
| o6208g0200o |  527925 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628898 | cleaned |  604555 | full    |  583659 | full  |  585358 | cleaned | 
| o6208g0207o |  527932 | i.00000 | STS.20121008                     | goto_cleaned_redo  |  628905 | cleaned |  604550 | full    |  583654 | full  |  585353 | cleaned | 
| o6209g0131o |  528511 | i.00000 | STS.20121009                     | goto_cleaned_redo  |  630583 | cleaned |  606259 | full    |  585344 | full  |  587043 | cleaned | 
| o6209g0138o |  528517 | i.00000 | STS.20121009                     | goto_cleaned_redo  |  630590 | cleaned |  606277 | full    |  585364 | full  |  587063 | cleaned | 
| o6209g0145o |  528524 | i.00000 | STS.20121009                     | goto_cleaned_redo  |  630597 | cleaned |  606289 | full    |  585379 | full  |  587078 | cleaned | 
| o6209g0152o |  528531 | i.00000 | STS.20121009                     | goto_cleaned_redo  |  630604 | cleaned |  606299 | full    |  585383 | full  |  587082 | cleaned | 
| o6209g0159o |  528538 | i.00000 | STS.20121009                     | goto_cleaned_redo  |  630611 | cleaned |  606318 | full    |  585400 | full  |  587099 | cleaned | 
| o6209g0166o |  528545 | i.00000 | STS.20121009                     | goto_cleaned_redo  |  630618 | cleaned |  606333 | full    |  585414 | full  |  587113 | cleaned | 
| o6209g0173o |  528552 | i.00000 | STS.20121009                     | goto_cleaned_redo  |  630715 | cleaned |  606349 | full    |  585427 | full  |  587126 | cleaned | 
| o6209g0180o |  528559 | i.00000 | STS.20121009                     | goto_cleaned_redo  |  630742 | cleaned |  606362 | full    |  585440 | full  |  587139 | cleaned | 
| o6209g0187o |  528566 | i.00000 | STS.20121009                     | goto_cleaned_redo  |  630749 | cleaned |  606364 | full    |  585446 | full  |  587145 | cleaned | 
| o6209g0194o |  528573 | i.00000 | STS.20121009                     | goto_cleaned_redo  |  630756 | cleaned |  606374 | full    |  585459 | full  |  587158 | cleaned | 
| o6209g0208o |  528587 | i.00000 | STS.20121009                     | goto_cleaned_redo  |  630770 | cleaned |  606397 | full    |  585503 | full  |  587202 | cleaned | 
| o6210g0126o |  529194 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632592 | cleaned |  608412 | full    |  587482 | full  |  589181 | cleaned | 
| o6210g0133o |  529201 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632599 | cleaned |  608419 | full    |  587489 | full  |  589188 | cleaned | 
| o6210g0140o |  529208 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632606 | cleaned |  608428 | full    |  587501 | full  |  589200 | cleaned | 
| o6210g0147o |  529215 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632692 | cleaned |  608445 | full    |  587518 | full  |  589217 | cleaned | 
| o6210g0154o |  529222 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632752 | cleaned |  608452 | full    |  587547 | full  |  589246 | cleaned | 
| o6210g0175o |  529242 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632774 | cleaned |  608518 | full    |  587584 | full  |  589283 | cleaned | 
| o6210g0182o |  529249 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632782 | cleaned |  608524 | full    |  587602 | full  |  589301 | cleaned | 
| o6210g0189o |  529256 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632789 | cleaned |  608531 | full    |  587603 | full  |  589302 | cleaned | 
| o6210g0168o |  529258 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632768 | cleaned |  608512 | full    |  587579 | full  |  589278 | cleaned | 
| o6210g0196o |  529264 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632796 | cleaned |  608547 | full    |  587612 | full  |  589311 | cleaned | 
| o6210g0210o |  529278 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632810 | cleaned |  608554 | full    |  587614 | full  |  589313 | cleaned | 
| o6210g0224o |  529292 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632824 | cleaned |  608572 | full    |  587640 | full  |  589339 | cleaned | 
| o6210g0231o |  529299 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632859 | cleaned |  608573 | full    |  587645 | full  |  589344 | cleaned | 
| o6210g0238o |  529306 | i.00000 | STS.20121010                     | goto_cleaned_redo  |  632866 | cleaned |  608589 | full    |  587677 | full  |  589376 | cleaned | 
| o6211g0148o |  529898 | i.00000 | STS.20121011                     | goto_cleaned_redo  |  635853 | cleaned |  610549 | full    |  589610 | full  |  591309 | cleaned | 
| o6211g0155o |  529905 | i.00000 | STS.20121011                     | goto_cleaned_redo  |  635860 | cleaned |  610558 | full    |  589621 | full  |  591320 | cleaned | 
| o6211g0162o |  529912 | i.00000 | STS.20121011                     | goto_cleaned_redo  |  635867 | cleaned |  610571 | full    |  589636 | full  |  591335 | cleaned | 
| o6211g0176o |  529926 | i.00000 | STS.20121011                     | goto_cleaned_redo  |  635881 | cleaned |  610590 | full    |  589661 | full  |  591360 | cleaned | 
| o6211g0197o |  529947 | i.00000 | STS.20121011                     | goto_cleaned_redo  |  635902 | cleaned |  610635 | full    |  589692 | full  |  591391 | cleaned | 
| o6211g0204o |  529954 | i.00000 | STS.20121011                     | goto_cleaned_redo  |  635946 | cleaned |  610647 | full    |  589703 | full  |  591402 | cleaned | 
| o6211g0211o |  529961 | i.00000 | STS.20121011                     | goto_cleaned_redo  |  635953 | cleaned |  610649 | full    |  589704 | full  |  591403 | cleaned | 
| o6211g0218o |  529968 | i.00000 | STS.20121011                     | goto_cleaned_redo  |  635960 | cleaned |  610665 | full    |  589735 | full  |  591434 | cleaned | 
| o6212g0144o |  530557 | i.00000 | STS.20121012                     | goto_cleaned_redo  |  637216 | cleaned |  612788 | full    |  591828 | full  |  593527 | cleaned | 
| o6212g0151o |  530564 | i.00000 | STS.20121012                     | goto_cleaned_redo  |  637223 | cleaned |  612796 | full    |  591837 | full  |  593536 | cleaned | 
| o6212g0158o |  530572 | i.00000 | STS.20121012                     | goto_cleaned_redo  |  637230 | cleaned |  612797 | full    |  591841 | full  |  593540 | cleaned | 
| o6212g0165o |  530579 | i.00000 | STS.20121012                     | goto_cleaned_redo  |  637237 | cleaned |  612808 | full    |  591846 | full  |  593545 | cleaned | 
| o6212g0172o |  530586 | i.00000 | STS.20121012                     | goto_cleaned_redo  |  637244 | cleaned |  612814 | full    |  591848 | full  |  593547 | cleaned | 
| o6212g0179o |  530593 | i.00000 | STS.20121012                     | goto_cleaned_redo  |  637251 | cleaned |  612823 | full    |  591857 | full  |  593556 | cleaned | 
| o6212g0186o |  530600 | i.00000 | STS.20121012                     | goto_cleaned_redo  |  637258 | cleaned |  612826 | full    |  591867 | full  |  593566 | cleaned | 
| o6212g0193o |  530607 | i.00000 | STS.20121012                     | goto_cleaned_redo  |  637309 | cleaned |  612835 | full    |  591868 | full  |  593567 | cleaned | 
| o6212g0200o |  530614 | i.00000 | STS.20121012                     | goto_cleaned_redo  |  637358 | cleaned |  612839 | full    |  591878 | full  |  593577 | cleaned | 
| o6212g0207o |  530621 | i.00000 | STS.20121012                     | goto_cleaned_redo  |  637429 | cleaned |  612848 | full    |  591885 | full  |  593584 | cleaned | 
| o6212g0214o |  530628 | i.00000 | STS.20121012                     | goto_cleaned_redo  |  637436 | cleaned |  612860 | full    |  591895 | full  |  593594 | cleaned | 
| o6213g0134o |  530795 | i.00000 | STS.20121013                     | goto_cleaned_redo  |  639330 | cleaned |  615337 | full    |  594357 | full  |  596056 | cleaned | 
| o6213g0156o |  530817 | i.00000 | STS.20121013                     | goto_cleaned_redo  |  639352 | cleaned |  615368 | full    |  594398 | full  |  596097 | cleaned | 
| o6215g0120o |  531760 | i.00000 | STS.20121015                     | goto_cleaned_redo  |  643581 | cleaned |  619532 | full    |  598500 | full  |  600199 | cleaned | 
| o6215g0127o |  531767 | i.00000 | STS.20121015                     | goto_cleaned_redo  |  643588 | cleaned |  619537 | full    |  598504 | full  |  600203 | cleaned | 
| o6215g0134o |  531774 | i.00000 | STS.20121015                     | goto_cleaned_redo  |  643621 | cleaned |  619547 | full    |  598515 | full  |  600214 | cleaned | 
| o6215g0141o |  531781 | i.00000 | STS.20121015                     | goto_cleaned_redo  |  643639 | cleaned |  619552 | full    |  598522 | full  |  600221 | cleaned | 
| o6215g0148o |  531788 | i.00000 | STS.20121015                     | goto_cleaned_redo  |  643661 | cleaned |  619560 | full    |  598526 | full  |  600225 | cleaned | 
| o6215g0113o |  531795 | i.00000 | STS.20121015                     | goto_cleaned_redo  |  643528 | cleaned |  619524 | full    |  598492 | full  |  600191 | cleaned | 
| o6215g0155o |  531796 | i.00000 | STS.20121015                     | goto_cleaned_redo  |  643668 | cleaned |  619574 | full    |  598538 | full  |  600237 | cleaned | 
| o6215g0162o |  531803 | i.00000 | STS.20121015                     | goto_cleaned_redo  |  643675 | cleaned |  619570 | full    |  598534 | full  |  600233 | cleaned | 
| o6215g0169o |  531810 | i.00000 | STS.20121015                     | goto_cleaned_redo  |  643682 | cleaned |  619586 | full    |  598550 | full  |  600249 | cleaned | 
| o6215g0176o |  531817 | i.00000 | STS.20121015                     | goto_cleaned_redo  |  643689 | cleaned |  619582 | full    |  598551 | full  |  600250 | cleaned | 
| o6215g0190o |  531830 | i.00000 | STS.20121015                     | goto_cleaned_redo  |  643703 | cleaned |  619598 | full    |  598566 | full  |  600265 | cleaned | 
| o6216g0129o |  532424 | i.00000 | STS.20121016                     | goto_cleaned_redo  |  645267 | cleaned |  621782 | full    |  600730 | full  |  602429 | cleaned | 
| o6216g0185o |  532480 | i.00000 | STS.20121016                     | goto_cleaned_redo  |  645323 | cleaned |  621848 | full    |  600793 | full  |  602492 | cleaned | 
| o6217g0095o |  532891 | i.00000 | STS.20121017                     | goto_cleaned_redo  |  646575 | cleaned |  623336 | full    |  602287 | full  |  603986 | cleaned | 
| o6217g0111o |  532906 | i.00000 | STS.20121017                     | goto_cleaned_redo  |  646591 | cleaned |  623350 | full    |  602304 | full  |  604003 | cleaned | 
| o6217g0139o |  532935 | i.00000 | STS.20121017                     | goto_cleaned_redo  |  646619 | cleaned |  623383 | full    |  602330 | full  |  604029 | cleaned | 
| o6208g0102o |  527827 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715539 | cleaned |  693656 | full    |  672510 | full  |  674220 | cleaned | 
| o6208g0116o |  527841 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715548 | cleaned |  693663 | full    |  672517 | full  |  674227 | cleaned | 
| o6208g0123o |  527848 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715555 | cleaned |  693665 | full    |  672516 | full  |  674226 | cleaned | 
| o6208g0130o |  527855 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715562 | cleaned |  693678 | full    |  672530 | full  |  674240 | cleaned | 
| o6208g0137o |  527862 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715569 | cleaned |  693685 | full    |  672529 | full  |  674239 | cleaned | 
| o6208g0144o |  527869 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715576 | cleaned |  693687 | full    |  672541 | full  |  674251 | cleaned | 
| o6208g0151o |  527876 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715583 | cleaned |  693696 | full    |  672551 | full  |  674261 | cleaned | 
| o6208g0158o |  527883 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715590 | cleaned |  693865 | full    |  672572 | full  |  674282 | cleaned | 
| o6208g0165o |  527890 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715597 | cleaned |  693868 | full    |  672582 | full  |  674292 | cleaned | 
| o6208g0172o |  527897 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715604 | cleaned |  693872 | full    |  672587 | full  |  674297 | cleaned | 
| o6208g0179o |  527904 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715611 | cleaned |  693881 | full    |  672606 | full  |  674727 | cleaned | 
| o6208g0186o |  527911 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715618 | cleaned |  693888 | full    |  672597 | full  |  674718 | cleaned | 
| o6208g0193o |  527918 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715625 | cleaned |  693895 | full    |  672615 | full  |  674736 | cleaned | 
| o6208g0200o |  527925 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715632 | cleaned |  693903 | full    |  672608 | full  |  674729 | cleaned | 
| o6208g0207o |  527932 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715639 | cleaned |  694351 | full    |  673048 | full  |  674758 | cleaned | 
| o6212g0144o |  530557 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715646 | cleaned |  694357 | full    |  673083 | full  |  674793 | cleaned | 
| o6212g0151o |  530564 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715653 | cleaned |  699950 | full    |  678657 | full  |  680367 | cleaned | 
| o6212g0158o |  530572 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715660 | cleaned |  699957 | full    |  678655 | full  |  680365 | cleaned | 
| o6212g0165o |  530579 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715667 | cleaned |  699965 | full    |  678669 | full  |  680379 | cleaned | 
| o6212g0172o |  530586 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715674 | cleaned |  699971 | full    |  678668 | full  |  680378 | cleaned | 
| o6212g0179o |  530593 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715681 | cleaned |  699986 | full    |  678687 | full  |  680397 | cleaned | 
| o6212g0186o |  530600 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715688 | cleaned |  699981 | full    |  678699 | full  |  680409 | cleaned | 
| o6212g0193o |  530607 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715695 | cleaned |  699991 | full    |  678690 | full  |  680400 | cleaned | 
| o6212g0200o |  530614 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715702 | cleaned |  699998 | full    |  678704 | full  |  680414 | cleaned | 
| o6212g0207o |  530621 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715709 | cleaned |  700003 | full    |  678708 | full  |  680418 | cleaned | 
| o6212g0214o |  530628 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715716 | cleaned |  700010 | full    |  678709 | full  |  680419 | cleaned | 
| o6213g0134o |  530795 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715743 | cleaned |  700043 | full    |  678742 | full  |  680452 | cleaned | 
| o6213g0156o |  530817 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715765 | cleaned |  700058 | full    |  678748 | full  |  680458 | cleaned | 
| o6215g0120o |  531760 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715782 | cleaned |  700078 | full    |  678769 | full  |  680479 | cleaned | 
| o6215g0127o |  531767 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715789 | cleaned |  700090 | full    |  678782 | full  |  680492 | cleaned | 
| o6215g0134o |  531774 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715796 | cleaned |  700088 | full    |  678781 | full  |  680491 | cleaned | 
| o6215g0141o |  531781 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715803 | cleaned |  700104 | full    |  678795 | full  |  680505 | cleaned | 
| o6215g0148o |  531788 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715810 | cleaned |  700110 | full    |  678799 | full  |  680509 | cleaned | 
| o6215g0113o |  531795 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715817 | cleaned |  700111 | full    |  678802 | full  |  680512 | cleaned | 
| o6215g0155o |  531796 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715818 | cleaned |  700117 | full    |  678808 | full  |  680518 | cleaned | 
| o6215g0162o |  531803 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715825 | cleaned |  700119 | full    |  678807 | full  |  680517 | cleaned | 
| o6215g0169o |  531810 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715832 | cleaned |  700122 | full    |  678819 | full  |  680529 | cleaned | 
| o6215g0176o |  531817 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715839 | cleaned |  700138 | full    |  678833 | full  |  680543 | cleaned | 
| o6215g0190o |  531830 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715852 | cleaned |  700153 | full    |  678844 | full  |  680554 | cleaned | 
| o6216g0129o |  532424 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715881 | cleaned |  700175 | full    |  678863 | full  |  680573 | cleaned | 
| o6216g0185o |  532480 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715937 | cleaned |  700223 | full    |  678911 | full  |  680621 | cleaned | 
| o6217g0095o |  532891 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715942 | cleaned |  700226 | full    |  678921 | full  |  680631 | cleaned | 
| o6217g0111o |  532906 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715957 | cleaned |  700249 | full    |  678939 | full  |  680649 | cleaned | 
| o6217g0139o |  532935 | i.00000 | STS.rerun.20121129               | goto_cleaned_redo  |  715986 | cleaned |  700287 | full    |  678977 | full  |  680687 | cleaned | 
| o6208g0086o |  527811 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900824 | cleaned |  871443 | full    |  849004 | full  |  851971 | cleaned | 
| o6208g0102o |  527827 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900838 | cleaned |  871489 | full    |  848988 | full  |  851955 | cleaned | 
| o6208g0116o |  527841 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900850 | cleaned |  871482 | full    |  848982 | full  |  851949 | cleaned | 
| o6208g0123o |  527848 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900856 | cleaned |  871475 | full    |  848973 | full  |  851940 | cleaned | 
| o6208g0130o |  527855 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900862 | cleaned |  871508 | full    |  849012 | full  |  851979 | cleaned | 
| o6208g0137o |  527862 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900868 | cleaned |  871510 | full    |  849020 | full  |  851987 | cleaned | 
| o6208g0144o |  527869 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900874 | cleaned |  871550 | full    |  849049 | full  |  852016 | cleaned | 
| o6208g0151o |  527876 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900880 | cleaned |  871507 | full    |  849011 | full  |  851978 | cleaned | 
| o6208g0158o |  527883 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900886 | cleaned |  871496 | full    |  848992 | full  |  851959 | cleaned | 
| o6208g0165o |  527890 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900892 | cleaned |  871534 | full    |  849053 | full  |  852020 | cleaned | 
| o6208g0172o |  527897 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900898 | cleaned |  871545 | full    |  849055 | full  |  852022 | cleaned | 
| o6208g0179o |  527904 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900904 | cleaned |  871549 | full    |  849044 | full  |  852011 | cleaned | 
| o6208g0186o |  527911 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900910 | cleaned |  871520 | full    |  849016 | full  |  851983 | cleaned | 
| o6208g0193o |  527918 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900916 | cleaned |  871552 | full    |  849043 | full  |  852010 | cleaned | 
| o6208g0200o |  527925 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900922 | cleaned |  871546 | full    |  849041 | full  |  852008 | cleaned | 
| o6208g0207o |  527932 | i.00000 | STS.rp.2013.20121008             | STS.rp.2013        |  900928 | cleaned |  871572 | full    |  849146 | full  |  852113 | cleaned | 
| o6209g0131o |  528511 | i.00000 | STS.rp.2013.20121009             | STS.rp.2013        |  900935 | cleaned |  871562 | full    |  849061 | full  |  852028 | cleaned | 
| o6209g0138o |  528517 | i.00000 | STS.rp.2013.20121009             | STS.rp.2013        |  900940 | cleaned |  871579 | full    |  849071 | full  |  852038 | cleaned | 
| o6209g0145o |  528524 | i.00000 | STS.rp.2013.20121009             | STS.rp.2013        |  900946 | cleaned |  871570 | full    |  849065 | full  |  852032 | cleaned | 
| o6209g0152o |  528531 | i.00000 | STS.rp.2013.20121009             | STS.rp.2013        |  900952 | cleaned |  871581 | full    |  849099 | full  |  852066 | cleaned | 
| o6209g0159o |  528538 | i.00000 | STS.rp.2013.20121009             | STS.rp.2013        |  900958 | cleaned |  871611 | full    |  849132 | full  |  852099 | cleaned | 
| o6209g0166o |  528545 | i.00000 | STS.rp.2013.20121009             | STS.rp.2013        |  900964 | cleaned |  871600 | full    |  849110 | full  |  852077 | cleaned | 
| o6209g0173o |  528552 | i.00000 | STS.rp.2013.20121009             | STS.rp.2013        |  900970 | cleaned |  871598 | full    |  849090 | full  |  852057 | cleaned | 
| o6209g0180o |  528559 | i.00000 | STS.rp.2013.20121009             | STS.rp.2013        |  900976 | cleaned |  871604 | full    |  849116 | full  |  852083 | cleaned | 
| o6209g0187o |  528566 | i.00000 | STS.rp.2013.20121009             | STS.rp.2013        |  900982 | cleaned |  871636 | full    |  849204 | full  |  852171 | cleaned | 
| o6209g0194o |  528573 | i.00000 | STS.rp.2013.20121009             | STS.rp.2013        |  900988 | cleaned |  871643 | full    |  849207 | full  |  852174 | cleaned | 
| o6209g0208o |  528587 | i.00000 | STS.rp.2013.20121009             | STS.rp.2013        |  901000 | cleaned |  871626 | full    |  849130 | full  |  852097 | cleaned | 
| o6210g0126o |  529194 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901006 | cleaned |  871638 | full    |  849179 | full  |  852146 | cleaned | 
| o6210g0133o |  529201 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901012 | cleaned |  871640 | full    |  849184 | full  |  852151 | cleaned | 
| o6210g0140o |  529208 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901018 | cleaned |  871669 | full    |  849240 | full  |  852207 | cleaned | 
| o6210g0147o |  529215 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901024 | cleaned |  871668 | full    |  849258 | full  |  852225 | cleaned | 
| o6210g0154o |  529222 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901030 | cleaned |  871653 | full    |  849224 | full  |  852191 | cleaned | 
| o6210g0175o |  529242 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901047 | cleaned |  871672 | full    |  849259 | full  |  852226 | cleaned | 
| o6210g0182o |  529249 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901053 | cleaned |  871677 | full    |  849312 | full  |  852279 | cleaned | 
| o6210g0189o |  529256 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901059 | cleaned |  871707 | full    |  849291 | full  |  852258 | cleaned | 
| o6210g0168o |  529258 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901061 | cleaned |  871711 | full    |  849300 | full  |  852267 | cleaned | 
| o6210g0196o |  529264 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901066 | cleaned |  871751 | full    |  849322 | full  |  852289 | cleaned | 
| o6210g0210o |  529278 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901078 | cleaned |  871749 | full    |  849307 | full  |  852274 | cleaned | 
| o6210g0224o |  529292 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901090 | cleaned |  871769 | full    |  849319 | full  |  852286 | cleaned | 
| o6210g0231o |  529299 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901096 | cleaned |  871777 | full    |  849325 | full  |  852292 | cleaned | 
| o6210g0238o |  529306 | i.00000 | STS.rp.2013.20121010             | STS.rp.2013        |  901102 | cleaned |  871832 | full    |  849349 | full  |  852315 | cleaned | 
| o6211g0148o |  529898 | i.00000 | STS.rp.2013.20121011             | STS.rp.2013        |  901116 | cleaned |  871836 | full    |  849363 | full  |  852330 | cleaned | 
| o6211g0155o |  529905 | i.00000 | STS.rp.2013.20121011             | STS.rp.2013        |  901122 | cleaned |  871865 | full    |  849373 | full  |  852340 | cleaned | 
| o6211g0162o |  529912 | i.00000 | STS.rp.2013.20121011             | STS.rp.2013        |  901128 | cleaned |  871868 | full    |  849372 | full  |  852339 | cleaned | 
| o6211g0176o |  529926 | i.00000 | STS.rp.2013.20121011             | STS.rp.2013        |  901140 | cleaned |  871859 | full    |  849364 | full  |  852331 | cleaned | 
| o6211g0197o |  529947 | i.00000 | STS.rp.2013.20121011             | STS.rp.2013        |  901158 | cleaned |  871879 | full    |  849395 | full  |  852362 | cleaned | 
| o6211g0204o |  529954 | i.00000 | STS.rp.2013.20121011             | STS.rp.2013        |  901164 | cleaned |  871899 | full    |  849418 | full  |  852385 | cleaned | 
| o6211g0211o |  529961 | i.00000 | STS.rp.2013.20121011             | STS.rp.2013        |  901170 | cleaned |  871883 | full    |  849404 | full  |  852371 | cleaned | 
| o6211g0218o |  529968 | i.00000 | STS.rp.2013.20121011             | STS.rp.2013        |  901176 | cleaned |  871904 | full    |  849452 | full  |  852419 | cleaned | 
| o6212g0144o |  530557 | i.00000 | STS.rp.2013.20121012             | STS.rp.2013        |  901182 | cleaned |  871928 | full    |  849443 | full  |  852410 | cleaned | 
| o6212g0151o |  530564 | i.00000 | STS.rp.2013.20121012             | STS.rp.2013        |  901188 | cleaned |  871933 | full    |  849500 | full  |  852467 | cleaned | 
| o6212g0158o |  530572 | i.00000 | STS.rp.2013.20121012             | STS.rp.2013        |  901194 | cleaned |  871942 | full    |  849450 | full  |  852417 | cleaned | 
| o6212g0165o |  530579 | i.00000 | STS.rp.2013.20121012             | STS.rp.2013        |  901200 | cleaned |  871950 | full    |  849460 | full  |  852427 | cleaned | 
| o6212g0172o |  530586 | i.00000 | STS.rp.2013.20121012             | STS.rp.2013        |  901206 | cleaned |  871946 | full    |  849453 | full  |  852420 | cleaned | 
| o6212g0179o |  530593 | i.00000 | STS.rp.2013.20121012             | STS.rp.2013        |  901212 | cleaned |  871959 | full    |  849466 | full  |  852433 | cleaned | 
| o6212g0186o |  530600 | i.00000 | STS.rp.2013.20121012             | STS.rp.2013        |  901218 | cleaned |  871991 | full    |  849525 | full  |  852492 | cleaned | 
| o6212g0193o |  530607 | i.00000 | STS.rp.2013.20121012             | STS.rp.2013        |  901224 | cleaned |  871965 | full    |  849464 | full  |  852431 | cleaned | 
| o6212g0200o |  530614 | i.00000 | STS.rp.2013.20121012             | STS.rp.2013        |  901230 | cleaned |  871969 | full    |  849479 | full  |  852446 | cleaned | 
| o6212g0207o |  530621 | i.00000 | STS.rp.2013.20121012             | STS.rp.2013        |  901236 | cleaned |  871977 | full    |  849480 | full  |  852447 | cleaned | 
| o6212g0214o |  530628 | i.00000 | STS.rp.2013.20121012             | STS.rp.2013        |  901242 | cleaned |  872021 | full    |  849513 | full  |  852480 | cleaned | 
| o6213g0134o |  530795 | i.00000 | STS.rp.2013.20121013             | STS.rp.2013        |  901265 | cleaned |  872013 | full    |  849515 | full  |  852482 | cleaned | 
| o6213g0156o |  530817 | i.00000 | STS.rp.2013.20121013             | STS.rp.2013        |  901284 | cleaned |  872040 | full    |  849538 | full  |  852505 | cleaned | 
| o6215g0120o |  531760 | i.00000 | STS.rp.2013.20121015             | STS.rp.2013        |  901299 | cleaned |  872080 | full    |  849574 | full  |  852541 | cleaned | 
| o6215g0127o |  531767 | i.00000 | STS.rp.2013.20121015             | STS.rp.2013        |  901305 | cleaned |  872073 | full    |  849575 | full  |  852542 | cleaned | 
| o6215g0134o |  531774 | i.00000 | STS.rp.2013.20121015             | STS.rp.2013        |  901311 | cleaned |  872087 | full    |  849578 | full  |  852545 | cleaned | 
| o6215g0141o |  531781 | i.00000 | STS.rp.2013.20121015             | STS.rp.2013        |  901317 | cleaned |  872124 | full    |  849616 | full  |  852583 | cleaned | 
| o6215g0148o |  531788 | i.00000 | STS.rp.2013.20121015             | STS.rp.2013        |  901323 | cleaned |  872101 | full    |  849654 | full  |  852621 | cleaned | 
| o6215g0113o |  531795 | i.00000 | STS.rp.2013.20121015             | STS.rp.2013        |  901329 | cleaned |  872188 | full    |  849684 | full  |  852651 | cleaned | 
| o6215g0155o |  531796 | i.00000 | STS.rp.2013.20121015             | STS.rp.2013        |  901330 | cleaned |  872159 | full    |  849645 | full  |  852612 | cleaned | 
| o6215g0162o |  531803 | i.00000 | STS.rp.2013.20121015             | STS.rp.2013        |  901336 | cleaned |  872139 | full    |  849625 | full  |  852592 | cleaned | 
| o6215g0169o |  531810 | i.00000 | STS.rp.2013.20121015             | STS.rp.2013        |  901342 | cleaned |  872160 | full    |  849648 | full  |  852615 | cleaned | 
| o6215g0176o |  531817 | i.00000 | STS.rp.2013.20121015             | STS.rp.2013        |  901348 | cleaned |  872196 | full    |  849688 | full  |  852655 | cleaned | 
| o6215g0190o |  531830 | i.00000 | STS.rp.2013.20121015             | STS.rp.2013        |  901359 | cleaned |  872227 | full    |  849722 | full  |  852689 | cleaned | 
| o6216g0129o |  532424 | i.00000 | STS.rp.2013.20121016             | STS.rp.2013        |  901384 | cleaned |  872236 | full    |  849727 | full  |  852694 | cleaned | 
| o6216g0185o |  532480 | i.00000 | STS.rp.2013.20121016             | STS.rp.2013        |  901432 | cleaned |  872296 | full    |  849777 | full  |  852744 | cleaned | 
| o6217g0095o |  532891 | i.00000 | STS.rp.2013.20121017             | STS.rp.2013        |  901436 | cleaned |  872316 | full    |  849814 | full  |  852781 | cleaned | 
| o6217g0111o |  532906 | i.00000 | STS.rp.2013.20121017             | STS.rp.2013        |  901449 | cleaned |  872361 | full    |  849859 | full  |  852826 | cleaned | 
| o6217g0139o |  532935 | i.00000 | STS.rp.2013.20121017             | STS.rp.2013        |  901474 | cleaned |  872379 | full    |  849872 | full  |  852839 | cleaned | 
| o6208g0086o |  527811 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943295 | update  |  912375 | full    |  889378 | full  |  892340 | cleaned | 
| o6208g0102o |  527827 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943309 | update  |  912391 | full    |  889401 | full  |  892363 | update  | 
| o6208g0116o |  527841 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943321 | update  |  912402 | full    |  889405 | full  |  892367 | cleaned | 
| o6208g0123o |  527848 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943327 | update  |  912407 | full    |  889412 | full  |  892374 | cleaned | 
| o6208g0130o |  527855 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943333 | update  |  912414 | full    |  889418 | full  |  892380 | cleaned | 
| o6208g0137o |  527862 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943338 | update  |  912417 | full    |  889421 | full  |  892383 | cleaned | 
| o6208g0144o |  527869 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943344 | update  |  912422 | full    |  889426 | full  |  892388 | cleaned | 
| o6208g0151o |  527876 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943350 | update  |  912428 | full    |  889448 | full  |  892410 | cleaned | 
| o6208g0158o |  527883 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943356 | update  |  912434 | full    |  889435 | full  |  892397 | cleaned | 
| o6208g0165o |  527890 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943362 | update  |  912440 | full    |  889441 | full  |  892403 | cleaned | 
| o6208g0172o |  527897 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943368 | update  |  912446 | full    |  889452 | full  |  892414 | cleaned | 
| o6208g0179o |  527904 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943374 | update  |  912452 | full    |  889455 | full  |  892417 | cleaned | 
| o6208g0186o |  527911 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943380 | update  |  912458 | full    |  889461 | full  |  892423 | cleaned | 
| o6208g0193o |  527918 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943386 | update  |  912465 | full    |  889468 | full  |  892430 | cleaned | 
| o6208g0200o |  527925 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943392 | update  |  912469 | full    |  889473 | full  |  892435 | cleaned | 
| o6208g0207o |  527932 | i.00000 | STS.rp.2014.20121008             | ps_ud_MOPS.2       |  943398 | update  |  912477 | full    |  889486 | full  |  892448 | cleaned | 
| o6209g0131o |  528511 | i.00000 | STS.rp.2014.20121009             | ps_ud_MOPS.2       |  943404 | update  |  912483 | full    |  889490 | full  |  892452 | cleaned | 
| o6209g0138o |  528517 | i.00000 | STS.rp.2014.20121009             | ps_ud_MOPS.2       |  943410 | update  |  912488 | full    |  889493 | full  |  892455 | cleaned | 
| o6209g0145o |  528524 | i.00000 | STS.rp.2014.20121009             | ps_ud_MOPS.2       |  943416 | update  |  912494 | full    |  889498 | full  |  892460 | cleaned | 
| o6209g0152o |  528531 | i.00000 | STS.rp.2014.20121009             | ps_ud_MOPS.2       |  943422 | update  |  912501 | full    |  889502 | full  |  892464 | cleaned | 
| o6209g0159o |  528538 | i.00000 | STS.rp.2014.20121009             | ps_ud_MOPS.2       |  943427 | update  |  912505 | full    |  889509 | full  |  892471 | cleaned | 
| o6209g0166o |  528545 | i.00000 | STS.rp.2014.20121009             | ps_ud_MOPS.2       |  943433 | update  |  912512 | full    |  889515 | full  |  892477 | update  | 
| o6209g0173o |  528552 | i.00000 | STS.rp.2014.20121009             | ps_ud_MOPS.2       |  943439 | update  |  912517 | full    |  889520 | full  |  892482 | update  | 
| o6209g0180o |  528559 | i.00000 | STS.rp.2014.20121009             | ps_ud_MOPS.2       |  943445 | update  |  912524 | full    |  889528 | full  |  892490 | cleaned | 
| o6209g0187o |  528566 | i.00000 | STS.rp.2014.20121009             | ps_ud_MOPS.2       |  943451 | update  |  912530 | full    |  889535 | full  |  892497 | update  | 
| o6209g0194o |  528573 | i.00000 | STS.rp.2014.20121009             | ps_ud_MOPS.2       |  943457 | update  |  912535 | full    |  889539 | full  |  892501 | cleaned | 
| o6209g0208o |  528587 | i.00000 | STS.rp.2014.20121009             | ps_ud_MOPS.2       |  943469 | update  |  912548 | full    |  889552 | full  |  892514 | cleaned | 
| o6210g0126o |  529194 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943475 | update  |  912553 | full    |  889557 | full  |  892519 | cleaned | 
| o6210g0133o |  529201 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943481 | update  |  912559 | full    |  889561 | full  |  892523 | cleaned | 
| o6210g0140o |  529208 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943487 | update  |  912564 | full    |  889567 | full  |  892529 | cleaned | 
| o6210g0147o |  529215 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943493 | update  |  912570 | full    |  889571 | full  |  892533 | update  | 
| o6210g0154o |  529222 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943499 | update  |  912576 | full    |  889580 | full  |  892542 | cleaned | 
| o6210g0175o |  529242 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943517 | update  |  912594 | full    |  889606 | full  |  892568 | cleaned | 
| o6210g0182o |  529249 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943523 | update  |  912607 | full    |  889623 | full  |  892585 | cleaned | 
| o6210g0189o |  529256 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943529 | update  |  912623 | full    |  889630 | full  |  892592 | update  | 
| o6210g0168o |  529258 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943511 | update  |  912588 | full    |  889595 | full  |  892557 | cleaned | 
| o6210g0196o |  529264 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943535 | update  |  912647 | full    |  889702 | full  |  892664 | update  | 
| o6210g0210o |  529278 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943547 | update  |  912693 | full    |  889703 | full  |  892665 | update  | 
| o6210g0224o |  529292 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943559 | update  |  912716 | full    |  889720 | full  |  892682 | update  | 
| o6210g0231o |  529299 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943565 | update  |  912722 | full    |  889726 | full  |  892688 | cleaned | 
| o6210g0238o |  529306 | i.00000 | STS.rp.2014.20121010             | ps_ud_MOPS.2       |  943571 | update  |  912729 | full    |  889733 | full  |  892695 | update  | 
| o6211g0148o |  529898 | i.00000 | STS.rp.2014.20121011             | ps_ud_MOPS.2       |  943585 | update  |  912742 | full    |  889743 | full  |  892705 | cleaned | 
| o6211g0155o |  529905 | i.00000 | STS.rp.2014.20121011             | ps_ud_MOPS.2       |  943591 | update  |  912748 | full    |  889752 | full  |  892714 | update  | 
| o6211g0162o |  529912 | i.00000 | STS.rp.2014.20121011             | ps_ud_MOPS.2       |  943597 | update  |  912754 | full    |  889753 | full  |  892715 | cleaned | 
| o6211g0176o |  529926 | i.00000 | STS.rp.2014.20121011             | ps_ud_MOPS.2       |  943609 | update  |  912766 | full    |  889769 | full  |  892731 | cleaned | 
| o6211g0197o |  529947 | i.00000 | STS.rp.2014.20121011             | ps_ud_MOPS.2       |  943627 | update  |  912784 | full    |  889794 | full  |  892756 | cleaned | 
| o6211g0204o |  529954 | i.00000 | STS.rp.2014.20121011             | ps_ud_MOPS.2       |  943633 | update  |  912790 | full    |  889792 | full  |  892754 | update  | 
| o6211g0211o |  529961 | i.00000 | STS.rp.2014.20121011             | ps_ud_MOPS.2       |  943639 | update  |  912796 | full    |  889797 | full  |  892759 | update  | 
| o6211g0218o |  529968 | i.00000 | STS.rp.2014.20121011             | ps_ud_MOPS.2       |  943645 | update  |  912802 | full    |  889805 | full  |  892767 | update  | 
| o6212g0144o |  530557 | i.00000 | STS.rp.2014.20121012             | ps_ud_MOPS.2       |  943651 | update  |  912808 | full    |  889812 | full  |  892774 | update  | 
| o6212g0151o |  530564 | i.00000 | STS.rp.2014.20121012             | ps_ud_MOPS.2       |  943657 | update  |  912814 | full    |  889817 | full  |  892779 | update  | 
| o6212g0158o |  530572 | i.00000 | STS.rp.2014.20121012             | ps_ud_MOPS.2       |  943663 | update  |  912820 | full    |  889833 | full  |  892795 | update  | 
| o6212g0165o |  530579 | i.00000 | STS.rp.2014.20121012             | ps_ud_MOPS.2       |  943669 | update  |  912826 | full    |  889829 | full  |  892791 | update  | 
| o6212g0172o |  530586 | i.00000 | STS.rp.2014.20121012             | ps_ud_MOPS.2       |  943675 | update  |  912832 | full    |  889834 | full  |  892796 | update  | 
| o6212g0179o |  530593 | i.00000 | STS.rp.2014.20121012             | ps_ud_MOPS.2       |  943681 | update  |  912838 | full    |  889839 | full  |  892801 | cleaned | 
| o6212g0186o |  530600 | i.00000 | STS.rp.2014.20121012             | ps_ud_MOPS.2       |  943687 | update  |  912844 | full    |  889849 | full  |  892811 | update  | 
| o6212g0193o |  530607 | i.00000 | STS.rp.2014.20121012             | ps_ud_MOPS.2       |  943693 | update  |  912850 | full    |  889850 | full  |  892812 | update  | 
| o6212g0200o |  530614 | i.00000 | STS.rp.2014.20121012             | ps_ud_MOPS.2       |  943699 | update  |  912856 | full    |  889858 | full  |  892820 | update  | 
| o6212g0207o |  530621 | i.00000 | STS.rp.2014.20121012             | ps_ud_MOPS.2       |  943705 | update  |  912862 | full    |  889865 | full  |  892827 | update  | 
| o6212g0214o |  530628 | i.00000 | STS.rp.2014.20121012             | ps_ud_MOPS.2       |  943711 | update  |  912868 | full    |  889869 | full  |  892831 | update  | 
| o6213g0134o |  530795 | i.00000 | STS.rp.2014.20121013             | ps_ud_MOPS.2       |  943734 | update  |  912891 | full    |  889889 | full  |  892851 | update  | 
| o6213g0156o |  530817 | i.00000 | STS.rp.2014.20121013             | ps_ud_MOPS.2       |  943753 | update  |  912910 | full    |  889905 | full  |  892867 | update  | 
| o6215g0120o |  531760 | i.00000 | STS.rp.2014.20121015             | ps_ud_MOPS.2       |  943770 | update  |  912927 | full    |  889923 | full  |  892885 | update  | 
| o6215g0127o |  531767 | i.00000 | STS.rp.2014.20121015             | ps_ud_MOPS.2       |  943776 | update  |  912933 | full    |  889929 | full  |  892891 | update  | 
| o6215g0134o |  531774 | i.00000 | STS.rp.2014.20121015             | ps_ud_MOPS.2       |  943782 | update  |  912939 | full    |  889942 | full  |  892904 | update  | 
| o6215g0141o |  531781 | i.00000 | STS.rp.2014.20121015             | ps_ud_MOPS.2       |  943788 | update  |  912945 | full    |  889940 | full  |  892902 | cleaned | 
| o6215g0148o |  531788 | i.00000 | STS.rp.2014.20121015             | ps_ud_MOPS.2       |  943794 | update  |  912951 | full    |  889944 | full  |  892906 | update  | 
| o6215g0113o |  531795 | i.00000 | STS.rp.2014.20121015             | ps_ud_MOPS.2       |  943764 | update  |  912921 | full    |  889917 | full  |  892879 | update  | 
| o6215g0155o |  531796 | i.00000 | STS.rp.2014.20121015             | ps_ud_MOPS.2       |  943800 | update  |  912957 | full    |  889951 | full  |  892913 | update  | 
| o6215g0162o |  531803 | i.00000 | STS.rp.2014.20121015             | ps_ud_MOPS.2       |  943806 | update  |  912963 | full    |  889959 | full  |  892921 | update  | 
| o6215g0169o |  531810 | i.00000 | STS.rp.2014.20121015             | ps_ud_MOPS.2       |  943812 | update  |  912969 | full    |  889962 | full  |  892924 | update  | 
| o6215g0176o |  531817 | i.00000 | STS.rp.2014.20121015             | ps_ud_MOPS.2       |  943818 | update  |  912975 | full    |  889967 | full  |  892929 | cleaned | 
| o6215g0190o |  531830 | i.00000 | STS.rp.2014.20121015             | ps_ud_MOPS.2       |  943830 | update  |  912987 | full    |  889981 | full  |  892943 | update  | 
| o6216g0129o |  532424 | i.00000 | STS.rp.2014.20121016             | ps_ud_MOPS.2       |  943854 | update  |  913425 | full    |  890411 | full  |  893373 | update  | 
| o6216g0185o |  532480 | i.00000 | STS.rp.2014.20121016             | ps_ud_MOPS.2       |  943902 | update  |  913473 | full    |  890458 | full  |  893420 | update  | 
| o6217g0095o |  532891 | i.00000 | STS.rp.2014.20121017             | ps_ud_MOPS.2       |  943906 | update  |  913477 | full    |  890477 | full  |  893439 | update  | 
| o6217g0111o |  532906 | i.00000 | STS.rp.2014.20121017             | ps_ud_MOPS.2       |  943920 | update  |  913491 | full    |  890478 | full  |  893440 | update  | 
| o6217g0139o |  532935 | i.00000 | STS.rp.2014.20121017             | ps_ud_MOPS.2       |  943944 | update  |  913514 | full    |  890501 | full  |  893463 | update  | 
| o5395g0050o |  192649 | g.00000 | ThreePi.20100718                 | goto_cleaned       |  114515 | cleaned |   97305 | full    |   83980 | full  |   83671 | cleaned | 
| o6356g0420o |  587015 | r.00000 | ThreePi.20130305                 | goto_cleaned_redo  |  788377 | cleaned |  763876 | cleaned |  741978 | full  |  743689 | cleaned | 
| o6356g0438o |  587033 | i.00000 | ThreePi.20130305                 | goto_cleaned_redo  |  788395 | cleaned |  763902 | cleaned |  742002 | full  |  743713 | cleaned | 
| o6356g0456o |  587051 | i.00000 | ThreePi.20130305                 | goto_cleaned_redo  |  788413 | cleaned |  763957 | cleaned |  742062 | full  |  743773 | cleaned | 
| o6357g0380o |  587642 | g.00000 | ThreePi.20130306                 | goto_cleaned_redo  |  789438 | cleaned |  765646 | cleaned |  743747 | full  |  745458 | cleaned | 
| o6401g0326o |  601772 | r.00000 | ThreePi.20130419                 | goto_cleaned_redo  |  826379 | cleaned |  803123 | cleaned |  780817 | full  |  783137 | cleaned | 
| o6401g0344o |  601790 | r.00000 | ThreePi.20130419                 | goto_cleaned_redo  |  826397 | cleaned |  803141 | cleaned |  780834 | full  |  783154 | cleaned | 
| o6412g0207o |  607408 | r.00000 | ThreePi.20130430                 | goto_cleaned_redo  |  832186 | cleaned |  808272 | cleaned |  785955 | full  |  788938 | cleaned | 
| o6427g0160o |  611797 | g.00000 | ThreePi.20130515                 | goto_cleaned_redo  |  837561 | cleaned |  813642 | cleaned |  791309 | full  |  794293 | cleaned | 
| o6427g0178o |  611816 | g.00000 | ThreePi.20130515                 | goto_cleaned_redo  |  837579 | cleaned |  813659 | cleaned |  791326 | full  |  794310 | cleaned | 
| o6427g0206o |  611845 | g.00000 | ThreePi.20130515                 | goto_cleaned_redo  |  837607 | cleaned |  813687 | cleaned |  791358 | full  |  794342 | cleaned | 
| o6430g0247o |  613048 | i.00000 | ThreePi.20130518                 | goto_cleaned_redo  |  838522 | cleaned |  814615 | cleaned |  792279 | full  |  795263 | cleaned | 
| o6430g0265o |  613073 | i.00000 | ThreePi.20130518                 | goto_cleaned_redo  |  838542 | cleaned |  814638 | cleaned |  792306 | full  |  795290 | cleaned | 
| o6478g0064o |  629141 | y.00000 | ThreePi.20130705                 | goto_cleaned       |  855286 | cleaned |  831247 | cleaned |  808853 | full  |  811834 | cleaned | 
| o6481g0080o |  630036 | z.00000 | ThreePi.20130708                 | goto_cleaned       |  855897 | cleaned |  831858 | cleaned |  809461 | full  |  812442 | cleaned | 
| o6592g0111o |  670085 | z.00000 | ThreePi.20131027                 | goto_cleaned_redo  |  906309 | cleaned |  876964 | full    |  854457 | full  |  857424 | cleaned | 
| o6674g0401o |  697666 | r.00000 | ThreePi.20140117                 | goto_cleaned       |  938097 | cleaned |  907593 | full    |  884601 | full  |  887563 | cleaned | 
| o6675g0565o |  698412 | z.00000 | ThreePi.20140118                 | goto_cleaned       |  938763 | cleaned |  908255 | full    |  885263 | full  |  888225 | cleaned | 
| o6709g0444o |  707443 | r.00000 | ThreePi.20140221                 | goto_cleaned       |  949701 | cleaned |  918234 | full    |  895187 | full  |  898149 | cleaned | 
| o6709g0461o |  707460 | r.00000 | ThreePi.20140221                 | goto_cleaned       |  949718 | cleaned |  918251 | full    |  895197 | full  |  898159 | cleaned | 
| o6727g0192o |  714050 | r.00000 | ThreePi.20140311                 | goto_cleaned       |  956587 | cleaned |  925681 | full    |  902555 | full  |  905507 | cleaned | 
| o7688g0672o | 1156941 | z.00000 | ThreePi.20161027                 | goto_cleaned       | 1835019 | cleaned | 1802536 | full    | 1775029 | full  | 1781644 | full    | 
+-------------+---------+---------+----------------------------------+--------------------+---------+---------+---------+---------+---------+-------+---------+---------+

There are a bunch of ThreePi, STS, STS.rerun, STS.rp, OSS, LAP.threepi, LAP.PV3, a few MOPSreq, and a single ESS

For the borken LAP.PV3 stuff (none of it is w-band), using the command listed on the PSNSC_MOPS page:

chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 714050
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 587642
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 587015
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 587033
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 587051
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 601772
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 601790
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 607408
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 705016
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 705035
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 705057
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 705079
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 706839
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 706859
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 706877
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 706896
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 707443
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 707460
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 611797
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 611816
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 611845
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 613048
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 613073
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 670085
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 192649
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 698412
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 697666
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 629141
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_id 630036

I might have sent a few dupes through too...

However, they all succeeded! Except perhaps exp_id=192649, where it has NULL for the fake and warp stage... (see later)

Next, let's try and reprocess the ESS and OSS ones:

chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenESS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 690914

chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 630216
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 642778
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 663086
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 663087
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 663102
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 663123
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 1306007
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 1403156
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 1403179
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 1403202
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 1403225
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 1406840
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 1406862
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 1406884
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenOSS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenESS.20190213 -set_reduction SWEETSPOT -exp_id 1406906

For the ThreePi ones, the majority of the exposures have already been reprocessed in the LAP sample, leaving only: exp_id=1156941 to be processed still.

chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenThreePi.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenThreePi.20190213 -exp_id 1156941 -pretend

Next, the STS stuff. Each label is actually just a reprocess of the original STS.20XXXX

chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527811
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527827
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527841
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527848
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527855
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527862
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527869
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527876
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527883
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527890
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527897
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527904
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527911
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527918
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527925
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 527932
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 528511
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 528517
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 528524
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 528531
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 528538
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 528545
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 528552
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 528559
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 528566
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 528573
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 528587
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529194
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529201
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529208
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529215
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529222
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529242
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529249
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529256
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529258
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529264
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529278
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529292
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529299
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529306
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529898
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529905
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529912
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529926
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529947
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529954
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529961
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 529968
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530557
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530564
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530572
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530579
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530586
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530593
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530600
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530607
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530614
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530621
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530628
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530795
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 530817
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 531760
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 531767
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 531774
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 531781
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 531788
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 531795
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 531796
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 531803
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 531810
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 531817
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 531830
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 532424
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 532480
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 532891
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 532906
chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenSTS.20190213  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenSTS.20190213 -set_reduction STS_DATASET -exp_id 532935

The final lot are the MOPSreq label - but al of those exposures have already been processed by the above.

I then queried each exposure (in a script) to make sure that it has chip state full, and warp state full. They all do! Except for exp_id=192649

So I try to reprocess it again:

chiptool -dbname gpc1 -definebyquery -set_label mopsfix  -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20190214  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20190214 -set_reduction LAP_SCIENCE -exp_id 192649

Failed again. Mark showed me that in the camProcessedExp table there is information on the FWHM and quality, and the FWHM was over 12 and the quality was 4007, meaning that it could not get past this stage because there is cutoff for such poor quality. Instead, reprocess it wit the QUB reduction which Mark knows is less strict on FWHM:

chiptool -dbname gpc1 -definebyquery -set_label mopsfix -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20190214 -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20190214 -set_reduction QUB_DEFAULT -exp_id 192649

This succeeded! Now everything has been reprocessed (not that it necessarily needed to be).

The cleaned chips can most often just be updated. So try to update first.

chiptool -dbname gpc1 -setimfiletoupdate  -set_label ps_ud_WEB -chip_id XXXXXX
warptool -dbname gpc1  -setskyfiletoupdate -set_label ps_ud_WEB -warp_id XXXXXX

NEXT: Checking PStamp server requests.

All the previous pstamp requests are saved under:

/data/ippc113.0/pstamp/work/YYYY/MM/DD/

followed by a directory for the request which contains the initial request form web_XXXXXX.fits. This is a fits table file, and to resubmit it requires a unique id. The previous one will likely say weryk_XXXXX_YY, just add something to the end to be able to upload it again. Also, change the email address in the table to your own so it doesn't mess with MOPS stuff.

Resubmitting the file on the PStamp upload page will process it. In this particular case there should hopefully be nothing that needs processing by IPP as I've fixed these ones!

CCL: set nodes ipp114.1, ippb08.0, ippb08.1 up->repair due to "DISK WARNING - free space: /export/ippb08.0 2634988 MB (2% inode=99%):"

Friday : 2019.02.15

  • 09:30 EAM : I discovered the ipp051 nebulous directory rsynced to ipp080. I'm running the neb reinsert script to let nebulous know where these instances now live (they get moved from the /data/ipp080.0/retire_rsyncX/ipp051.0 subdirectory to the regular ipp080 nebulous directory).

JRF: More pstamp shenanigans:

Yesterday, the job I sent got interrupted because of the daily reset. This resulted in a fault being reported for it. To clear this away you can follow the steps on the PSNSC_MOPS page:

pstamptool -dbname ippRequestServer -dbserver ipp113 -updatereq  -set_state stop -req_id XXXXX

You can also clean it away, but it will get cleaned up in 14 days anyway.

Sent another much shorter request that contains a few of the problem exposures. There is a log file that can be viewed by following the links on the psstamp website. It contains a relatively limited amount of information. The reqfinish log in the pstamp directory contains more information.

There were still from fault [25] errors reported. To see what the issue issue is look at the log file by following the 'outputbase' in the reqlog. In this new log there is the actual error reported, along with the nebulous paths for the files used, specifically the skycell fits file. The fits files used can be loaded into DS9 and the co-ordinates double checked. In this particular case all of the errors were because the 'Region of Interest' was outside of the actual skycell. It's not really that the stamp is unavailable, just that the location requested isn't in the relevant skycell. It can occur when looking at overlapping skycells. The error reported is misleading, it should really be changed to report differently. So they're not actually missing.

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD

Last modified 7 years ago Last modified on Feb 19, 2019, 10:29:46 PM
Note: See TracWiki for help on using the wiki.