Changes between Version 4 and Version 5 of PS1_IPP_Czarlog_20190510

Timestamp:

May 11, 2019, 12:54:43 PM (7 years ago)

Author:

tdeboer

Comment:

--

PS1_IPP_Czarlog_20190510

@@ -64 +64 @@
  In fact, it has already been diffed, since we see two separate diff_ids for this obejct, diff_id=1764462 (made from 2107382 and 2107403) and diff_id=1765059 (made from 2107403 and 2107442). I am guessing the nightly_science.pl script kicked in here, at the end of the night.
 
+ ----------------
+
+ There are some warps complaining about missing .mk files. First set them to update to see if that fixes the problem:
+{{{
+warptool -dbname gpc1 -setskyfiletoupdate -set_label ps_ud_WEB -warp_id 1292074
+warptool -dbname gpc1 -setskyfiletoupdate -set_label ps_ud_WEB -warp_id 1292075
+warptool -dbname gpc1 -setskyfiletoupdate -set_label ps_ud_WEB -warp_id 1292079
+}}}
+ failing that, send to reprocess:
+{{{
+chiptool -dbname gpc1 -definebyquery -set_label mopsfix -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_name o6259g0268o
+chiptool -dbname gpc1 -definebyquery -set_label mopsfix -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_name o6259g0280o
+chiptool -dbname gpc1 -definebyquery -set_label mopsfix -set_workdir neb://@HOST@.0/gpc1/mops.fixbrokenLAP.20180323  -set_dist_group NULL -set_tess_id RINGS.V3 -set_end_stage warp -set_data_group mops.fixbrokenLAP.20180323 -set_reduction LAP_SCIENCE -exp_name o6315g0144o
+}}}
+ and clean up their chip and warp ID:
+{{{
+chiptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -chip_id 1342029 
+chiptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -chip_id 1342030 
+chiptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -chip_id 1342034 
+
+warptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -warp_id 1292074
+warptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -warp_id 1292075
+warptool -dbname gpc1 -updaterun -set_state goto_cleaned -set_label goto_cleaned -warp_id 1292079
+}}}
+
+ --------------------
+
+ There are also some I/O relatd errors with ippb18, with fault codes 3 and 4. Judging by Ganglia, it is seeing some fairly heavy usage.
+send the exps to revert, but put ippb18 into repair for the time being to give it a break.
+{{{
+neb-host --state repair --host ippb18 --note 'TdB: up->repair, due to it getting hammered and producing I/O errors'
+}}}
 
 === Sunday : 2019.05.12 ===