Changes between Version 31 and Version 32 of PS1_IPP_Czarlog_20170116

Timestamp:

Jan 22, 2017, 10:48:42 PM (10 years ago)

Author:

Mark Huber

Comment:

--

PS1_IPP_Czarlog_20170116

@@ -31 +31 @@
   * last gpc1 dump was successful ~0000 last night before ippdb machines went down last night
   * AC up but leaving machines off until morning and winds lighten in case AC crashes again 
- * MEH: was QUB night, will restart pantasks now that things appear to be back online
+ * MEH: was QUB night, will restart pantasks now that things appear to be back online -- 
   * ipp032 also not booting -- normally neb-host repair, but must set neb-host down
   * ipp121 not booting -- must be set neb-host down until fixed -- hopefully wont bork summitcopy since power glitch interrupted download o7775g0046f-o7775g0049f but only flats
@@ -39 +39 @@
   * not sure if nebdiskd is running still or where it runs now (ippdb08 seems to be from latest czarlog entry?)
  * EAM: restarted nebdiskd on ippdb01 (location of nebulous mysql master server)
+ * MEH: setting nightly processing up for another night of QUB targeted followup -- no changes should be made to nightly processing
  * MEH: Gene starting data shuffle on stare04 started killing ipp088,104 (date nodes critical for priority QUB processing...) -- 140 remote_md5sum.pl jobs and rising -- so set to neb-host repair -- setting ~ipptest/replication stop -- cannot have datanode minefield for nightly processing...
  * MEH: ipp118, ipp120 appears to be having XFS issues -- neb-host repair until can look into... -- regular faults then started clearing