Changes between Version 5 and Version 6 of PS1_IPP_Czarlog_20161205

Timestamp:

Dec 7, 2016, 11:16:42 PM (10 years ago)

Author:

Mark Huber

Comment:

--

PS1_IPP_Czarlog_20161205

@@ -9 +9 @@
 
 === Wednesday : 2016.12.07 ===
- * MEH: ipp118-122.x were not completely setup and/or checked for ipp use, link for /local/ipp broken -- fixed, adding as group s7, adding to nightly science targeting and heavily loading CPU and IO with MOPS/QUB processing until weather clears
+ * MEH: ipp118-122.x were not completely setup and/or checked for ipp use, link for /local/ipp broken -- fixed, adding as group s7 in pantasks_hosts.input, adding to nightly science targeting and heavily loading CPU and IO with MOPS/QUB processing until weather clears
  * MEH: ippcXX nodes lately have been having NFS hangs that stall MOPS pstamp request (and QUB if there had been any data) -- restarting nfs/rpc.statd seems to clear -- czar needs to keep eye on 
  * MEH: stamps/updates for LAP.PV3 showing ongoing faults due missing files from stsci shuffle -- some reporting ipp118 but looks like due to IP being reused
  * MEH: gpc1 dumps to ipp001 have been 0 size for past couple days it appears due to /data/ipp001.0 being full -- adding an email warning crontab in ipp@ipp001, running cleanup script and rm'ing incomplete dumps so not to contaminate archive...
   * will set one up in ipp@ippc18 to also monitor homedir 
- * MEH: Haydn replace ipp122 10G card and wanted to replace ipp083 BBU -- out of processing and neb-host down
-  * ipp122 back up but cannot log into while disks mount ok -- leave out of processing and in repair -- then later was able to log into again
+ * MEH: Haydn to replace ipp122 10G card and wanted to replace ipp083 BBU -- out of processing and neb-host down
+  * ipp122 back up but cannot log into while disks mount ok -- leave out of processing and in repair -- then later was able to log into again, gmond restart and neb-host up
   * ipp083 unclear, leave in repair until morning
 === Thursday : 2016.12.08 ===