Changes between Version 28 and Version 29 of PS1_IPP_Czarlog_20131007

Timestamp:

Oct 8, 2013, 5:45:30 PM (13 years ago)

Author:

Mark Huber

Comment:

--

PS1_IPP_Czarlog_20131007

@@ -35 +35 @@
   * ippc18 disk is overused, cannot rsync the logs ippc18.0 to ippc18.1 w/o over-driving load and killing processing.. suspect too many accounts are writing too many logs to home disk.  have to set rsync to <800KPS
  * 09:50 MEH: ipp049 had long running (>2hr) pswarp and ppImage, killed and reverted but looks like another pair ppImage stalling -- 
+ * 10:00 MEH: working in nodes that have been recently fixed
+  * ipp031,032
+  * ipp047
  * 11:15 MEH: still no LAP stacks, putting 2x compute3 from stack into pstamp for MOPS (18.5k jobs)
   * possibly a lot of data for MOPS on ipp017 requested at once? driving load >200.. settling down now.  need to check if also was from access by UMD for high throughput tests at that time
@@ -48 +51 @@
 }}}
  * 16:10 MEH: ippmonitor hasn't been updating because czarpoll crashed when ippdb01 went down. restarted on ippc11
- 
+ * 17:40 MEH: ipp032 looks like it went down.. -- ganglia doesn't indicate any overload so likely is down..
 === Wednesday : YYYY.MM.DD ===