Changes between Version 87 and Version 88 of PS1_IPP_Czarlog_20141215
- Timestamp:
- Dec 20, 2014, 7:09:24 PM (12 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
PS1_IPP_Czarlog_20141215
v87 v88 149 149 * 19:50 MEH: nightly is struggling.. stdlocal has 6x:c2, if >2-3 stacks will greatly stall nightly processing also using those nodes so reducing to 2x, some idle anyways.. 150 150 151 === Friday : YYYY.MM.DD===151 === Friday : 2014.12.19 === 152 152 153 153 * 7:23 haf: bunch of faulted things , clearing them manually regtool -dbname gpc1 -revertprocessedimfile -exp_id_begin 840400 … … 168 168 * suspect registration is still having a problem because ipp067-071 is still targeted and across the overloaded 10G link 169 169 * 00:30 MEH: looks like stdlocal has triggered chip-warp off automatically because >50 chips in stdsci -- summitcopy pretty much caught up but chip processing isn't keeping up and dtime is slower -- turning down ippmd some to see if recovers due to overuse of the new datanodes w/o the larger r/wsize mounts (more space so more targeted) 170 === Saturday : YYYY.MM.DD === 170 171 172 173 === Saturday : 2014.12.20 === 171 174 * 11:20 MEH: reallocation for summitcopy+registration seemed to help, will make autoloading and restart them+stsci today w/ data targeting modifications also 175 * 13:30 MEH: Haydn and Sifan brought new storage nodes ipp090,093,095 online 176 * Chris put into nebulous 177 * 178 * 19:05 MEH: stdlocal overloading the 10G link is pushing stdsci to large pileup in chip, halfway to point of full auto-shutdown of stdlocal. overloading doesn't gain anything, found last night if chip ~50 then kept link just about full -- will set to that again to see how goes. when stacks and warps are available, they will run at larger poll values 179 * summitcopy seems to be keeping up w/ 180 181 182 183 184 185 186 172 187 === Sunday : YYYY.MM.DD === 188 189
