IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 87 and Version 88 of PS1_IPP_Czarlog_20141215


Ignore:
Timestamp:
Dec 20, 2014, 7:09:24 PM (12 years ago)
Author:
Mark Huber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PS1_IPP_Czarlog_20141215

    v87 v88  
    149149 * 19:50 MEH: nightly is struggling.. stdlocal has 6x:c2, if >2-3 stacks will greatly stall nightly processing also using those nodes so reducing to 2x, some idle anyways..
    150150
    151 === Friday : YYYY.MM.DD ===
     151=== Friday : 2014.12.19 ===
    152152
    153153 * 7:23 haf: bunch of faulted things , clearing them manually regtool -dbname gpc1 -revertprocessedimfile -exp_id_begin 840400
     
    168168  * suspect registration is still having a problem because ipp067-071 is still targeted and across the overloaded 10G link
    169169 * 00:30 MEH: looks like stdlocal has triggered chip-warp off automatically because >50 chips in stdsci -- summitcopy pretty much caught up but chip processing isn't keeping up and dtime is slower -- turning down ippmd some to see if recovers due to overuse of the new datanodes w/o the larger r/wsize mounts (more space so more targeted)
    170 === Saturday : YYYY.MM.DD ===
     170
     171
     172
     173=== Saturday : 2014.12.20 ===
    171174 * 11:20 MEH: reallocation for summitcopy+registration seemed to help, will make autoloading and restart them+stsci today w/ data targeting modifications also
     175 * 13:30 MEH: Haydn and Sifan brought new storage nodes ipp090,093,095 online
     176  * Chris put into nebulous
     177  *
     178 * 19:05 MEH: stdlocal overloading the 10G link is pushing stdsci to large pileup in chip, halfway to point of full auto-shutdown of stdlocal. overloading doesn't gain anything, found last night if chip ~50 then kept link just about full -- will set to that again to see how goes. when stacks and warps are available, they will run at larger poll values
     179  * summitcopy seems to be keeping up w/
     180
     181
     182
     183
     184
     185
     186
    172187=== Sunday : YYYY.MM.DD ===
     188
     189