IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links
wiki:PS1_IPP_Czarlog_20170807

Version 2 (modified by Mark Huber, 9 years ago) ( diff )

--

PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

    1. PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD
      1. Monday : YYYY.MM.DD
      2. Tuesday : 2017.08.08
      3. Wednesday : YYYY.MM.DD
      4. Thursday : YYYY.MM.DD
      5. Friday : 2017.08.11
      6. Saturday : 2017.08.12
      7. Sunday : YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Monday : YYYY.MM.DD

Tuesday : 2017.08.08

  • MEH: another LAP stack missing files and blocking WSdiffs, set qual 42 to clear
  • MEH: QUB stamps stalled since 8/7 due to fault 3, manually cleared

Wednesday : YYYY.MM.DD

Thursday : YYYY.MM.DD

Friday : 2017.08.11

  • MEH: large number of summit faults from Dell switch fallover (see Sidik email) -- fault 202 to clear manually

Saturday : 2017.08.12

  • MEH: chip processing has been having some very large times during nightly and appear to be slowing the nightly processing rate -- manually adjusted ippitc/stdscience node allocation, removing si0 2x and adding ci0+1 2x roughly seemed to help last night -- monitoring
    • si0 nodes have dvo/psps processing and seems to be multiple rsync jobs for dvo to ippb nodes from Gene tonight (unsure how long that has been running)

Sunday : YYYY.MM.DD

Note: See TracWiki for help on using the wiki.