PS1 IPP Czar Logs for the week 2015.01.12 - 2015.01.18

(Up to PS1 IPP Czar Logs)

Monday : 2015.01.12

• 00:40 MEH: clearing a fault 5 diff

  difftool -updatediffskyfile -fault 0 -set_quality 42 -diff_id 635299 -skycell_id skycell.1607.047  -dbname gpc1
  

• 00:50 MEH: stdsci poll still low, restarted
• 01:00 MEH: stdlocal set.poll 100 in case do catch up, nightly has priority and be finished in time for fiber switchover
• 02:00 MEH: ipp095 overloaded a bit again, to repair
• 06:57 Bill: two warps were repeatedly getting fault 4 from the "cannot compute curve of growth psf is invalid everywhere" assertion. Set quality for them

  warptool -set_quality 42 -fault 0 -updateskyfile -warp_id 1345413 -skycell_id skycell.0509.081
  warptool -set_quality 42 -fault 0 -updateskyfile -warp_id 1345472 -skycell_id skycell.1084.070
  

• 07:00 Bill ipp090 and ipp093 have rather high loads as well.
  • but processing rate was generally okay
• 10:20 MEH: clearing some fault 5 WSdiffs

  difftool -updatediffskyfile -fault 0 -set_quality 42 -diff_id 635300 -skycell_id skycell.1607.038  -dbname gpc1
  difftool -updatediffskyfile -fault 0 -set_quality 42 -diff_id 635300 -skycell_id skycell.1607.047  -dbname gpc1
  difftool -updatediffskyfile -fault 0 -set_quality 42 -diff_id 635450 -skycell_id skycell.1247.076  -dbname gpc1
  

• 10:30 MEH: clearing some old WSdiff that won't complete due to old and image cleanup.. -- when time may be able to update warps to complete but...

  difftool -updaterun -diff_id 634284 -dbname gpc1 -set_label ThreePi.WS.nightlyscience.missed
  difftool -updaterun -diff_id 621228 -dbname gpc1 -set_label OSS.WS.nightlyscience.missed
  

  • were actually fault 5 and need to have quality set

    difftool -updatediffskyfile -fault 0 -set_quality 42 -diff_id 634284 -skycell_id skycell.0679.021  -dbname gpc1
    difftool -updatediffskyfile -fault 0 -set_quality 42 -diff_id 621228 -skycell_id skycell.1529.088  -dbname gpc1
    

• 11:50 MEH: starting stdlocal chip only test for 10G link usage, stopping all other processing since will need cleared for fiber changes in couple hours anyways
• 11:55 MEH: ippc11 screen issue,

  There are screens on:
  	13722.robo	(Detached)
  	12542.RoboCzar	(Dead ???)
  	19893.CzarPoll	(Dead ???)
  	13565.plot	(Detached)
  Remove dead screens with 'screen -wipe'.
  

  • wiped, cleared other names and restarted screens

Tuesday : YYYY.MM.DD

Wednesday : YYYY.MM.DD

Thursday : YYYY.MM.DD

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD