== PS1 IPP Czar Logs for the week 2013.01.07 - 2013.01.13 == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2013.01.07 ===
 * 04:40 Gene tried power cycling ipp027 and no response, further details pending  
 * 10:00 Bill restarted pstamp and update pantasks. pcontrol for deepstack is using 30% of a cpu on ippc17. Since the task durations are so large this isn't too wasteful. 
 * 13:17 Bill restarted cleanup pantasks. Added label goto_cleaned.rerun. Setting all chip and warp runs from LAP.2011% to state = goto_cleaned, label = goto_cleaned.rerun. This will cause cleanup to be rerun using the new code which will cause chip warp cmfs to be deleted.

=== Tuesday : 2013.01.08 ===
 * 03:20 MEH: looks like registration pantasks has crashed, starting. also looks like distribution is stuck, restarting.
 * 07:45 MEH: MD03 o6300g0570o is strange, looks like streaks. observation started while still slewing? nightly stack in progress, will it be okay? so far looks okay.
 * 15:00 Serge: Restarted a bunch of mysql servers on storage nodes. Made a backup of the mysql configuration files in the operational repository: file:///data/ippc18.0/home/operational_repository/current/databases
 * 15:33 Bill: fixed a bug in ipp_cleanup.pl that was causing cleanup to be run repeatedly on the same runs
=== Wednesday : 2013.01.09 ===
 mark is czar (calendar loaded again through end of month)
 * 08:30 Serge: Stopped nebulous replication on ippdb02. Performing backup to ippc63 (through script being tested).
=== Thursday : 2013.01.10 ===
 mark is czar
 * 10:25 Serge: stsci04.1 is full
{{{
neb-host --volume stsci04.1 repair
}}}
 * 10:45 Serge: The neb-df figure in czartool has not been updated for about two days. I restarted nebdiskd on ippdb00.
=== Friday : YYYY.MM.DD ===

=== Saturday : YYYY.MM.DD ===

=== Sunday : YYYY.MM.DD ===