PS1 IPP Czar Logs for the week 2014.04.21 - 2014.04.27

(Up to PS1 IPP Czar Logs)

Monday : 2014.04.21

Bill is czar today

• 06:20 : EAM : preparing to add ps2_tc3 to the ippdb03 mysql replication (dumped the ippdb01 database & copied to ippdb03):

  mysql> show master status;
  +-------------------+----------+--------------+------------------+
  | File              | Position | Binlog_Do_DB | Binlog_Ignore_DB |
  +-------------------+----------+--------------+------------------+
  | mysqld-bin.030242 | 44889610 |              |                  | 
  +-------------------+----------+--------------+------------------+
  

• 11:48 Bill set pantasks to stop in preparation for periodic restart
  • added ipp055 to the hosts ignore list while the raid is rebuilding. Haydn put it in repair
• 11:55 restarted stdscience, distribution, cleanup, pstamp, registration, stack, and summitcopy. Leaving stack in stopped state because I forgot to wait for jobs started by the previous instance to finish.
  • 12:12 stack set to run
• 13:50 investigated corrupt raw file neb://ipp036.0/gpc1/20120817/o6156g0277o/o6156g0277o.ota55.fits. Turns out that there were 5 instances only one of which was corrupt. Fixed this.

Tuesday : 2014.04.22

Bill is czar today

• 12:52 setting pantasks to stop in preparation for periodic restart
  • with restart ipp055 has been removed from the hosts ignore list since it has successfully finished it's RAID rebuild

Wednesday : 2014.04.23

Thursday : 2014.04.24

Friday : 2014.04.25

Saturday : 2014.04.26

Sunday : 2014.04.27