IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 3 and Version 4 of PS1_IPP_Czarlog_20120716


Ignore:
Timestamp:
Jul 16, 2012, 1:56:51 PM (14 years ago)
Author:
bills
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PS1_IPP_Czarlog_20120716

    v3 v4  
    66Bill is czar today.
    77 * 10:10 stdscience was stuck. There are several thousand chip runs outstanding but no jobs are getting queued. Czartool indicates that chip processing stopped making progress around 20:15 last night while outstanding warps got finished just after midnight. The book chipPendingImfile had lots of entries in init state but nothing was getting run. I restarted stdscience.
     8 * 13:40 Chris discovered that ippc03 was out of space in /tmp which was causing nebulous requests to fail. This undoubtedly contributed to the processing failures. Bill deleted the log files and restarted the apache servers.
     9 * 13:55 chip reverts are off because we have a number of exposures that are faulted repeatedly due to incorrect burntool state. Chris is working on fixing the burntool states.
    810 
    911=== Tuesday : 2012.07.17 ===