Changes between Version 22 and Version 23 of PS1_IPP_CzarLog_20110418

Timestamp:

Apr 21, 2011, 9:47:59 AM (15 years ago)

Author:

bills

Comment:

--

PS1_IPP_CzarLog_20110418

@@ -4 +4 @@
 (Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])
 
-=== Monday : 2011.04.18 ===
+=== Monday : 2011-04-18 ===
  * 10:36 Bill queued stacks for STS.refstack.20110418
  * 11:08 CZW: Noticed that burntool crashed last night, leaving an exposure in state check_burntool instead of properly processing.  Reset state to pending_burntool, and registration started up again to finish. Registration logfile (neb://ipp047.0/gpc1/20110418/o5669g0493o.326098/o5669g0493o.326098.reg.ota67.log) suggests a database error.
@@ -20 +20 @@
  * 13:55 distribution seemed sluggish so Bill restarted it.
  * 14:18 CZW: concern about diskspace prompted me to look at the replication pantasks to see what shuffle was doing.  The replication pantasks was not doing anything, because it failed to load nebulous.site.pro as part of the setup macro.  It appears that this file was not transferred over between tag changes, so I've copied the version from /data/ippc18.0/home/ipp/psconfig/ipp-20110218.lin64/share/pantasks/modules into the working tag.  This has unstuck the shuffle task.
-=== Thursday : YYYY.MM.DD ===
+=== Thursday : 2011-04-21 ===
 
  * 02:06 CZW: noticed registration was lagging. register_imfile complained about a read only filesystem: neb://ipp005.0/gpc1/20110421/o5672g0156o/o5672g0156o.ota02.burn.log . Resetting the data_state for that imfile seems to have cleared subsequent exposures, but o5672g0156o seems to be stuck and unable to register the exposure correctly. I'll look at it tomorrow.
@@ -27 +27 @@
 Bill is acting czar today.
 
- * 06:20 warp was stuck with a book full of entries in state DONE. warp.reset seems to have cleared the problem. pcontrol is using a whole cpu which is often a sign of trouble. I've turned camera and stack off. Once the stacks that are running finish I will restart stdscience.
+ * 06:20 warp was stuck with a book full of entries in state DONE. warp.reset seems to have cleared the problem. pcontrol is using a whole CPU which is often a sign of trouble. I've turned camera and stack off. Once the stacks that are running finish I will restart stdscience.
  * 06:35 stdscience pantasks restarted.
  * 06:54 ipp005 console said [761674.464312] Kernel panic - not syncing: Attempted to kill init! so I power cycled it.
  * 07:05 many diff failures. ran -revertdiffskyfile and then turned diff.revert.off to investigate. ipp005 is back up.
+ * 09:45 The diff failures were due to ipp005 being unavailable. It looks like we may have some MD stack files that are not replicated.
 
+=== Friday : 2011-04-22 ===
 
+=== Saturday : 2011-04-22 ===
 
-=== Friday : YYYY.MM.DD ===
+=== Sunday : 2011-04-23 ===
 
-=== Saturday : YYYY.MM.DD ===
-
-=== Sunday : YYYY.MM.DD ===
-