Changes between Version 3 and Version 4 of PS1_IPP_Czarlog_20190114

Timestamp:

Jan 17, 2019, 12:38:31 PM (8 years ago)

Author:

Mark Huber

Comment:

--

PS1_IPP_Czarlog_20190114

@@ -5 +5 @@
 
 === Monday : 2019.01.14 ===
+ * MEH: very old WW mops diffs and WS catalogs on datastore -- clean out (dsreg for WW and distribution goto_cleaned) to past couple months so page loads more easily for checking
+ * MEH: PS1 subset/interleaved OSS chunks interfering with normal MOPS operations -- comment string was changed from OSS.R% to EU.r% and manually alter nightly_science.config to proceed with processing
+ * MEH: PS2 back on sky -- WW diffs not queuing -- looks like the dark done in afternoon didn't have the expected comment of darks_afternoon?
+{{{
+ENGINEERING     42707   o8498h0002d     2019-01-15 03:25:58     CLEAR.00002     30      darks_morn - Dark exp 30s 1 of 1
+
+nightly_science.pl --dbname gpc2 --camera GPC2 --verbose  --queue_diffs --date 2019-01-15
+}}}
+  * setup crontab to occasionally run this...
+ * MEH: PS2 WWdiffs taking very long time to run on a few skycells per diff_id -- manually killing ppSub on long running jobs and set quality 42 to allow the rest of the skycells to get to MOPS  
   
-=== Tuesday : YYYY.MM.DD ===
+
+=== Tuesday : 2019.01.15 ===
+ * MEH: messed up PS1 EU config that blocked processing on the last OSS chunk, fixed for tonight tests okay -- new nightly processing label EU.nightlyscience added to stdscience with priority just below %SS
+ * MEH: PS2 WWdiffs still regularly taking >1-2hrs and delaying processing (also using excessive memory that overloads the heavily loaded nodes) -- create crontab to kill jobs after 3600s
+  * manually add c2 group a 2-3x (more could also add more memory overload)
+  * looks like excessive detections on known odd group of cells from last Fall
 
 === Wednesday : 2019.01.16 ===
  * MEH: bad exposure (GPC1 crash, multiple ota missing) stuck in summitcopy -- o8499g0264o -- should set to drop
  * MEH: another example of pantasks autorestart borking job in summitcopy at toadvance state -- o8500g0002d -- manually stop & restart summitcopy again is one way  to clear
-=== Thursday : YYYY.MM.DD ===
+
+
+=== Thursday : 2019.01.17 ===
+
 
 === Friday : YYYY.MM.DD ===