Changes between Version 2 and Version 3 of NightlyScienceV2

Timestamp:

Dec 21, 2010, 3:42:36 PM (16 years ago)

Author:

watersc1

Comment:

--

NightlyScienceV2

@@ -20 +20 @@
 == Configuration ==
  
- The configuration is stored in {{{ippconfig/recipes/nightly_science.config}}} and has a standard IPP metadata format.
+The configuration is stored in {{{ippconfig/recipes/nightly_science.config}}} and has a standard IPP metadata format.
 
 === CLEAN_MODES ===
@@ -62 +62 @@
 These rules define how the detrend verify runs are queued.  Each has a {{{NAME}}} and use {{{DETTYPE,EXPTYPE}}} to determine the type of verify run to generate.  The reference detrend is set using the {{{REF_ID,REF_ITER}}} values.  If a {{{FILTER}}} value is set, then only exposures in that filter are used for the verification.  The number of exposures to use in the verification process is set by {{{MAX_EXP}}}.
 
-== Faults ==
-
 == Manual Operation Modes ==
 
+These commands show how to investigate the current status of the system, and how to manually fix certain issues. This assumes we are working with gpc1 data.  All dates and times are UTC unless otherwise noted.
+
+=== Look at the current status of observations/registration ===
+
+ This should probably be a mode to regtool with date and class_id options:
+{{{
+SQL: 
+SELECT DISTINCT exp_name, 
+  summitExp.dateobs AS registered, summitExp.imfiles, summitExp.fault AS summit_fault,
+  pzDownloadExp.state AS download_state, newExp.state AS newExp_state, rawExp.state AS rawExp_state,    
+  rawImfile.data_state AS imfile_state,
+  newExp.exp_id, summitExp.exp_type, 
+  rawExp.dateobs, rawExp.obs_mode AS obs_mode, rawExp.object AS object,
+  rawImfile.burntool_state AS burntool_state, rawImfile.class_id,
+  (pzDownloadExp.state = 'stop') AS is_downloaded,(rawImfile.burntool_state IS NOT NULL) AS is_registered 
+FROM summitExp LEFT JOIN pzDownloadExp USING(exp_name) LEFT JOIN newExp ON exp_name = tmp_exp_name         
+  LEFT JOIN rawExp USING(exp_id, exp_name) LEFT JOIN rawImfile USING(exp_name)
+WHERE rawImfile.dateobs > '2010-12-18T00:00:00' AND class_id = 'XY45';
+}}}
+
+=== Look at the current list of pending imfiles ===
+
+{{{ regtool -pendingimfile -dbname gpc1 }}}
+
+=== Look at the current list of pending burntool imfiles ===
+
+{{{ regtool -pendingburntoolimfile -date YYYY-MM-DD -valid_burntool 14 -dbname gpc1 }}}
+
+=== Look at the current list of pending exposures ===
+
+{{{ regtool -pendingexp -dbname gpc1 }}}
+
+=== Revert an imfile that has failed registration ===
+
+This has not happened before, but is increasingly likely as we start processing while still downloading and registering data. There is a revert task in the {{{registration}}} pantasks, but it is off by default.
+
+{{{ regtool -revertprocessedimfile -dbname gpc1 }}}
+
+=== Manually queue a burntool run ===
+
+This writes the burntool table and logfile into nebulous, and sets the database values as described above upon success.
+
+{{{ ipp_apply_burntool_single.pl --exp_id EXP_ID --class_id CLASS_ID --this_uri URI_FOR_EXP_ID --previous_uri URI_FOR_PREVIOUS_EXPOSURE --dbname gpc1 }}}
+
+=== Look at the status of burntool for each science survey ===
+
+{{{ nightly_science.pl --burntool_status --date YYYY-MM-DD }}}
+
+=== Check to see if stacks can be made ===
+
+{{{ nightly_science.pl --check_stacks --date YYYY-MM-DD }}}
+{{{ nightly_science.pl --check_stacks --date YYYY-MM-DD --this_target_only MD99 --this_filter_only g.00000 }}}
+{{{ nightly_science.pl --check_stacks --date YYYY-MM-DD --this_target_only MD99 --force_stack_count }}}
+
+=== Queue a set of stacks ===
+
+{{{ nightly_science.pl --queue_stacks --date YYYY-MM-DD }}}
+
+=== Check to see if diffs can be made ===
+
+{{{ nightly_science.pl --check_diffs --date YYYY-MM-DD }}}
+{{{ nightly_science.pl --check_diffs --date YYYY-MM-DD --this_target_only MD99 --this_filter_only g.00000 }}}
+
+=== Queue a set of diffs ===
+
+{{{ nightly_science.pl --queue_diffs --date YYYY-MM-DD }}}
+{{{ nightly_science.pl --queue_diffs --date YYYY-MM-DD --this_target_only MD99 --this_filter_only g.00000 }}}
+
+=== Clean old data ===
+
+{{{ nightly_science.pl --clean_old --date YYYY-MM-DD }}}
+
+=== Look at what data would be cleaned ===
+
+{{{ nightly_science.pl --clean_old --debug --date YYYY-MM-DD }}}
+
+
+
+
+
+
+
+
+
+
+