Changes between Version 6 and Version 7 of NightlyScienceCode

Timestamp:

Jul 2, 2010, 2:33:22 PM (16 years ago)

Author:

watersc1

Comment:

--

NightlyScienceCode

@@ -46 +46 @@
 Both chip and stack mode options support the --this_target_only and --this_filter_only which allow a specific target or filter to be queued without checking the rest.  This is safe because previously queued runs will not be duplicated.
 
+Other modes:
+
+{{{automate_stacks.pl --date YYYY-MM-DD --check_detrends}}}
+
+{{{automate_stacks.pl --date YYYY-MM-DD --queue_detrends}}}
+
+Check and queue detrend verify runs based on the types of detrends defined in the config file.
+
+{{{automate_stacks.pl --date YYYY-MM-DD --check_dqstats}}}
+
+{{{automate_stacks.pl --date YYYY-MM-DD --queue_dqstats}}}
+
+Check and queue dqstats runs to bundle image statistics for the IPP-OTIS interface.
+
+{{{automate_stacks.pl --date YYYY-MM-DD --clean_old --debug}}}
+
+{{{automate_stacks.pl --date YYYY-MM-DD --clean_old}}}
+
+Check and queue old data cleanup based on the rules defined in the configuration file.
+
 == automate_stacks.pl config file == 
+
+The configuration file defines the parameters for four different components:
+
+=== FILTERS ===
+
+This is a simple MULTI metadata object containing the strings of all filters defined.
+
+=== CLEAN_MODES ===
+
+These define control how data from different stages is cleaned.  The parameters are:
+
+|| MODE || STR || Mode name being cleaned. Used to index the methods, and to allow overrides (see below). ||
+|| COMMAND || STR || Command used to queue the data for cleanup. ||
+|| RETENTION_TIME || S16 || Number of days to retain data before cleanup. Values less than one day disable cleaning for that target. ||
+|| ALTERNATE_CMD || STR || Used to mark the format of the options for the cleanup command.||
+||               || A || -dbname $dbname -updaterun -set_state goto_cleaned -full -set_label goto_cleaned -time_stamp_end $cleaning_date ||
+||               || B || -dbname $dbname -updaterun -set_state goto_cleaned -state full -set_label goto_cleaned -label $label ||
+||               || NULL|| -dbname $dbname -updaterun -set_state goto_cleaned -state full -set_label goto_cleaned -label $label -data_group $data_group ||
+
+=== TARGETS ===
 
 Each target in ippconfig/recipes/nightly_science.config has the following parameters:
@@ -56 +96 @@
 || COMMENT || STR || comment from rawExp. Also a LIKE search. ||
 || STACKABLE || BOOL || This flag determines whether or not nightly stacks can be made of this target. ||
+|| CLEAN_MODE || S16 || Using the clean mode names defined in the CLEAN_MODE section, an alternate retention time can be specified for this target/mode combination.||
 
-The known filters are also stored in this file, although this is only used while queueing stacks.
+=== DETRENDS ===
 
-The RETENTION_TIME value stores the number of days data should be stored before being marked for cleanup.  This is commented out (it is part of ns.initday.load), but is fully implemented.
+Nightly detrend verification runs are controlled by the following parameters:
+
+|| NAME || STR || Detrend name. ||
+|| DETTYPE || STR || Value of the det_type column in the rawExp table to select on. ||
+|| EXPTYPE || STR || Value of the exp_type column in the rawExp table to select on. ||
+|| REF_ID  || S64 || det_id of the reference detrend to verify against. ||
+|| REF_ITER|| S32 || iteration of the reference detrend to verify against. ||
+|| FILTER  || STR || Optional filter to select on.||
+|| MAX     || S32 || Optional maximum number of detrend frames to use in this verify run. ||