Changeset 9042

Timestamp:

Sep 28, 2006, 3:08:01 PM (20 years ago)

Author:

Paul Price

Message:

Adding modes for JPG creation (processed and stacked exposures).
Renaming the statistics triplets to make clear that they are
different at different stages.

File:

• trunk/doc/ipptools/dettool.txt (modified) (8 diffs)

trunk/doc/ipptools/dettool.txt

@@ -55 +55 @@
         ppImage -file <imfile_uri> <processed_uri> -recipe PPIMAGE <ppImage_recipe> -stat <temporary_statistics_file>
 
-One processing has been performed, "dettool -addprocessed" adds an
-imfile to detProcessedImfiles:
-
-        dettool -addprocessed -det_id <det_id> -exp_id <exp_id> -class_id <class_id> -uri <processed_uri> -recip <ppImage_recipe> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat>
+In addition to removing the nominated instrumental signatures, this
+also generates binned images to create a JPEG in the next step.  Once
+processing has been performed, "dettool -addprocessed" adds an imfile
+to detProcessedImfiles:
+
+        dettool -addprocessed -det_id <det_id> -exp_id <exp_id> -class_id <class_id> -uri <processed_uri> -recip <ppImage_recipe> -proc_bg <stat> -proc_bg_stdev <stat> -proc_bg_mean_stdev <stat> -b1_uri <binned1_uri> -b2_uri <binned2_uri>
 
 In practise, the above "ppImage" and "dettool -addprocessed" commands
-will be performed by a script, "detrend_process.pl".
+will be performed by a script, "detrend_process_imfile.pl".
+
+--------------------------------------------------------------------------------
+
+Pan-Tasks regularly runs:
+
+        dettool -processedexp
+
+This command returns a list of exposures that have had all imfiles
+processed.  This list includes the exposure id.  Using this exposure
+id, a list of imfiles may be gathered using:
+
+        dettool -processed -exp_id <exp_id>
+
+The output includes b1_uri, b2_uri, bg, bg_stdev and bg_mean_stdev for
+the processed imfiles.  These are combined into a JPEG by:
+
+        ppImage -list <file_list> <output_uri> -recipe PPIMAGE PPIMAGE_J
+
+The resultant image, along with summary statistics (calculated from
+the imfile statistics) is added into the database for the exposure:
+
+        dettool -addprocessedexp -exp_id <exp_id> -proc_bg <stat> -proc_bg_stdev <stat> -proc_bg_mean_stdev <stat> -uri <jpeg_uri> -jpeg_uri <jpeg_uri>
+
+In practise, the above "dettool -processed", "ppImage" and "dettool
+-addprocessedexp" commands will be performed by a script,
+"detrend_process_exp.pl".
 
 --------------------------------------------------------------------------------
@@ -99 +127 @@
 stacked class_id and and optionally marks it for normalization:
 
-        dettool -addstacked [-pleasenormalize] -det_id <det_id> -iter <iter> -class_id <class_id> -recip <recipe> -uri <stacked_uri> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat> 
+        dettool -addstacked [-pleasenormalize] -det_id <det_id> -iter <iter> -class_id <class_id> -recip <recipe> -uri <stacked_uri> -norm_bg <stat> -norm_bg_stdev <stat> -norm_bg_mean_stdev <stat>
 
 The "-pleasenormalize" flag indicates that the normalisation steps are
 to be performed (in the case of flat-fields); if this flag is missing,
 then the normalisation steps are to be skipped and the stacked imfiles
-are inserted in table of normalised imfiles.
-
-In practise, the above three commands ("dettool -processed
--wholestack", "ppMerge" and "dettool -addstacked") will be performed
-by a Perl script, "detrend_stack.pl".
+are inserted in table of normalised imfiles.  The statistics fields 
+have "norm" since these statistics will be used to determine the
+normalisations.
+
+In practise, the above three commands ("dettool -processed", "ppMerge"
+and "dettool -addstacked") will be performed by a Perl script,
+"detrend_stack.pl".
 
 
@@ -144 +174 @@
 In practise, the above three commands ("dettool -processed -unmask",
 "ppNorm" and "dettool -addnormstat") will be performed by a Perl
-script, "detrend_calc_norm.pl".
+script, "detrend_norm_calc.pl".
 
 *** To do: possible upgrade path is to have "dettool -addnormstat"
@@ -165 +195 @@
 For each imfile, we apply the normalisation:
 
-        ppNormApply -file <in_uri> <out_uri> -norm <normalisation>
+        ppArith -file1 <in_uri> -operation * -constant <normalisation> -out <out_uri>
 
 The normalised imfile is then inserted into detNormalizedImfile:
@@ -171 +201 @@
         dettool -addnormalizedimfile -det_id <det_id> -iter <iter> -class_id <class_id> -uri <out_uri>
 
-In practise, the above two commands ("ppNormApply" and "dettool
+In practise, the above two commands ("ppArith" and "dettool
 -addnormalizedimfile") will be performed by a Perl script,
-"detrend_apply_norm.pl".
-
+"detrend_norm_apply.pl".
+
+
+--------------------------------------------------------------------------------
+
+<<< Normalisation has completed. The skip introduced >>>
+<<< by leaving out -pleasenormalize lands here.      >>>
+
+--------------------------------------------------------------------------------
+
+Pan-Tasks regularly runs:
+
+        dettool -stackedimfile
+
+which returns a list of stack imfiles which have been normalised (or
+had normalisation skipped).  This includes the exposure id, class id
+and uri.
+
+For each imfile, we measure statistics and bin the images:
+
+        ppImage -file <in_uri> <out_uri> -recipe PPIMAGE PPIMAGE_BIN -stat <stats_file>
+
+The resultant statistics and the binned images are pushed into the database:
+
+        dettool -addstackimfilesummary -exp_id <exp_id> -class_id <class_id> -b1_uri <b1_uri> -b2_uri <b2_uri> -stack_bg <stat> -stack_bg_stdev <stat> -stack_bg_mean_stdev <stat>
+
+In practise, the above commands ("ppImage" and "dettool
+-addnormimfilesummary") will be performed by a Perl script,
+"detrend_poststack_imfile.pl".
+
+--------------------------------------------------------------------------------
+
+Pan-Tasks regularly runs:
+
+        dettool -normalizedexp
+
+which returns a list of stack exposures which have had normalised
+imfile summaries added.  This includes the exposure id.
+
+For each exposure, we get the list of component imfiles:
+
+        dettool -stackimfile -exp_id <exp_id>
+
+This includes the statistics (bg, bg_stdev, bg_mean_stdev) and the
+URIs for the binned images.  From these, we generate summary
+statistics for the exposure, and produce a JPEG:
+
+        ppImage -list <input_list> <out_uri> -recipe PPIMAGE PPIMAGE_J
+
+The statistics and JPEG uri are pushed into the database:
+
+        dettool -addstackexpsummary -exp_id <exp_id> -jpeg1_uri <jpeg1_uri> -jpeg2_uri <jpeg2_uri> -norm_bg <stat> -norm_bg_stdev <stat> -norm_bg_mean_stdev <stat>
+
+In practise, the above three commands ("dettool -stackimfile",
+"ppImage" and "dettool -addnormexpsummary") will be performed by a
+Perl script, "detrend_poststack_exp.pl".
 
 --------------------------------------------------------------------------------
@@ -194 +278 @@
 The residual is then inserted into the database:
 
-        dettool -addresidimfile -det_id <det_id> -exp_id <exp_id> -class_id <class_id> -recip <recipe> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat> -b1_uri <b1_fits> -b2_uri <b2_fits> 
+        dettool -addresidimfile -det_id <det_id> -exp_id <exp_id> -class_id <class_id> -recip <recipe> -resid_bg <stat> -resid_bg_stdev <stat> -resid_bg_mean_stdev <stat> -b1_uri <b1_fits> -b2_uri <b2_fits> 
 
 In practise, the above two commands ("ppImage" and "dettool
 -addresidimfile") will be performed by a Perl script,
-"detrend_create_resid.pl".
+"detrend_resid.pl".
 
 --------------------------------------------------------------------------------
@@ -229 +313 @@
 problems; these could be removed by other means):
 
-        dettool -addresidexp -det_id <det_id> -iter <iter> -exp_id <exp_id> -recip <recipe> -b1_uri <uri_jpeg1> -b2_uri <uri_jpeg2> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat> [-reject]
+        dettool -addresidexp -det_id <det_id> -iter <iter> -exp_id <exp_id> -recip <recipe> -b1_uri <uri_jpeg1> -b2_uri <uri_jpeg2> -resid_bg <stat> -resid_bg_stdev <stat> -resid_bg_mean_stdev <stat> [-reject]
 
 where the "-reject" flag indicates that the exposure is not to be
@@ -266 +350 @@
 iteration as a whole we can run:
 
-        dettool -adddetrunsummary -det_id <det_id> -iter <iter> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat> [-reject]
-
-Which allows per detrend run iteration summary statics to be registered and
-can flag the iteration's results as being acceptable for use as a master.
+        dettool -adddetrunsummary -det_id <det_id> -iter <iter> -det_bg <stat> -det_bg_stdev <stat> -det_bg_mean_stdev <stat> [-accept]
+
+Which allows per detrend run iteration summary statics to be
+registered and can flag the iteration's results as being acceptable
+for use as a master (if -accept is indicated).
 
 Once this has been done, a decision is made whether to make another