Changeset 9050

Timestamp:

Sep 28, 2006, 4:56:08 PM (20 years ago)

Author:

Paul Price

Message:

Removed excess step (detrend_poststack_imfile.pl) by having everything
go through normalisation, and having the normalisation application also
do binning.

File:

• trunk/doc/ipptools/dettool.txt (modified) (12 diffs)

trunk/doc/ipptools/dettool.txt

@@ -60 +60 @@
 to detProcessedImfiles:
 
-        dettool -addprocessed -det_id <det_id> -exp_id <exp_id> -class_id <class_id> -uri <processed_uri> -recip <ppImage_recipe> -proc_bg <stat> -proc_bg_stdev <stat> -proc_bg_mean_stdev <stat> -b1_uri <binned1_uri> -b2_uri <binned2_uri>
+        dettool -addprocessed -det_id <det_id> -exp_id <exp_id> -class_id <class_id> -uri <processed_uri> -recip <ppImage_recipe> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat> -b1_uri <binned1_uri> -b2_uri <binned2_uri>
 
 In practise, the above "ppImage" and "dettool -addprocessed" commands
@@ -85 +85 @@
 the imfile statistics) is added into the database for the exposure:
 
-        dettool -addprocessedexp -exp_id <exp_id> -proc_bg <stat> -proc_bg_stdev <stat> -proc_bg_mean_stdev <stat> -uri <jpeg_uri> -jpeg_uri <jpeg_uri>
+        dettool -addprocessedexp -exp_id <exp_id> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat> -uri <jpeg_uri> -jpeg_uri <jpeg_uri>
 
 In practise, the above "dettool -processed", "ppImage" and "dettool
@@ -125 +125 @@
 
 Once the stack has been completed, "dettool -addstacked" adds a
-stacked class_id and and optionally marks it for normalization:
-
-        dettool -addstacked [-pleasenormalize] -det_id <det_id> -iter <iter> -class_id <class_id> -recip <recipe> -uri <stacked_uri> -norm_bg <stat> -norm_bg_stdev <stat> -norm_bg_mean_stdev <stat>
-
-The "-pleasenormalize" flag indicates that the normalisation steps are
-to be performed (in the case of flat-fields); if this flag is missing,
-then the normalisation steps are to be skipped and the stacked imfiles
-are inserted in table of normalised imfiles.  The statistics fields 
-have "norm" since these statistics will be used to determine the
-normalisations.
+stacked class_id:
+
+        dettool -addstacked -det_id <det_id> -iter <iter> -class_id <class_id> -recip <recipe> -uri <stacked_uri> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat>
 
 In practise, the above three commands ("dettool -processed", "ppMerge"
@@ -155 +148 @@
 
 This returns the complete list of processed imfiles for the specified
-det_id, providing exposure id, class id, statistic, which we use in
-the normalisation:
+det_id, providing exposure id, class id, statistic, detrend type,
+which we use in the normalisation:
 
         ppNormCalc INPUT.mdc
@@ -175 +168 @@
 "ppNorm" and "dettool -addnormstat") will be performed by a Perl
 script, "detrend_norm_calc.pl".
+
+If the detrend type is not suitable for normalisation, then
+normalisation values of unity will be generated without solving the
+matrix with ppNormCalc.
 
 *** To do: possible upgrade path is to have "dettool -addnormstat"
@@ -193 +190 @@
 detNormalizedImfile.
 
-For each imfile, we apply the normalisation:
-
-        ppArith -file1 <in_uri> -operation * -constant <normalisation> -out <out_uri>
+For each imfile, we apply the normalisation, generate binned images
+and statistics:
+
+        ppNormApply -file <in_uri> <out_uri> -norm <normalisation> -stat <statictics_file>
 
 The normalised imfile is then inserted into detNormalizedImfile:
 
-        dettool -addnormalizedimfile -det_id <det_id> -iter <iter> -class_id <class_id> -uri <out_uri>
+        dettool -addnormalizedimfile -det_id <det_id> -iter <iter> -class_id <class_id> -uri <out_uri> -b1_uri <b1_uri> -b2_uri <b2_uri> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat>
 
 In practise, the above two commands ("ppArith" and "dettool
@@ -208 +206 @@
 --------------------------------------------------------------------------------
 
-<<< Normalisation has completed. The skip introduced >>>
-<<< by leaving out -pleasenormalize lands here.      >>>
-
---------------------------------------------------------------------------------
-
-Pan-Tasks regularly runs:
-
-        dettool -stackedimfile
-
-which returns a list of stack imfiles which have been normalised (or
-had normalisation skipped).  This includes the exposure id, class id
-and uri.
-
-For each imfile, we measure statistics and bin the images:
-
-        ppImage -file <in_uri> <out_uri> -recipe PPIMAGE PPIMAGE_BIN -stat <stats_file>
-
-The resultant statistics and the binned images are pushed into the database:
-
-        dettool -addstackimfilesummary -exp_id <exp_id> -class_id <class_id> -b1_uri <b1_uri> -b2_uri <b2_uri> -stack_bg <stat> -stack_bg_stdev <stat> -stack_bg_mean_stdev <stat>
-
-In practise, the above commands ("ppImage" and "dettool
--addnormimfilesummary") will be performed by a Perl script,
-"detrend_poststack_imfile.pl".
-
---------------------------------------------------------------------------------
-
-Pan-Tasks regularly runs:
-
-        dettool -normalizedexp
+Pan-Tasks regularly runs:
+
+        dettool -normexp
 
 which returns a list of stack exposures which have had normalised
@@ -244 +215 @@
 For each exposure, we get the list of component imfiles:
 
-        dettool -stackimfile -exp_id <exp_id>
+        dettool -normimfile -exp_id <exp_id>
 
 This includes the statistics (bg, bg_stdev, bg_mean_stdev) and the
@@ -254 +225 @@
 The statistics and JPEG uri are pushed into the database:
 
-        dettool -addstackexpsummary -exp_id <exp_id> -jpeg1_uri <jpeg1_uri> -jpeg2_uri <jpeg2_uri> -norm_bg <stat> -norm_bg_stdev <stat> -norm_bg_mean_stdev <stat>
+        dettool -addnormexp -exp_id <exp_id> -j1_uri <jpeg1_uri> -j2_uri <jpeg2_uri> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat>
 
 In practise, the above three commands ("dettool -stackimfile",
-"ppImage" and "dettool -addnormexpsummary") will be performed by a
-Perl script, "detrend_poststack_exp.pl".
+"ppImage" and "dettool -addnormexp") will be performed by a
+Perl script, "detrend_norm_exp.pl".
 
 --------------------------------------------------------------------------------
@@ -278 +249 @@
 The residual is then inserted into the database:
 
-        dettool -addresidimfile -det_id <det_id> -exp_id <exp_id> -class_id <class_id> -recip <recipe> -resid_bg <stat> -resid_bg_stdev <stat> -resid_bg_mean_stdev <stat> -b1_uri <b1_fits> -b2_uri <b2_fits> 
+        dettool -addresidimfile -det_id <det_id> -exp_id <exp_id> -class_id <class_id> -recip <recipe> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat> -b1_uri <b1_fits> -b2_uri <b2_fits> 
 
 In practise, the above two commands ("ppImage" and "dettool
@@ -313 +284 @@
 problems; these could be removed by other means):
 
-        dettool -addresidexp -det_id <det_id> -iter <iter> -exp_id <exp_id> -recip <recipe> -b1_uri <uri_jpeg1> -b2_uri <uri_jpeg2> -resid_bg <stat> -resid_bg_stdev <stat> -resid_bg_mean_stdev <stat> [-reject]
+        dettool -addresidexp -det_id <det_id> -iter <iter> -exp_id <exp_id> -recip <recipe> -j1_uri <uri_jpeg1> -j2_uri <uri_jpeg2> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat> [-reject]
 
 where the "-reject" flag indicates that the exposure is not to be
@@ -350 +321 @@
 iteration as a whole we can run:
 
-        dettool -adddetrunsummary -det_id <det_id> -iter <iter> -det_bg <stat> -det_bg_stdev <stat> -det_bg_mean_stdev <stat> [-accept]
+        dettool -adddetrunsummary -det_id <det_id> -iter <iter> -bg <stat> -bg_stdev <stat> -bg_mean_stdev <stat> [-accept]
 
 Which allows per detrend run iteration summary statics to be