Changeset 11740

Timestamp:

Feb 9, 2007, 3:40:02 PM (19 years ago)

Author:

Paul Price

Message:

Updating phase 4 work flow.

File:

• trunk/doc/ipptools/p4tool.txt (modified) (1 diff)

trunk/doc/ipptools/p4tool.txt

@@ -1 @@
---------------------------------------------------------------------------------
-
-The "p4tool -definerun" command creates a new p4Run for a single cell. A
-unique identifier for the run, or a p4Run ID, is returned.
-
-    p4tool -definerun -survey_mode <str> -filter <str> -skycell_id <str> -tess_id <str>  -ra <f64> -decl <f64> -input_ss <str> -output_ss <str> -imfiles <s32>
-
-    p4tool -addinputimfile -p4_id <str> -exp_tag <str> -p3_version <s32> -class_id <str>
-
---------------------------------------------------------------------------------
-
-        p4tool -towarpedimfile [-p4_id <str>]
-
-Blocks until the appropriate # of inputs have been registered in a given
-p4run.  Returns a list of imfiles along with their skycell_id & tessalation id
-that have not yet been warped.
-
-        p4tool -addwarpedimfile -p4_id <str> -exp_tag <str> -p3_version <s32>
--class_id <str> -uri <str> [-bg ...]
-
-Registers a warped imfile with the database.
-
-        p4tool -warpedimfile [-p4_id <str>]
-
-Lists warped imfiles.
-
---------------------------------------------------------------------------------
-
-        p4tool -tostackedimfile -p4_id <str>
-
-List a complete set of class_ids for a given p4run that are ready to be
-stacked.
-
-        p4tool -addstackedimfile -p4_id <str> -exp_tag <str> -p3_version <s32>
--class_id <str> -uri <str> [-bg ...]
-
-Registered a stacked imfiles with the database
-
-        p4tool -stackedimfile [-p4_id <str>]
-
-Lists stacked imfiles.
-
---------------------------------------------------------------------------------
-
-        p4tool -todiffimfile [-p4_id <str>]
-
-Lists stacked imfiles along with the uri for an imfile to use for
-differencing.
-
-        p4tool -adddiffimfile -p4_id <str> -exp_tag <str> -p3_version <s32>
--class_id <str> -uri <str> [-bg ...]
-
-Registered a difference imfiles with the database.
-
-        p4tool -diffimfile [-p4_id <str>]
-
-Lists difference imfiles.
-
---------------------------------------------------------------------------------
+Give me a list of all pending exposures which could have phase 4 run:
+        p4tool -query -survey_mode <str> -filter <str> -ra <f64> -decl <f64> (plus other search options)
+
+The "p4tool -definerun" command creates a new p4Run for an exposure. A
+unique identifier for the run, or a p4Run ID, is returned:
+
+        p4tool -definerun -exp_tag <str> -p3_ver <s32>
+
+Give me all p4runs to have overlaps calculated:
+        p4tool -tooverlap
+
+Calculate overlaps for each exposure:
+        phase4_overlap.pl --p4_id <str>
+{
+        Get list of component imfiles for exposure:
+                p4tool -imfile -p4_id <str>
+
+        (Calculates, in some as-yet unknown way the overlaps between imfiles
+and skycells)
+
+        Set the relationship between imfiles and skycells:
+                p4tool -overlap -p4_id <str> -info FILE
+        where FILE contains:
+                imfile.0        STR     skycell.123 skycell.456
+                imfile.1        STR     skycell.789
+        OR:
+                imfile.0        MULTI
+                imfile.0        STR     skycell.123
+                imfile.0        STR     skycell.456
+                imfile.1        STR     skycell.789
+}
+
+Give me all skycells that need to have data warped to them:
+        p4tool -towarped
+
+Do the warp for each skycell:
+        phase4_warp.pl --p4_id <str> --skycell <str>
+{
+        Get list of imfiles for skycell:
+                p4tool -imfile -p4_id <str> -skycell <str>
+
+        Do the warping:
+                pswarp <something>
+
+        Register the skycell as done warping:
+                p4tool -warped -p4_id <str> -skycell <str> -uri <str>
+}
+
+================================================================================
+
+Give me a list of all skycells which could have subtractions performed
+(more than two warps):
+        p5tool -query (some search parameters, including RA, Dec, seeing)
+
+Register an exposure for subtraction processing.  We need to specify which warp
+is the input and which is the template.
+        p5tool -definerun -skycell <str> -input_p4_id <str> -template_p4_id <str>
+
+Give me all skycells that need to be subtracted:
+        p5tool -pending
+
+Do the subtraction for each skycell:
+        phase5_subtract.pl --p5_id <str> --in_uri <str> --template_uri <str>
+{
+        Do the subtraction:
+                pois in_uri.fits template_uri sub.fits <other stuff>
+
+        Register the subtraction as done:
+                p5tool -done -p5_id <str> -uri <str>
+}
+
+================================================================================
+
+[Here I make some assumptions about how magic works]
+
+Get a list of exposures for which magic processing is possible
+        magictool -query (some search options)
+
+There may be multiple subtractions generated out of each exposure.
+Given an exposure, what subtractions do I have available?
+
+        magictool -options -exp_tag <str>
+
+Register an exposure for magic processing, specifying a subtraction set:
+        magictool -definerun -exp_tag <str> -subs FILE
+
+        where FILE contains something like:
+                skycell.123       STR       p5id.1
+                skycell.456       STR       p5id.2
+                skycell.789       STR       p5id.3
+
+Give me all skycells that need to have magic performed:
+        magictool -pendingskycell
+
+Do magic for each skycell:
+        magic_skycell.pl --magic_id <str> --skycell <str> --uri <str>
+{
+        Do streak finding on skycell
+
+        Register sky cell as done:
+                magictool -doneskycell -magic_id <str> -skycell <str> -uri <str>
+}
+
+Give me all imfiles that need to have magic performed:
+        magictool -pendingimfile
+
+Do magic for each imfile:
+        magic_imfile.pl --magic_id <str> --class_id <str>
+{
+        Get list of all magic1 output relevant to this imfile
+                magictool -skycell -magic_id <str> -class_id <str>
+
+        Do streak finding for imfile
+
+        Register imfile as done:
+                magictool -doneimfile -magic_id <str> -class_id <str> -uri <str>
+}
+
+[Might be a mid-level process or processes, halfway between imfiles and
+full exposures, but for now, assume there's not, or that it can all be
+done quickly at the exposure level.]
+
+Give me all exposures that need to have magic performed:
+        magictool -pendingexp
+
+Do magic for each exposure:
+        magic_exp.pl --magic_id <str>
+{
+        Get list of all magic2 output relevant to this exposure
+                magictool -imfile -magic_id <str>
+
+        Do streak finding on exposure
+
+        Register exposure as done:
+                magictool -doneexp -magic_id <str> -uri <str>
+}
+
+
+[Need to generate masks, but we need to decide at what level: imfile, or
+skycell?  If imfile, we need to do the warp over again.  If skycell, we
+can never give out the flattened CCD images unless they've been warped. 
+How about doing both --- it's not too hard.]
+
+
+Give me all imfiles that need a magic mask:
+        magictool -pendingmaskimfile
+
+Generate mask for each imfile:
+        magic_mask_imfile.pl --magic_id <str> --class_id <str> --uri <str>
+{
+        Make mask
+
+        Register mask:
+                magictool -donemaskimfile -magic_id <str> -class_id <str> -uri <str>
+}
+
+Give me all skycells that need a magic mask:
+        magictool -pendingmaskskycell
+
+Generate mask for each skycell:
+        magic_mask_skycell.pl --magic_id <str> --skycell <str> --uri <str>
+{
+        Make mask
+
+        Register mask:
+                magictool -donemaskskycell -magic_id <str> -skycell <str> -uri <str>
+}
+
+Give me a list of all exposures which are clean (so we can release):
+        magictool -cleanexp (some search criteria)
+
+Give me a list of all imfiles which are clean (so we can release):
+        magictool -cleanimfile (some search criteria)
+
+Give me a list of all skycells which are clean (so we can release):
+        magictool -cleanskycell (some search criteria)
+
+================================================================================
+
+Give me a list of all skycells for which combination processing is possible:
+        p6tool -query -min <s32> (some other search parameters, including ra, dec, time range)
+
+Register skycells for combination processing:
+        p6tool -definerun -skycell <stR> -p4_id <str> -p4_id <str> -p4_id <str>
+
+Give me all pending operations
+        p6tool -pending
+
+Run phase 6:
+        phase6.pl -p6_id <str>
+{
+        Give me the list of all inputs for this run:
+                p6tool -inputs -p6_id <str>
+
+        Combine:
+                ppStac output.fits input1.fits input2.fits ...
+
+        Register the completion:
+                p6tool -done -p6_id <str> -uri <str>
+}
+
+Give me a list of all stacks (for release):
+        p6tool -list <some search criteria>
+