Changeset 13846

Timestamp:

Jun 15, 2007, 5:33:38 AM (19 years ago)

Author:

eugene

Message:

update text

File:

• trunk/doc/manual/manual.tex (modified) (5 diffs)

trunk/doc/manual/manual.tex

@@ -1 @@
- %%% $Id: manual.tex,v 1.5 2007-02-13 01:28:42 jhoblitt Exp $
+ %%% $Id: manual.tex,v 1.6 2007-06-15 15:33:38 eugene Exp $
 \documentclass[panstarrs,spec]{panstarrs}
 
@@ -42 +42 @@
 \section{Overview}
 
-The Pan-STARSS Image Processing Pipeline (IPP) provides a software
-suite used for the reduction of astronomical images.  Although
-designed principally for the Pan-STARRS project, it is highly
-configurable and extensible, and hence highly applicable for other
-projects.
-
-The IPP consists of several software products, built on top of a
-number of extenal libraries and Perl packages (see
-\S\ref{sec:dependencies} for a list).
+The Pan-STARRS Image Processing Pipeline (IPP) is a software suite for
+the reduction of astronomical images.  Although designed principally
+for the Pan-STARRS project, it is highly configurable and extensible,
+and hence highly applicable for other projects.
+
+The IPP consists of a number of elements: programs which perform
+specific image analysis steps; a parallel processing environment in
+which the image processes steps are automatically sequenced and
+tracked; a databasing system for associating and calibrating the
+detections of astronomical objects.  These programs use a number of
+internal and external libraries (see \S\ref{sec:dependencies}),
+simplifying the process of adding additional elements as needed.
+
+The core functionality of the IPP is implemented by NN principal
+programs: \code{psphot} is used for the detection and analysis of
+objects in astronomical images; \code{psastro} performs the
+astrometric calibration of the images; \code{ppImage} is used for a
+wide range of single image analysis step, include image detrending and
+the generation of Q/A images and plots; \code{ppMerge} combines
+collections of input detrend images into high-quality masters;
+\code{pswarp} transforms images between different pixel projections
+and coordinate systems; \code{ppStack} is used to combine multiple
+science images, including outlier rejection which is sensitive to
+varying image quality; \code{ppSub} performs image differencing
+incorporating the variations in seeing with multiple optional
+convolution kernels.
+
+In addition to the image-level analysis, the IPP provides DVO, the
+Desktop Virtual Observatory, a database system for tracking
+astronomical objects and detections.  DVO includes tools for querying
+and manipulating the contents of the object database. It also provides
+a number of tools for performing the analysis of quantities at the
+database level.  These include: \code{relphot}, which performs relative
+photometry and calculates robust ensemble photometry for objects;
+\code{relastro}, which calculates average astrometry quatities for
+objects, including proper motion and parallax, as well as iterative
+improvements to the astrometric calibrations of images injested by the
+database; \code{uniphot}, which is used to calculate consistent
+photometric calibrations and transformations.  
+
+The IPP provides a system for bulk automation of all stages of the
+image analysis process within a parallel processing environment.  The
+parallelization scheme is very light-weight, and makes use of
+distributed UNIX jobs operating on multiple machines within a cluster.
+Process scheduling and distribution of the resulting jobs to the
+parallel cluster is performed by \code{pantasks}.  A set of
+\code{pantasks} scripts, \code{ippTasks}, is used to define the
+processing stages.  
+
+Data flow within the IPP is managed via interaction with a collection
+of database tables, representing the steps of the analysis pipeline.
+The IPP examines the state of these database tables to determine which
+jobs should be performed next.  A set of simple text files
+(\code{dbconfig}) defines the database scheme and are also used to
+automatically generate C code used to query the database tables.
+Higher level command-line programs (\code{ippTools}), built on these
+APIs, are available to both the end user and to \code{pantasks)} to
+examine the state of the pipeline database.  A set of Perl scripts
+(\code{ippScripts}) provide the glue between the individual IPP
+analysis programs and the parallel processing environment.  A
+web-based tool, \code{ippMonitor} provides the user interface for
+monitoring the pipeline and the current status of the data analysis
+process.
+
+\note{this is not needed here: move elsewhere}
 
 \code{psLib} is the Pan-STARRS library, containing a range of
@@ -59 +115 @@
 manipulation.
 
-Built on these libraries are several programs that provide the data
-reduction functionality: \code{psphot} for photometry, \code{psastro}
-for astrometry, \code{ppStats} for image statistics, \code{ppImage}
-for image detrending, \code{ppMerge} for merging detrends and
-\code{ppNorm} for normalising detrends.
-
-We have also built prototype programs for image manipulation
-(combination and subtraction).  \code{pois} and \code{stac} are the
-original prototypes, built directly on \code{psLib}.  We are currently
-(January 2007) moving most of the code into \code{psModules} and
-finalising algorithms.  The result is that these programs work, but do
-not currently have the entire range of functionality that we envision
-for the final product.  \code{pswarp} and \code{ppStac} are the first
-steps toward the final product; they have limited capability and have
-not been tested as thoroughly as other products.
-
-Data flow through the pipeline is achieved through a state-based
-system using a mysql database.  \code{ippdb} provides a library to
-interact with the database, and \code{ippTools} uses this to determine
-pending operations and register completed operations.
-
 We have some Perl modules which are used to facilitate the data flow
 in processing: \code{PS::IPP::Metadata::Config} reads ``metadata
@@ -86 +121 @@
 interprets output lists from the \code{ippTools}; and
 \code{PS::IPP::Config} reads the configuration files.
-
-The data processing operations are performed by the \code{ippScripts},
-written in Perl.  Process scheduling and controlling is achieved using
-\code{panTasks} within the \code{Ohana} project; configurations
-particular for the IPP are in \code{ippTasks}.  Finally, the
-\code{ippMonitor} provides a means of monitoring the pipeline through
-a web server.
 
 %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
@@ -286 +314 @@
 \end{itemize}
 
-
 \subsubsection{psconfig}
 
-\tbd{EAM to document psconfig.}
-
+* pscheckperl : search for and install, if needed, external Perl modules
+* pschecklibs : search for and install, if needed, external C libraries
+* psconfig : set up the UNIX shell environment
+* psbuild : build and install the software
+* psdist : build IPP distributions (requires CVS access)
+* tagsets : tables defining the C and Perl components to be built
+
+The IPP is a large and complex software system.  A major goal of the
+IPP build system is to be user-friendly for those end users which do
+not have root access on their machines.  Using the IPP build tools, it
+is possible to install the complete system as a non-priviledged user.
+The build system also makes it possible to maintain multiple
+simultaneous installations with different versions of the
+software. This latter feature is particularly important for developers
+who need to be able to make tests and comparisons of different
+versions.
+
+\subsubsubsection{UNIX environment}
+
+With the psconfig system, the complete collection of libraries and 
+
+When building software 
 
 \subsubsection{jhbuild}