IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changeset 6434


Ignore:
Timestamp:
Feb 16, 2006, 8:57:51 AM (20 years ago)
Author:
eugene
Message:

changes leading to CDR

Location:
trunk/doc
Files:
12 edited

Legend:

Unmodified
Added
Removed
  • trunk/doc/design/Makefile

    r6167 r6434  
    1 # $Id: Makefile,v 1.15 2006-01-22 09:54:47 eugene Exp $
     1# $Id: Makefile,v 1.16 2006-02-16 18:57:50 eugene Exp $
    22
    33PDFLATEX = env TEXINPUTS=../../latex/inputs:$(TEXINPUTS):.: pdflatex
     
    88        @echo "  targets: srs ssdd scd all"
    99
    10 cdr: ippCDR.pdf
     10scd: ippSCD.pdf
    1111srs: ippSRS.pdf
    1212ssdd: ippSSDD.pdf
    13 scd: ippSCDps1.pdf
    1413
    15 all : srs ssdd scd cdr
     14scd-draft: ippSCDdraft.pdf
     15
     16all : srs ssdd scd
    1617
    1718ippSRS.pdf : ippSRS.tex
     
    2223
    2324%.pdf: %.tex
    24         $(PSLATEX) $*.tex
    25         $(PSLATEX) $*.tex
    26         dvips -z -t letter -o $*.ps $*.dvi
    27         ps2pdf $*.ps $*.pdf
    28         thumbpdf --modes=dvips $*.pdf
    29         $(PSLATEX) $*.tex
    30         dvips -z -t letter -o $*.ps $*.dvi
    31         ps2pdf $*.ps $*.pdf
     25        $(PSLATEX) $*.tex || exit
     26        $(PSLATEX) $*.tex || exit
     27        dvips -z -t letter -o $*.ps $*.dvi || exit
     28        ps2pdf $*.ps $*.pdf || exit
     29        thumbpdf --modes=dvips $*.pdf || exit
     30        $(PSLATEX) $*.tex  || exit
     31        dvips -z -t letter -o $*.ps $*.dvi || exit
     32        ps2pdf $*.ps $*.pdf || exit
    3233        @rm -f $*.ps $*.dvi $*.aux $*.log $*.tbr $*.tbd $*.lof $*.toc $*.tpm body.tmp head.tmp
    3334
  • trunk/doc/design/ippSCD.tex

    r6008 r6434  
    1 %%% $Id: ippSCD.tex,v 1.1 2006-01-16 00:39:24 eugene Exp $
     1%%% $Id: ippSCD.tex,v 1.2 2006-02-16 18:57:50 eugene Exp $
    22\documentclass[panstarrs]{panstarrs}
    33
    44% basic document variables
    55\title{Pan-STARRS PS-1 Image Processing Pipeline}
    6 \subtitle{System/Subsystem Design Description}
    7 \shorttitle{IPP SSDD}
     6\subtitle{System Concept Design}
     7\shorttitle{IPP SCD}
    88\author{Eugene A. Magnier, Paul A. Price, Josh Hoblitt}
    99\audience{Pan-STARRS PMO}
     
    1111\project{Pan-STARRS Image Processing Pipeline}
    1212\organization{Institute for Astronomy}
    13 \version{00}
     13\version{01}
    1414\docnumber{PSDC-430-011}
    1515
     
    2828DR.04     & 2004.04.13 & Most sections fleshed out \\ \hline
    2929DR.05     & 2004.04.29 & Reorganization for consistency \\ \hline
    30 DR.06     & 2004.10.21 & Major revision in prep of PDR \\ \hline
     30DR.06     & 2004.10.21 & Major revision in prep of PDR \\ \hline \hline
     3100        & 2004.11.04 & Incorporation of PDR Comments \\ \hline
     3201        & 2006.01.22 & Name change (SSDD to SCD) before CDR \\ \hline
    3133\RevisionsEnd
    3234
     
    9294\subsection{Document Overview}
    9395
    94 The Pan-STARRS IPP System/Subsystem Design Description (SSDD) contains
    95 the complete design description of the Pan-STARRS PS-1 IPP in order to
     96The Pan-STARRS IPP System Concept Definition (SCD) contains the
     97complete design concepts of the Pan-STARRS PS-1 IPP in order to
    9698achieve the requirements specified by the Pan-STARRS PS-1 IPP Software
    9799Requirements Specification (SRS; PSDC-430-005).  The requirements flow
    98 begun in the SGS and SCD and continued in the SRS is used to guide the
    99 design presented here.
     100begun in the SGS and full Pan-STARRS SCD and continued in the SRS is
     101used to guide the design presented here.
    100102
    101103\subsection{Requirements Definitions}
  • trunk/doc/design/ippSSDD.tex

    r6167 r6434  
    1 %%% $Id: ippSSDD.tex,v 1.7 2006-01-22 09:54:47 eugene Exp $
    21\documentclass[panstarrs]{panstarrs}
    32
     
    1110\project{Pan-STARRS Image Processing Pipeline}
    1211\organization{Institute for Astronomy}
    13 \version{00}
    14 \docnumber{PSDC-430-011}
     12\version{DR.02}
     13\docnumber{PSDC-430-014}
    1514
    1615% allow paragraphs to be listed in TOC for now
     
    2322\RevisionsStart
    2423% version     Date         Description
    25 DR.01     & 2004.01.01 & First draft  \\ \hline
    26 DR.02     & 2004.03.05 & Second draft \\ \hline
    27 DR.03     & 2004.03.25 & Section reorganization \\ \hline
    28 DR.04     & 2004.04.13 & Most sections fleshed out \\ \hline
    29 DR.05     & 2004.04.29 & Reorganization for consistency \\ \hline
    30 DR.06     & 2004.10.21 & Major revision in prep of PDR \\ \hline
     24DR.01     & 2006.01.19 & First draft release to CDR committee \\ \hline
     25DR.02     & 2006.01.22 & Second draft release to CDR committee \\ \hline
    3126\RevisionsEnd
    3227
     
    9691achieve the requirements specified by the Pan-STARRS PS-1 IPP Software
    9792Requirements Specification (SRS; PSDC-430-005).  The requirements flow
    98 begun in the SGS and SCD and continued in the SRS is used to guide the
    99 design presented here.
     93which began in the SGS and Pan-STARRS system SCD, and continued in the
     94IPP SRS and the IPP SCD, is used to guide the design presented here.
    10095
    10196\subsection{Requirements Definitions}
     
    135130PSDC-430-007  &   Pan-STARRS IPP PSLib Supplementary Design Requirements Specification \\ \hline
    136131PSDC-430-010  &   Pan-STARRS IPP Perl Code Conventions \\ \hline
     132PSDC-430-011  &   Pan-STARRS IPP System Concept Definition \\ \hline
    137133PSDC-430-012  &   Pan-STARRS IPP Modules Supplementary Design Requirements Specification \\ \hline
    138134PSDC-430-014  &   Pan-STARRS IPP PS-1 Cluster Support \\ \hline
     
    140136Posix Standard & Open Group Based Specifications Issue 6, IEEE Std 1003.1, 2003 \\
    141137\DocumentsEnd
    142 
    143 \tbd{add the other subsystem SDDs}
    144138
    145139\section{Subsystem Overview}
     
    232226\subsection{System Design Decisions}
    233227
    234 \tbd{add discussion of the AP survey and other survey projects}
    235 
    236228Since Pan-STARRS is a survey project, all data from the telescopes
    237229will be uniformly analyzed by the Pan-STARRS Image Processing Pipeline
     
    263255elements.
    264256
    265 \subsection{Analysis Tasks and Stages}
    266 
    267 \tbd{this discussion is confusing, and even more so in the context of
    268   the PanTasks definition of a 'task'}.
    269 
    270 Specific programs are required to perform the processing steps listed
    271 above.  These can be divided into well-defined analysis stages, each
    272 of which operates on a particular unit of data, such as a single OTA
    273 image or a collection of astronomical objects.  Analysis tasks
    274 representing the different analysis stages are performed on the IPP
    275 computer cluster.  Note the distinction between the generic analysis
    276 {\em stage} and a specific analysis {\em task}.  An analysis stage
    277 represents a type of analysis which is performed, such as the basic
    278 image calibration and object detection analysis.  An analysis task is
    279 a particular realization of an analysis stage, e.g., the analysis of
    280 OTA number 61 from exposure 654321 to produce a specific set of output
    281 data products.  The analysis stages are discussed in detail in
    282 Section~\ref{sec:AnalysisStages}.
     257\subsection{Analysis Stages, Programs, and Libraries}
     258
     259The processing steps listed above can be divided into well-defined
     260analysis stages, each of which operates on a particular unit of data,
     261such as a single OTA image or a collection of astronomical objects.
     262The different analysis stages defined by the IPP are discussed in
     263Sections~\ref{sec:AnalysisStages} - \ref{sec:MiscStages}.  The
     264analysis defined by these analysis stages is further divided into
     265specific programs which perform the particular operation.  Some
     266analysis stages are primarily defined by a single programs; others
     267require multiple programs and substages.  It is also possible for a
     268single program to be involved in multiple analysis stages.  The major
     269IPP analysis programs are discussed in some detail in
     270Section~\ref{sec:AnalysisPrograms}.  Other important programs in the
     271collection called \code{ippTools} are used to define the analysis
     272stages and their interrelationships (Section~\ref{sec:ipptools}).  The
     273IPP software hierarchy is further divided into \code{psModules}
     274(representing complex analysis operations, often astronomical in
     275nature) and \code{psLib}, the data and analysis foundation library
     276(Section~\ref{sec:SoftwareHierarchy}).
     277
     278Analysis tasks representing the different analysis stages are
     279performed on the IPP computer cluster.  Note the distinction between
     280the generic analysis {\em stage} and a specific analysis {\em task}.
     281An analysis stage represents a type of analysis which is performed,
     282such as the basic image calibration and object detection analysis.  An
     283analysis task is a particular realization of an analysis stage, e.g.,
     284the analysis of OTA number 61 from exposure 654321 to produce a
     285specific set of output data products.  The analysis stages are
     286discussed in detail in Section~\ref{sec:AnalysisStages}.
    283287
    284288A particular stage may process individual images, collections of
     
    507511monitor the age of images and delete them on some schedule.  This
    508512functionality currently resides in the IPP Scheduler
    509 (Section~\ref{sec:scheduler}).
     513(Section~\ref{sec:pantasks}).
    510514
    511515As shown in Figure~\ref{fig:Nebulous}, Nebulous consists of the
     
    521525\begin{center}
    522526\resizebox{4.5in}{!}{\includegraphics{pics/ImageServer}}
    523 \caption{The components of the IPP Image Server.}
    524 \label{fig:ImageServer}
     527\caption{The components of Nebulous, the IPP Image/File Server.}
     528\label{fig:Nebulous}
    525529\end{center}
    526530\end{figure}
     
    11061110detection data, the {\tt Regions} table allows for multiple computers
    11071111to serve the database tables.  The region file specifies the machine
    1108 which stores the specific table.  Figure~\ref{fig:APDBRegions}
     1112which stores the specific table.  Figure~\ref{fig:DVOskyregions}
    11091113illustrates schematically the subdivision of the sky and the
    11101114association between different levels of the hierarchy with different
     
    15821586
    15831587\subsection{PanTasks : the IPP Scheduler}
     1588\label{sec:pantasks}
    15841589
    15851590PanTasks is the IPP tool which manages the sequencing of data analysis
     
    21482153\begin{center}
    21492154\includegraphics[scale=0.85,angle=-90]{pics/pantasks.05.ps}
    2150 \caption{\label{queues} pcontrol job states.  Transitions labeled Ux
     2155\caption{\label{JobStates} pcontrol job states.  Transitions labeled Ux
    21512156  are issued by the pcontrol user (including PanTasks).  Transitions
    21522157  labeled Px are initiated by pcontrol.  Transitions labeled Tx are
     
    22502255\begin{center}
    22512256\includegraphics[scale=0.85,angle=-90]{pics/pantasks.07.ps}
    2252 \caption{\label{queues} PanTasks queues and MDDB tables}
     2257\caption{\label{Pclient} Pclient monitor loop.}
    22532258\end{center}
    22542259\end{figure}
     
    31263131%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
    31273132
    3128 \section{System Design : Miscellaneous Tasks}
     3133\section{System Design : Miscellaneous Analysis Stages}
     3134\label{sec:MiscStages}
    31293135
    31303136This section discusses additional operations which are performed by
     
    31713177
    31723178\section{Software Hierarchy}
     3179\label{sec:SoftwareHierarchy}
    31733180
    31743181In order to facilitate testing and development, and to encourage
     
    32293236\subsection{IPP Analysis Programs}
    32303237
    3231 \tbd{clean this up}
    3232 
    32333238The major IPP processing tasks are organized into stages, which
    32343239consist of multiple modules.  Each stage represents a collection of
     
    32403245images from multiple telescopes and search for transients).
    32413246
    3242 \section{Top-Level IPP Analysis Routines}
     3247\section{Top-Level IPP Analysis Programs}
     3248\label{sec:AnalysisPrograms}
    32433249
    32443250The IPP uses a handful of high-level analysis routines which perform
     
    39043910
    39053911\section{IPPTools}
     3912\label{sec:ipptools}
    39063913
    39073914PanTasks is the IPP tool which manages the sequencing of data analysis
     
    54015408
    54025409\section{AP Database Table Contents}
    5403 \label{sec:APDBTableContents}
     5410\label{sec:DVOTableContents}
    54045411
    54055412\begin{table}[bh]
     
    57005707\begin{table}[bh]
    57015708\begin{center}
    5702 \caption{Database Hosts\label{tab:APDBHosts}}
     5709\caption{Database Hosts\label{tab:DVOHosts}}
    57035710\begin{tabular}{lll}
    57045711\hline
  • trunk/doc/dvo/dvo.tex

    r6055 r6434  
    11\documentclass[panstarrs,spec]{panstarrs}
    22
    3 \title{DVO : the Desktop Virtual Observatory}
    4 \subtitle{Astronomical Object Databasing in the IPP}
     3\title{DVO Software Design Description}
     4\subtitle{The Desktop Virtual Observatory: \\ Astronomical Object Databasing in the IPP}
    55\author{Eugene Magnier}
    66\audience{IPP}
    7 \shorttitle{PanTasks for IPP}
     7\shorttitle{DVO SDD}
    88\group{Pan-STARRS IPP}
    99\project{Pan-STARRS IPP}
    1010\organization{Institute for Astronomy}
    1111\version{DR}
    12 \docnumber{PSDC-xxx-xxx}
     12\docnumber{PSDC-430-016}
    1313
    1414\begin{document}
     
    10291029 dvo: style -pt 7; cat -all -m 12 18 -g
    10301030\end{verbatim}
     1031%$
    10311032
    10321033\section{other user tools}
  • trunk/doc/ipptools/ipptools.tex

    r6168 r6434  
    11\documentclass[panstarrs,spec]{panstarrs}
    22
    3 \title{IPPTools, PanTasks \& the IPP Analysis Stages}
    4 \subtitle{Job Relationships and Data Flow}
     3\title{IPPTools Software Design Description}
     4\subtitle{The IPP Analysis Stages: \\ Job Relationships, Database Tables, and Data Flow}
    55\author{Eugene Magnier}
    66\audience{IPP}
    7 \shorttitle{PanTasks for IPP}
     7\shorttitle{IPPTools SDD}
    88\group{Pan-STARRS IPP}
    99\project{Pan-STARRS IPP}
    1010\organization{Institute for Astronomy}
    1111\version{DR}
    12 \docnumber{PSDC-xxx-xxx}
     12\docnumber{PSDC-430-019}
    1313
    1414\begin{document}
     
    928928particularly true of the \code{submit.Px} type of commands. 
    929929
     930\tbd{this section should be reconciled with the section in the SSDD which takes precedence}
     931
    930932\begin{verbatim}
    931933
  • trunk/doc/modules/ChangeLogSDRS.tex

    r6172 r6434  
    1 %%% $Id: ChangeLogSDRS.tex,v 1.42 2006-01-22 10:45:30 eugene Exp $
     1%%% $Id: ChangeLogSDRS.tex,v 1.43 2006-02-16 18:57:51 eugene Exp $
    22
    33\subsection{Changes from version 00 (16 August 2004) to version 01 (12 October 2004)}
     
    189189\item modification of bias subtraction API
    190190\item updates to object/psphot APIs
    191 
    192191\end{itemize}
    193192
     193\subsection{Changes from version 11 (22 Jan 2006) to present}
     194
     195\begin{itemize}
     196\item clarified pmFindImagePeaks to use parent coordinates
     197\end{itemize}
     198
  • trunk/doc/modules/ModulesSDRS.tex

    r6422 r6434  
    1 %%% $Id: ModulesSDRS.tex,v 1.74 2006-02-11 03:16:55 price Exp $
     1%%% $Id: ModulesSDRS.tex,v 1.75 2006-02-16 18:57:51 eugene Exp $
    22\documentclass[panstarrs,spec]{panstarrs}
    33
     
    24052405making flat-topped regions have single peaks at the (+x,+y) corner.
    24062406When selecting the peaks, their type must also be set.  The result of
    2407 this function is an array of \code{pmPeak} entries.
     2407this function is an array of \code{pmPeak} entries.  The resulting set
     2408of peaks should be considered a starting point, not an unambiguous
     2409sample of the only real peaks.  If the input image is a subimage, the
     2410output peak coordinates should be in the {\em parent} coordinate
     2411frame.
    24082412
    24092413\begin{prototype}
     
    24332437The first form defines \code{psImage} subarrays (pixel, weight, and
    24342438mask) for the source located at coordinates \code{x,y} on the image
    2435 set defined by \code{readout}.  The pixels defined by this operation
    2436 consist of a square window (of full width $2 Radius + 1$) centered on
    2437 the pixel which contains the given coordinate, in the frame of the
    2438 readout.  The window is defined to have limits which are valid within
    2439 the boundary of the \code{readout} image, thus if the radius would
    2440 fall outside the image pixels, the subimage is truncated to only
    2441 consist of valid pixels.  If \code{readout->mask} or
     2439set defined by \code{readout} (in parent coords).  The pixels defined
     2440by this operation consist of a square window (of full width $2 Radius
     2441+ 1$) centered on the pixel which contains the given coordinate, in
     2442the frame of the readout.  The window is defined to have limits which
     2443are valid within the boundary of the \code{readout} image, thus if the
     2444radius would fall outside the image pixels, the subimage is truncated
     2445to only consist of valid pixels.  If \code{readout->mask} or
    24422446\code{readout->weight} are not \code{NULL}, matching subimages are
    24432447defined for those images as well.  This function fails if no valid
  • trunk/doc/pantasks/pantasks.tex

    r6055 r6434  
    11\documentclass[panstarrs,spec]{panstarrs}
    22
    3 \title{PanTasks}
    4 \subtitle{the IPP Scheduler and Controller system}
     3\title{PanTasks Software Design Description}
     4\subtitle{The IPP Scheduler and Controller system}
    55\author{Eugene Magnier}
    66\audience{IPP}
    7 \shorttitle{PanTasks}
     7\shorttitle{PanTasks SDD}
    88\group{Pan-STARRS IPP}
    99\project{Pan-STARRS IPP}
    1010\organization{Institute for Astronomy}
    1111\version{DR}
    12 \docnumber{PSDC-xxx-xxx}
     12\docnumber{PSDC-430-017}
    1313
    1414\begin{document}
  • trunk/doc/pslib/ChangeLogSDRS.tex

    r6371 r6434  
    1 %%% $Id: ChangeLogSDRS.tex,v 1.193 2006-02-08 03:26:01 jhoblitt Exp $
     1%%% $Id: ChangeLogSDRS.tex,v 1.194 2006-02-16 18:57:51 eugene Exp $
    22
    33\subsection{Changes from version 00 to version 01}
     
    842842\item convert \code{complex} declarations to be explicitly \code{double complex}
    843843\item change \code{psFitsReadTable()}'s \code{fits} param to be \code{const}
     844\item changes to make image, subimage, and region consistent:
     845\begin{itemize}
     846\item all region operations refer to the parent image coordinates
     847\item all functions which take a psImage and some coordinate refer to the parent coordinate frame
     848\item psRegionForImage : changed definition of region to refer to parent coords
     849\item psImageCountPixelMask : changed definition of region to refer to parent coords
     850\item psImageSubset : specified definition of region to refer to parent coords
     851\end{itemize}
    844852\item Added S8,S16,U8,U16,U32 to \code{psDataType} (bug 579).
    845853\item \code{psVector.n} and \code{psArray.n} to be initially set to zero
  • trunk/doc/pslib/psLibSDRS.tex

    r6371 r6434  
    1 %%% $Id: psLibSDRS.tex,v 1.382 2006-02-08 03:26:01 jhoblitt Exp $
     1%%% $Id: psLibSDRS.tex,v 1.383 2006-02-16 18:57:51 eugene Exp $
    22\documentclass[panstarrs,spec]{panstarrs}
    33
     
    36903690limits.  The output of this function is always a valid region, though
    36913691it may define an area of 0 pixels.  If \code{image} is a subimage, the
    3692 region coordinates refer to the subimage pixel coordinates.
     3692input and output region coordinates refer to the parent pixel
     3693coordinates.  The only exception to this statement is that the
     3694negative limits should be applied to the upper limits of the subimage,
     3695not the parent.  Thus, if we have an input subimage with
     3696\code{col0,row0} of (10,20), and \code{numCols,numRows} of 1000,1000
     3697(implying parent image dimensions of at least 1010,1020), we would
     3698have the following conversions:
     3699\begin{verbatim}
     3700(50:100,50:100) -> (50:100,50:100)   : no change (region within image)
     3701(0:0,0:0)       -> (10:1010,20:1020) : upper and lower limits constrained
     3702(5:-5,5:-5)     -> (10:1005,20:1015)
     3703(5:1020,5:1020) -> (10:1010,20:1020)
     3704\end{verbatim}
    36933705
    36943706\begin{prototype}
     
    50835095\subsubsection{Image Structure Manipulation}
    50845096
     5097\tbd{clarify that subset and trim regions are in the *parent* coords}
     5098
    50855099\begin{prototype}
    50865100psImage *psImageSubset(psImage *image, psRegion region);
     
    50955109In the event that \code{x1} or \code{y1} are negative, they shall be
    50965110interpreted as being relative to the size of the parent image in that
    5097 dimension.  The entire subraster must be contained within the raster
    5098 of the parent image.  Note that the \code{refCounter} for the parent
    5099 should be incremented.  This function must be defined for the
    5100 following types: \code{psU8}, \code{psU16}, \code{psS8}, \code{psS16},
    5101 \code{psF32}, \code{psF64}, \code{psC32}, \code{psC64}.
     5111dimension.  If the input image is a subimage, the region coordinates
     5112refer to the \em{parent} pixel coordinates.  The entire subraster must
     5113be contained within the raster of the parent image.  Note that the
     5114\code{refCounter} for the parent should be incremented.  This function
     5115must be defined for the following types: \code{psU8}, \code{psU16},
     5116\code{psS8}, \code{psS16}, \code{psF32}, \code{psF64}, \code{psC32},
     5117\code{psC64}.  \tbd{allow this function to saturate on the edges like
     5118psRegionForImage?}
    51025119
    51035120\begin{prototype}
     
    51325149
    51335150\subsubsection{Image Pixel Extractions}
     5151
     5152\tbd{clarify that subset and trim regions are in the *parent* coords}
    51345153
    51355154\begin{datatype}
     
    54185437region) results in a return value of -1.  The \code{region} refers to
    54195438the pixels of the \code{mask}; if \code{mask} is a subimage, the
    5420 region must be defined relative to those subimage pixels.
     5439region must be defined relative to the parent pixel coordinate.
    54215440
    54225441\begin{prototype}
  • trunk/doc/psphot/psphot.tex

    r6009 r6434  
    11\documentclass[panstarrs,spec]{panstarrs}
    22
    3 \title{PSPhot} % put in your title
     3\title{PSPhot Sofware Design Description} % put in your title
    44\subtitle{The Pan-STARRS IPP Object Photometry Tool}
    55\author{Eugene Magnier}
    66\audience{IPP}
    7 \shorttitle{PSPhot}
     7\shorttitle{PSPhot SDD}
    88\group{Pan-STARRS IPP}
    99\project{Pan-STARRS IPP}
    1010\organization{Institute for Astronomy}
    1111\version{DR}
    12 \docnumber{PSDC-xxx-xxx}
     12\docnumber{PSDC-430-021}
    1313
    1414\newcommand\ugriz{$u^\prime g^\prime r^\prime i^\prime z^\prime$}
Note: See TracChangeset for help on using the changeset viewer.