IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changeset 25150


Ignore:
Timestamp:
Aug 20, 2009, 11:49:09 AM (17 years ago)
Author:
watersc1
Message:
 
Location:
branches/czw_branch/cleanup
Files:
8 edited

Legend:

Unmodified
Added
Removed

  • branches/czw_branch/cleanup/DataStore/scripts/dsget

    r24374 r25150  
    6969        undef $copies;
    7070    }
    71     if ($copies < 1) {
     71    elsif ($copies < 1) {
    7272        die "--copies must be >= 1";
    7373    }

  • branches/czw_branch/cleanup/doc/misc/docgen.pl

    • Property svn:mergeinfo changed (with no actual effect on merging)

  • branches/czw_branch/cleanup/doc/stamps/stampsICD.tex

    r18745 r25150  
    7676\end{table}
    7777
    78 The IPP will be configured to periodicly query a set of Data Stores for new request file sets.
    79 We will refer to each Data Store as a \emph{requestor}.
    80 Each request table submitted by a given requstor must have a unique value for
    81 the keyword \code{REQ_NAME}. This value is used for the name of the request's output file set.
     78The IPP will be configured to periodically query a set of Data Stores for new request file sets.
     79We will refer to each Data Store as a \emph{requester}.
     80Each request table submitted must have a unique value for the keyword \code{REQ_NAME}.
     81This value is used for the name of the request's output file set.
    8282
    8383The postage stamp server supports three types of "postage stamp" job specified by the value
    8484the column \code{JOB_TYPE}. If the value is \emph{stamp} postage stamp images will be extracted
    8585from the images of interest.  If the value of \code{JOB_TYPE} is \emph{get\_image} a copy
    86 of the selected image(s) with (satelite streaks removed) will be placed on the output file set.
     86of the selected image(s) with (satellite streaks removed) will be placed on the output file set.
    8787Finally if the \code{JOB_TYPE} is \emph{list} the results file will list the input selected images
    8888but no postage stamps will be made.
    … …  
    100100\code{JOB_TYPE} & string    & stamp or get\_image       &        \\
    101101\\
    102 \code{STAMP_NAME}  & string & letters, numbers, '.',  '\-' and '\_'  &        \\
    103 \code{OPTION_MASK} & U32    & \code{[0-5]} see text     &      \\
    104 \\
    105 \code{REQ_TYPE} & string    & bycoord, byexp, or byid   &        \\
     102\code{OPTION_MASK} & U32    & see text                  &      \\
     103\\
     104\code{REQ_TYPE} & string    & bycoord, byexp, byid, bydiff, or byskycell   &        \\
    106105\code{IMG_TYPE} & string    & raw, chip, warp, diff, or stack  & \\
    107 \code{ID} & string          & depends on project and REQ\_TYPE& \\
    108 \code{CLASS_ID} & string    & see text                  &        not null for byid raw \& byexp raw \\
     106\code{ID} & string          & depends on REQ\_TYPE& \\
     107\code{TESS_ID}  & string    & see text                  &        \\
     108\code{COMPONENT} & string   & see text                  &        \\
    109109\\
    110110\code{COORD_MASK} &  U32    & \code{[0-3]} see text     &        not null if \code{JOB_TYPE} = stamp \\
    … …  
    130130\subsubsection{Request Types}
    131131The method used to select the images of interest is determined by the value of the REQ\_TYPE column.
    132 There are three alllowed values.
     132There are three allowed values.
    133133
    134134\begin{itemize}
    135135
    136 \item{\textbf{bycoord}} The center of the region of interest is specifed in celestial coordinates.
     136\item{\textbf{bycoord}} The center of the region of interest is specified in celestial coordinates.
    137137Images are selected by querying the project's DVO database for images with the provided
    138138IMG\_TYPE that contain the center.
    139139The set of images of interest may be limited by specifying certain criteria as described
    140 in Section \ref{ImageFilters}.
     140in Section \ref{ImageFilters}. NOTE: this request type is not yet implemented.
    141141
    142142\item{\textbf{byexp}}  The image of interest is the image resulting from a particular exposure
    … …  
    146146The ID parameter refers to IPP image database key that corresponds to the IMG\_TYPE.
    147147e.g: either exp\_id, chip\_id, warp\_id, diff\_id, or stack\_id.
     148
     149\item{\textbf{bydiff}}  The image with given IMG\_TYPE corresponding to the difference image
     150whose IMAGEID is equal to ID. This type is useful for requesting stamps based on difference detections.
     151
     152\item{\textbf{byskycell}}  Images are selected from the skycell with skycell\_id equal to the
     153value for COMPONENT from the tessellation identified by TESS\_ID. This request type may be
     154used only with IMG\_TYPE warp, stack, and diff.
     155The set of images may be limited by filter and date as described in Section \ref{ImageFilters}.
     156NOTE: this request type is not yet implemented.
     157
    148158\end{itemize}
    149159
    150 CLASS\_ID must also be supplied for request specifications where all of the following are true
     160COMPONENT must also be supplied for request specifications where all of the following are true
    151161
    152162\begin{itemize}
    … …  
    158168\item REQ\_TYPE=byexp or REQ\_TYPE=byid
    159169
    160 \item IMG\_TYPE=chip or IMG\_TYPE=raw
    161 
    162170\end{itemize}
    163171
     172The special value 'all' may be used to select all component images but this is not recommended.
    164173
    165174\subsubsection{Image Selection Filters}\label{ImageFilters}
    … …  
    169178
    170179The REQFILT parameter may be used to restrict the search for images taken with a given filter.
    171 XXX: What is the namespace for filter names.  For example Megacam has filter names like U.9301 which
    172 IPP maps to u.  Which name do we allow? Probably should used the mapped names g r i etc.
     180XXX: What is the name space for filter names.  For example Megacam has filter names like U.9301 which
     181IPP maps to u.  Which name do we allow?  Perhaps use a LIKE query.
     182
    173183
    174184\subsection{The Region of Interest}
    … …  
    177187a center,  width, and height.
    178188
    179 The center is defined by the columns CENTER\_X and CENTER\_Y and the dimenions of the rectangle are
     189The center is defined by the columns CENTER\_X and CENTER\_Y and the dimensions of the rectangle are
    180190the values of WIDTH and HEIGHT. The two least signficant bits in value for the column COORD\_MASK
    181191determine the type for the coordinates
    … …  
    189199{\bf COORD\_MASK value} &  {\bf Description } \\
    190200\hline
    191 0                       &   center in RA/DEC; width \& height in arcseconds\\
    192 1                       &   center in pixel coordinates; width and height in arcseconds\\
     2010                       &   center in RA/DEC; width \& height in arc seconds\\
     2021                       &   center in pixel coordinates; width and height in arc seconds\\
    1932032                       &   center in RA/DEC; width and height in pixels\\
    1942043                       &   center x/y, width, and height in pixel coordinates\\
    … …  
    198208\end{table}
    199209
    200 Another way to describe this is that COORD\_MASK is two bit mask that bitmask that
     210Another way to describe this is that COORD\_MASK is two bit mask that bit mask that
    201211determines the format of the ROI values. In the C language:
    202212
    … …  
    212222
    213223When pixel coordinates are used the coordinate system is that of the image. (For chip level images, before
    214 the stamps are created the cells are mosaiced into a single image.)
     224the stamps are created the cells are mosaicked into a single image.)
    215225
    216226Any fractional parts in pixel coordinate values are ignored.
    217227
    218 The value of \code{OPTION_MASK} is used to select various other options. It is a bitwise or of the following
     228The value of \code{OPTION\_MASK} is used to select various other options. It is a bitwise or of the following
    219229values.
    220230
    … …  
    2292391                       &   create postage stamp of the image pixels \\
    2302402                       &   create postage stamp of the mask pixels \\
    231 4                       &   create postage stamp of the weight pixels \\
    232 8                       &   convert WCS to approximate linear transformation in the stamp images\\
    233 \hline
    234 \end{tabular}
    235 \end{center}
    236 \end{table}
     2414                       &   create postage stamp of the variance pixels \\
     2421024                    &   select images based on inverse difference images \\
     243\hline
     244\end{tabular}
     245\end{center}
     246\end{table}
     247
     248Note that bit zero is currently ignored. A stamp of the image pixels is always included.
     249
     250For requests with IMG\_TYPE diff or if \code{REQ_TYPE} is bydiff \code{if (OPTION_MASK & 1024)} and
     251the corresponding difference run is a warp versus warp diff (bothways) the images
     252are selected based on the 'inverse' difference images. If this bit is zero the 'positive'
     253images are selected;
    237254
    238255
    … …  
    293310\code{REQ_TYPE} & string    & \\
    294311\code{IMG_TYPE} & string    & \\
    295 \code{CLASS_ID} & string    & \\
     312\code{TESS_ID} & string    & \\
     313\code{COMPONENT} & string    & \\
    296314\\
    297315\code{COORD_MASK} &  U32    & \\
    … …  
    317335IMG\_NAME give the name of the image in the Data Store file set.
    318336
    319 If no STAMP\_NAME is provided in the request specifiction the image file names will be in the form
    320 \code{ROWNUM_I.fits}  where I is an integer that ranges from 1 to the number of jobs that the request
    321 specification generated. Mask images will be named \code{ROWNUM_I.mk.fits} and weight images will
     337The image file names will be in the form \code{ROWNUM_I.fits}  where I is an integer that ranges from 1
     338to the number of jobs generated for the row's request
     339specification.
     340
     341Mask images will be named \code{ROWNUM_I.mk.fits} and variance images will
    322342have the name \code{ROWNUM_I.mk.fits}
    323343
    324 If STAMP\_NAME is provided in the resquest specification, the name of the image, mask, and weight files
    325 will be
    326 \begin{itemize}
    327 \item\code{ROWNUM_I_STAMP_NAME.fits}
    328 \item\code{ROWNUM_I_STAMP_NAME.mk.fits},
    329 \item\code{ROWNUM_I_STAMP_NAME.wt.fits}
    330 \end{itemize}
    331 
    332 JOB\_ID is the value of the postage stamp server's internal job id. This is provided primarily as a debugging
    333 aid.
     344JOB\_ID is the value of the postage stamp server's internal job id. (This is provided primarily as a debugging
     345aid.)
    334346
    335347The column named PROJECT and following columns are copies from the input request specification.
    336348
    337 If ERROR\_CODE is non-zero one or more errors occured processing the request specification.
    338 The value of ERROR\_CODE is a bitwise or of the values given in the table\ref{Rerrorcodes}.
     349If ERROR\_CODE is non-zero an error occurred when processing the request specification or the job.
     350The values for ERROR\_CODE are listed in table\ref{Rerrorcodes}.
    339351
    340352
    … …  
    348360{\bf Name} & {\bf ERROR\_CODE value} &  {\bf Description } \\
    349361\hline
    350 NO\_ERROR            &   0   &   \\
    351 NO\_MATCHING\_IMAGE   &   1   & No image matches the request specification \\
    352 RA\_DEC\_NOT\_ON\_PIXEL &   2   & The central RA, DEC are on a blind area (for example a inter-chip gap) \\
    353 RA\_DEC\_NOT\_IN\_FOV   &   4   & Wrong position on sky for given MJD \\
    354 NO\_IMAGE\_FOR\_FILTER &   8   & Image is not in the filter requested \\
    355 NOT\_AVAILABLE\_PERM  &   16  & Image matched but is permanantly unavailable\\
    356 NOT\_AVAILABLE\_TEMP  &   32  & Image matched but is currently unaccessible\\
    357 INVALID\_REQUEST     &   64  & request specification was invalid \\
    358 SYSTEM\_ERROR        &   128 & system error \\
    359 \hline
    360 \end{tabular}
    361 \end{center}
    362 \end{table}
    363 
    364 In the case of an error parsing the request table a file containing a textual description of the errror
    365 may be added to the resulting file set.
     362PSTAMP\_SUCCESS           & 0  &    No errors \\
     363PSTAMP\_SYSTEM\_ERROR     & 10 &    some unspecified system error occurred during processing\\
     364PSTAMP\_NOT\_IMPLEMENTED  & 11 &    Feature not yet implemented \\
     365PSTAMP\_UNKNOWN\_ERROR    & 12 &    Unknown error \\
     366PSTAMP\_DUP\_REQUEST      & 20 &    Request name is a duplicate \\
     367PSTAMP\_INVALID\_REQUEST  & 21 &    Error in a request specification. See parse\_error.txt \\
     368PSTAMP\_UNKNOWN\_PRODUCT  & 22 &    Unknown product in request specification \\
     369PSTAMP\_NO\_IMAGE\_MATCH  & 23 &   No images matched the request \\
     370PSTAMP\_NOT\_DESTREAKED   & 24 &    Image matched, but not yet de-streaked \\
     371PSTAMP\_NOT\_AVAILABLE    & 25 &    Image not available (temporary) \\
     372PSTAMP\_GONE              & 26 &     Image is no longer available \\
     373PSTAMP\_NO\_JOBS\_QUEUED  & 27 &   Request specification yielded no jobs. See parse\_error.txt \\
     374
     375\hline
     376\end{tabular}
     377\end{center}
     378\end{table}
     379
     380If parsing of the request specification file results in error output, the contents will be placed
     381in the file 'parse\_error.txt' in the resulting file set.
    366382
    367383\end{document}

  • branches/czw_branch/cleanup/ippScripts/scripts/magic_destreak_revert.pl

    • Property svn:mergeinfo changed (with no actual effect on merging)

  • branches/czw_branch/cleanup/ippScripts/scripts/publish_file.pl

    r25051 r25150  
    3939
    4040# Parse the command-line arguments
    41 my ( $pub_id, $exp_id, $camera, $stage, $stage_id, $format, $product, $workdir );
     41my ( $pub_id, $camera, $stage, $stage_id, $format, $product, $workdir );
    4242my ( $dbname, $verbose, $no_update, $save_temps, $redirect );
    4343
    4444GetOptions(
    4545    'pub_id=s'          => \$pub_id, # Publish identifier
    46     'exp_id=s'          => \$exp_id, # Exposure identifier
    4746    'camera=s'          => \$camera, # Camera name
    4847    'stage=s'           => \$stage,       # Stage of interest
    … …  
    5857
    5958pod2usage( -msg => "Unknown option: @ARGV", -exitval => 2 ) if @ARGV;
    60 pod2usage( -msg => "Required options: --pub_id --exp_id --camera --stage --stage_id --product --workdir",
     59pod2usage( -msg => "Required options: --pub_id --camera --stage --stage_id --product --workdir",
    6160           -exitval => $PS_EXIT_CONFIG_ERROR) unless
    6261    defined $pub_id and
    63     defined $exp_id and
    6462    defined $camera and
    6563    defined $product and
    … …  
    6866    defined $workdir;
    6967
    70 my $outroot = "$workdir/$product.$pub_id.$exp_id"; # Output root name
     68my $outroot = "$workdir/$product.$pub_id"; # Output root name
    7169
    7270my $ipprc = PS::IPP::Config->new( $camera ) or
    … …  
    8381my %files;                      # Input filenames
    8482my %zp;                         # Zero points
     83my %exp_id;                     # Exposure identifiers
     84my %exp_name;                   # Exposure names
     85my %direction;                  # Direction of subtraction
    8586{
    8687    my $command;                # Command to run
    … …  
    114115        if ($stage eq 'diff') {
    115116            my $skycell_id = $comp->{skycell_id};
     117
     118            # Positive direction
    116119            $files{"$skycell_id.pos"} = $ipprc->filename( "PPSUB.OUTPUT.SOURCES", $path_base );
    117             $files{"$skycell_id.neg"} = $ipprc->filename( "PPSUB.INVERSE.SOURCES", $path_base ) if
    118                 defined $comp->{bothways} and $comp->{bothways};
    119120            $zp{"$skycell_id.pos"} = $zp;
    120             $zp{"$skycell_id.neg"} = $zp if defined $comp->{bothways} and $comp->{bothways};
     121            $exp_id{"$skycell_id.pos"} = $comp->{exp_id_1};
     122            $exp_name{"$skycell_id.pos"} = $comp->{exp_name_1};
     123            $direction{"$skycell_id.pos"} = 1;
     124
     125            # Negative direction
     126            if (defined $comp->{bothways} and $comp->{bothways}) {
     127                $files{"$skycell_id.neg"} = $ipprc->filename( "PPSUB.INVERSE.SOURCES", $path_base );
     128                $zp{"$skycell_id.neg"} = $zp;
     129                $exp_id{"$skycell_id.neg"} = $comp->{exp_id_2};
     130                $direction{"$skycell_id.neg"} = 0;
     131                $exp_name{"$skycell_id.neg"} = $comp->{exp_name_2};
     132            }
     133
    121134        } elsif ($stage eq 'camera') {
    122135            my $cam_id = $comp->{cam_id};
    123136            $files{$cam_id} = $ipprc->filename( "PSASTRO.OUTPUT", $path_base );
    124137            $zp{$cam_id} = $zp;
     138            $exp_id{$cam_id} = $comp->{exp_id};
     139            $exp_name{$cam_id} = $comp->{exp_name};
     140            $direction{$cam_id} = 1;
    125141        }
    126142    }
    … …  
    137153
    138154    my $zp = $zp{$comp};
     155    my $exp_id = $exp_id{$comp};
     156    my $exp_name = $exp_name{$comp};
     157    my $direction = $direction{$comp};
    139158    if ($product eq "IPP-MOPS") {
    140159        my $outuri = "$outroot.$comp.fits";
    … …  
    142161            &my_die( "Unable to resolve output file $outuri", $pub_id, $PS_EXIT_SYS_ERROR);
    143162
    144         my $command = "$ppMops $file $zp $out";
     163        my $command = "$ppMops $file $zp $exp_id $exp_name $direction $out";
    145164        my ( $success, $error_code, $full_buf, $stdout_buf, $stderr_buf ) =
    146165            run(command => $command, verbose => $verbose);

  • branches/czw_branch/cleanup/ippTasks/dist.pro

    r25051 r25150  
    122122
    123123  task.exec
     124    if ($NETWORK == 0) break
    124125    book npages distToProcess -var N
    125126    if ($N == 0)
    … …  
    128129    end
    129130    periods -exec 0.05
    130 
    131     if ($NETWORK == 0) break
    132131   
    133132    # look for new components to process (pantaskState == INIT)

  • branches/czw_branch/cleanup/ippTasks/publish.pro

    r24707 r25150  
    161161    book getword publishRun $pageName stage -var STAGE
    162162    book getword publishRun $pageName stage_id -var STAGE_ID
    163     book getword publishRun $pageName exp_id -var EXP_ID
    164163    book getword publishRun $pageName dbname -var DBNAME
    165164
    … …  
    170169    strsub $WORKDIR_TEMPLATE @HOST@ $default_host -var WORKDIR
    171170
    172     $run = publish_file.pl --pub_id $PUB_ID --exp_id $EXP_ID --camera $CAMERA --workdir $WORKDIR --product $PRODUCT --stage $STAGE --stage_id $STAGE_ID --redirect-output
     171    $run = publish_file.pl --pub_id $PUB_ID --camera $CAMERA --workdir $WORKDIR --product $PRODUCT --stage $STAGE --stage_id $STAGE_ID --redirect-output
    173172    add_standard_args run
    174173

  • branches/czw_branch/cleanup/ippTasks/summit.copy.pro

    r24835 r25150  
    449449        end
    450450        if (("$MD5SUM" != "NULL") && ("$MD5SUM" != "0") && (not($COMPRESS)))
     451# && (($YEAR > 2008) || (("$YEAR" = "2007") && ($MONTH > 8))))
    451452            $run = $run --md5 $MD5SUM
    452453        end
Note: See TracChangeset for help on using the changeset viewer.