Changes between Version 237 and Version 238 of ippToPsps

Timestamp:

Apr 2, 2012, 5:55:40 PM (14 years ago)

Author:

rhenders

Comment:

--

ippToPsps

@@ -140 +140 @@
 
 
-The master list of items to be processed comes from the gpc1 {{{addRun}}} table, which lists all items currently merged into the DVO database we are using. From this list, {{{queue.py}}} subtracts items that have already been successfully loaded to the datastore already, as well as items that have routinely failed to process, and queues up the remainder.
+For stacks and single-frame detections, the master list of items to be processed comes from the gpc1 {{{addRun}}} table, which lists all items currently merged into the DVO database we are using. From this list, {{{queue.py}}} subtracts items that have already been successfully loaded to the datastore already, as well as items that have routinely failed to process, and queues up the remainder. For object batches, the DVO database is queries for individual regions that fall within the box. So:
+
+|| '''Batch type''' || '''unique identifier''' ||
+|| P2 || cam_id from gpc1 database ||
+|| ST || stack_id from gpc1 database ||
+|| OB || INDEX from SkyTable.DAT file in DVO database ||
 
 Each config requires an entry in the '''{{{epoch}}}''' column.  An ''epoch'' in the context of {{{ippToPsps}}} is the date that we count as ''the beginning''. If we loaded IPP data to PSPS once and only once, this would not be necessary: we would queue up all available exposures and stacks and simply load them. However, the early stages of the project have required multiple re-loads of data while the IPP perfects the science, necessitating that the same exposures and stacks are loaded more than once. Because {{{queue.py}}} queues available items by taking into account those that have already been published to PSPS, we have to give it an epoch date from which to accurately determine those exposure that have been loaded or not. Generally speaking, the epoch is reset every time we delete the main PSPS database.