Changes between Version 262 and Version 263 of ippToPsps

Timestamp:

Apr 4, 2012, 12:22:35 PM (14 years ago)

Author:

rhenders

Comment:

--

ippToPsps

@@ -125 +125 @@
 It can take a long time to convert a relatively small DVO database to MySQL, however, querying the MySQL database is hugely faster than accessing DVO directly, especially for regions of sky with a high density of detections such as the medium deep fields. (This was seen when loading MD4 prior to the Boston meeting in May 2011. DVO access per exposure was 40 minutes, whereas, once imported to MySQL, query time was roughly 30 seconds.)
 
-The decision of when to pre-ingest DVO into MySQL is made in the {{{loader.py}}}. The {{{queue.py}}} program has already queued up smfs to load by box on the sky. {{{loader.py}}} uses the ratio of smfs files to size of DVO files on disk to determine if it is worthwhile pre-ingesting, or else importing DVO data the usual (C-interface) way. When the loader progresses to the next, neighboring box, the same ratio is calculated. If the decision is made again to pre-ingest from DVO, then any DVO regions overlapping both boxes are retained, while the remainder are purged before importing the new ones.
+The decision of when to pre-ingest DVO into MySQL is made in the {{{loader.py}}}. The {{{queue.py}}} program has already queued up smfs to load by box on the sky. {{{loader.py}}} uses the ratio of smfs files to size of DVO files on disk to determine if it is worthwhile pre-ingesting, or else importing DVO data the usual (C-interface) way. When the loader progresses to the next, neighboring box, the same ratio is calculated. If the decision is made again to pre-ingest from DVO, then any DVO regions overlapping both boxes are retained, while the remainder are purged before importing the new ones. Most of this functionality is handles in the various dvo classes. The Dvo base class can be found [browser:trunk/ippToPsps/jython/dvo.py here].
 
 '''Image ID confusion'''