| | 7 | |
| | 8 | I spent most of my week working on issues related to the repair of the ThreePi database which was corrupted when /data/ipp004.0 filled. I created a tool to make repairs on a DVO database, and have been applying it to this case. The program, dvorepair, does several things: |
| | 9 | |
| | 10 | * it scans the full database and counts the number of detections associated with each image. This count is compared with the number of detections expected to come from each image, and any that are short are noted. |
| | 11 | * given a list of image IDs (as output from the previous analysis), dvorepair will locate all associated detections from the database measurement tables and delete them |
| | 12 | * given a list of image IDs, dvorepair will delete the images from the image table, and if all chips for an exposure are deleted, it will delete the exposure entry as well. |
| | 13 | * given a list of defective 'cpt' and/or 'cps' files (average objects and photometry tables), dvorepair will use the information in the (valid) 'cpm' table (detections) to re-generate a consistent set of cpt and cps tables. |
| | 14 | |
| | 15 | These types of repairs are needed because of the ways in which the database was corrupted: some detection tables were partially written (dropping the new detections, and in other cases, the average object tables were partially written. |
| | 16 | |
| | 17 | I've run these on the ThreePi database, and deleted the images with incomplete detection sets -- these correspond to the images written during the step in which the disk filled up, and will be re-added after the database is internally consistent. I've also discovered another interesting issue: There are a number of images which were added to the database multiple times. This seems to be caused by an error raised by 'addstar', in coordination with an error in the addstar sequencing logic (ie, a failed exposure should not be automatically retried). Now that we have a working tool to delete images, we can identify the multiple copies and use this new tool to delete them, and then re-add them after we correct the error that caused them to fail in the first place. |
| | 18 | |
| | 19 | I have also created a tool to scan the dvo database and check for inconsistencies. This tool should be run on the minidvodbs before they are merged, and on the full dvo database after a merge. I have also updated the addstar and dvomerge code to catch the errors which result from a disk failure, so we will be less likely to require this type of repair in the future. |
