Changes between Initial Version and Version 1 of AddStar_PV2

Timestamp:

Oct 10, 2014, 2:25:38 PM (12 years ago)

Author:

heather

Comment:

--

AddStar_PV2

@@ +1 @@
+= This is a wiki for how Heather has been running the addstars  for PV2 3pi =
+
+== Current Status of AddStar/Merge ==
+
+ * all minidvodbs for all slices are ready
+ 
+ * all is done on ippdvo2's account, merging is primarily managed through addstar.stsci00 and addstar.stsci01
+
+ * slice00.skycal -- DONE!
+ * slice02.skycal -- DONE!
+ * slice04.skycal -- DONE!
+ * slice06.skycal --- broken! it's ready to be redone, but will take some careful queueing, please leave for heather
+ * slice08.skycal --- broken! it's ready to be redone, but will take some careful queueing, please leave for heather
+ * slice10.skycal -- DONE!
+ * slice12.skycal -- DONE!
+ * slice14.skycal -- DONE!
+ * slice16.skycal -- DONE!
+ * slice18.skycal -- DONE!
+ * slice20.skycal -- DONE!
+ * slice22.skycal -- DONE!
+ * pole.skycal -- DONE!
+
+ * slice00.cam --- broken!
+ * slice02.cam --- broken!
+ * slice04.cam --- broken!
+ * slice06.cam --- broken!
+ * slice08.cam - ready for merging into slices!
+ * slice10.cam - ready for merging into slices!
+ * slice12.cam - ready for merging into slices!
+ * slice14.cam - ready for merging into slices!
+ * slice16.cam - ready for merging into slices!
+ * slice18.cam - ready for merging into slices!
+ * slice20.cam - ready for merging into slices!
+ * slice22.cam   - ready for merging into slices!
+ * pole.skycal  - ready for merging into slices!
+
+
+
+== Creation of minidvodbs  ==
+
+=== Queueing/ requeing of addstar runs ===
+
+to queue addstar runs for three pi v2, 
+
+run the following script
+
+{{{
+tools/heather/queuesliceslap2.pl 
+}}}
+
+This will do addtool -queries for the missing slices (chopped by ra dec ranges, based on centers of the exposure)
+
+=== pantasks stuff ===
+
+
+
+
+
+=== check that all the exposures are done, and that the minidvodb is in the proper state ===
+
+== Merging of minidvodbs into slices ==
+
+This is a bit tricky, because we do the first one first, STOP IT, then make a hosttable.dat and dvodist -out it
+
+ 1. Decide which stscii will be the head node (usually I've been using either stsci00 or stsci01), and edit ippdvo2's ippconfig/site.config so that the slice we are working on (for this example, slice04.cam), points to stsci00.1  (or whatever disk is decided). Check that there is at least 500GB of space on the chosen drive.
+
+ 
+  2. queue the 'segments' 
+{{{
+survey.add.merge cam.slice04.0 LAP.PV2.V2.20130717.slice04.cam LAP.PV2.V2.slice04.0.cam
+survey.add.merge cam.slice04.1 LAP.PV2.V2.20130717.slice04.cam LAP.PV2.V2.slice04.1.cam
+survey.add.merge cam.slice04.2 LAP.PV2.V2.20130717.slice04.cam LAP.PV2.V2.slice04.2.cam
+survey.add.merge cam.slice04.3 LAP.PV2.V2.20130717.slice04.cam LAP.PV2.V2.slice04.3.cam
+survey.add.merge cam.slice04.4 LAP.PV2.V2.20130717.slice04.cam LAP.PV2.V2.slice04.4.cam
+survey.add.merge cam.slice04.5 LAP.PV2.V2.20130717.slice04.cam LAP.PV2.V2.slice04.5.cam
+survey.add.merge cam.slice04.6 LAP.PV2.V2.20130717.slice04.cam LAP.PV2.V2.slice04.6.cam
+survey.add.merge cam.slice04.7 LAP.PV2.V2.20130717.slice04.cam LAP.PV2.V2.slice04.7.cam
+survey.add.merge cam.slice04.8 LAP.PV2.V2.20130717.slice04.cam LAP.PV2.V2.slice04.8.cam
+}}}
+
+
+ * the exception is the pole is one segment : LAP.PV2.V2.pole.cam (no segments)
+
+ 3.  check mysql that the first segment is loaded
+
+{{{
+select * from mergedvodbRun where mergedvodb = 'LAP.PV2.V2.20130717.slice04.cam';
+}}}
+
+ 4. once there is at least 1 mergedvodbRUn listed, start the merge on that and turn it off:
+add.mergedvodb LAP.PV2.V2.20130717.slice04.cam
+mergedvodb.merge.on
+status (until you see that it is working on the first merge)
+mergedvodb.merge.off
+}}}
+
+ 5. wait for the merge to be done, without faults, for that first one
+
+ 6. copy hosttable from one of the other LAP.ThreePi slices, do the following edits:
+   * make sure the slice# is correct
+   * make sure it is skycal or cam (as necessary)
+   * give it a unique dir name (I usually go with ……slice04.cam.(merge_id), or ……slice04.cam.(mergeid).V2 as necessary
+   * do not use stsci drives with < 500GB -- find those in the HostTable.dat and shift around as necessary
+
+ 7. dvodist -out (path)
+
+ 8. go back to pantasks and 
+
+{{{
+mergedvodb.merge.on
+}}} 
+
+ 9. periodically check with commands
+
+{{{
+select minidvodb_path, minidvodb_id, merge_id, merge_order, mergedvodbRun.state, mergedvodbProcessed.fault from mergedvodbRun left join mergedvodbProcessed using (merge_id) left join minidvodbRun using (minidvodb_id) where mergedvodb like "LAP%slice04.cam";
+}}}
+
+ 10. if it faults, it needs to be redone (but you can set the mergedvodbProcessed.fault = 0 and the mergedvodbRun.state = 'broken', and revert to find if the others work fine 
+
+{{{
+   mergetool -revertmerged -mergedvodb (stuff) -dbname gpc1
+}}}
+
+
+
+== Miscellaneous ==
+
+=== Check disk space on stscis ===
+
+I have a couple of tools, checked into tools/heather
+{{{
+check_stsci0.pl
+check_stsci1.pl
+check_stsci2.pl
+}}}
+