various bugs / issues with ippTools

[eugene]

I'm placing a number of low-level but important bugs / issues related to various ippTools programs in this bug.

• flatcorr -newrun : without any arguments, should not create a new flatcorr run (currently, it creates a flatcorr run with all chip_ids)

• flatcorr -newrun : be default, this should select by exp_type = object, and only allow other types if overridden

• flatcorr -newrun : this probably should set the initial state to 'register', we should provide a -addexp mode with which the user can add more exp_ids to the flatcorr run, and a -updateflatcorr -state run mode to change the current state.

• chiptool -queue : without any other arguments, this currently queues all chip_ids for processing. Like flatcorr, without any arguments, this command should not do anything (too dangerous).

• chiptool -queue : be default, this should select by exp_type = object, and only allow other types if overridden

• chiptool -queue : needs a -pretend option

[eugene]

flatcorr -pending has three bugs:

• it only requires the selected images to complete the chip stage, but we need it to wait for the images to complete the camera stage

• even given that logic, it is not waiting until all chips are done before listing an entry as pending.

• it is reporting flatcorr runs even though they are in the state of 'stop'

flatcorr -flatcorrimfile lists only the completed chip imfiles, not all imfiles associated with the flatcorr run.

[eugene]

some more flatcorr things:

need to add a region element (STR) to flatcorrRun which is set on newrun and returned on pending

[jhoblitt]

I have:

• changed flatcorr -newrun to require additional parameters

• changed flatcorr -newrun to default to an exp_type of object

• flatcorr -newrun already created new run with a state of 'reg'

• added a flatcorr -addexp mode

• added a flatcorr -updaterun mode

• changed chiptool -queue to require additional parameters

• changed chiptool -queue to default to an exp_type of object

• added a chiptool -queue -pretend flag that prints the SQL query to the stderr and stops

• added a flatcorrRun.region field and a -set_region option to flatcorr -newrun

These two aren't well tested:

• changed flatcorr -pending to to wait for all chip_id's in a flatcorrRun to finish

• changed flatcorr -pending to not report flatcorrRun's in the state of stop

Regarding the other changes to flatcorr, this is the first mention of chip_id's needing to pass through the camtool step. Does some information from that step need to be passed back somewhere?

What exactly should flatcorr flatcorrimfile` be returning? It's currently returning chiptool processed imfiles and all of the associated stats. In order to return everything as stated in the bug, I'd need to return the rawImfiles which will obviously change the returned fields. Is that what you want?

[rodney]

A few more to add to the list:

chiptool / camtool / warptool :
-updaterun does not allow an update of the tess_id parameter, required by warp, though it is possible to reach the warp stage with tess_id=NULL.

The dvodb parameter is also not updatable, although it may not make sense to update the dvodb value in the table without also ingesting the data from the .smf files into said dvodb.

camtool :
-masked -unmasked and -block are listed options, but don't seem to be operable. I think the labels are being inherited from the chipRun table, but the blocks are not.

[rodney]

warptool -revertwarped SQL error:

warptool -revertwarped -warp_id 12 -dbname sress

Executing SQL:

DELETE FROM warpSkyfile
USING warpSkyfile, warpRun, camRun, chipRun, rawExp
WHERE

warpSkyfile.warp_id = warpRun.warp_id
AND warpRun.cam_id = camRun.cam_id
AND camRun.chip_id = chipRun.chip_id
AND chipRun.exp_id = rawExp.exp_id
AND warpSkyfile.fault != 0

AND warp_id = 12
p_psDBRunQuery (psDB.c:837) : Failed to execute SQL query. Error: Column 'warp_id' in where clause is ambiguous
revertwarpedMode (warptool.c:1123) : database error

---

NOTE: similar error for
warptool -revertwarped -cam_id
camtool -revertprocessedexp -cam_id
etc.

[eugene]

the warptool revert problem has been addressed with the pxwarpGetSearchArgs function, which supplies the correct table name.

the flatcorr issues have been worked through in multiple test runs fall 2008.

chiptool / camtool / warptool -updaterun should not be used to reset the tess_id and dvodb. if an entry is queued without these set, then a new camRun or warpRun should be initiated (or further upstream) with the desired value. However, it would be good to require tess_id if the end_stage is warp. i filed this as its own ticket (#1204)