Changes between Version 10 and Version 11 of IPP_Progress_Report_20110117

Timestamp:

Jan 26, 2011, 3:06:17 PM (16 years ago)

Author:

eugene

Comment:

--

IPP_Progress_Report_20110117

@@ -5 +5 @@
 
 === Eugene Magnier ===
+
+I have continued to worry about the magic masking coming from the PSF-matching code.  I have adjusted the code to make a choice for the single 1, single 2, or dual convolution based on the improvements to the chisq of the fit (it must improve enough to justify more terms).  This modification helps, but does not solve all issues.  Looking further, I discovered 2 additional interesting issues.  First, the PSF-matching code scales sizes of the set of Gaussians based on the FWHM of the input images.  This input FWHM is measured on the warp images from sources at the locations determined by detection on the chip images.  I discovered that sometimes these reported FWHM is significantly different from the actual FWHM -- the rate of bad PSF model fits seems to be higher in the warp output than in the chip analysis.  I modified the code to use the moments-based analysis of the FWHM in the ppSub process to determine the Gaussian sizes, rather than trusting the warp values.  Second, I discovered an error in the code which truncates the convolution kernel : this function is called before propagating the covariance matrix through the convolution kernel.  The effect of the error is ignore the convolution kernel when propagating the covariance, resulting in the wrong covariance values.  The error in the code has been in place since early 2010, but is only triggered in cases when the convolution kernel is large compared with the kernel window, so the effect was only rarely triggered.  The modifications in 8-2010 included reduction of that window to speed the processing, resulting much greater frequency of the error being triggered.  Applying these fixes has improved the masking rate relative to the 8/23 installation, but does not reach the levels from before that code change, so I am still investigating this issue.
+
+I also addressed the relphot / dvo error that Heather and Roy were hitting.  I extended 'dvoverify' to test the values of the internal and external indexes, and found that the last entry in a dvo average table can claim the wrong number of measurements -- an off-by-one counting error.  this is now fixed in my branch and is addressed by re-running 'addstar -resort'.
+
+I also spent some time on the magic and burntool processing throughputs: when we have a large data backlog, the magic queries can result in under-filled processing queues, slowing the magic processing rate.  This can be addressed by requesting more jobs per query; making this change pushed the magic throughput to ~50-70 exposures per hour.  For burntool, the relevant pantasks was being slowed by other stdscience processing taking place on the same machine.  Reducing the number of processing connections to that pantasks host made the burntool (and the rest of registration) zip along much faster.
 
 === Serge Chastel ===
@@ -17 +23 @@
  * relphot investigations -- Why can't we relphot ThreePi.283?  I tried building different minidvodbs from those cam_ids, and I have not found any useful clues. 
    * binary sort (keep chopping the number of cam_ids in half until you find which cam_id is corrupted) - this didn't work.  On the chop from 81->40, I created 2 dvodbs that can have relphot run on them successfully.
-   * taking one of those, and adding in the missing cam_ids one at a time, and running relphot on it inbetween, didn't work - I added in all the cam_ids and it was fine.
+   * taking one of those, and adding in the missing cam_ids one at a time, and running relphot on it in between, didn't work - I added in all the cam_ids and it was fine.
    * running relphot on 284- it ran fine.
    * roy also tried to investigate, also unsuccessfully.