| | 7 | |
| | 8 | We had two big accomplishments this week: the first was the acceptance by the Air Force of the IPP + Magic automatic processing and data release (excluding y-band). This is extremely important as the manual inspection previously required of the Magic results was the primary block for fully automated processing with the IPP. Enabled by this acceptance, we performed the second big accomplishment: a full-scale throughput demonstration of a night's worth of data (see [wiki:ThroughputDemo_20091104 Throughput Demo 2009.11.04]). Although there were some hiccups and issues identified during the test, in the end we were able to achieve a throughput equivalent to the processing of 600 exposures in a 15 hour period. We also smoothed out a number of rough edges to the automatic processing, and identified a few bottlenecks that can help us improve the throughput further. |
| | 9 | |
| | 10 | I spent much of the week working on this throughput test, though I had a bit of time to work with Chris on the difference image photometry. We have concluded that we must modify the variance for source fitting to match the expected mean chi-square, but we are still somewhat unclear why this is so. This is a poorly understood (at least by us!) effect of attempting to do chi-square fitting in the presence of correlated errors. Chris showed that the variance agrees with our expectation for per pixel noise, that the photometry in apertures agrees with our expectations if we include the information in the covariance matrix, but we still find unexpected chi-square distributions for model fits in the faint source regime (in the bright source regime, we expect to have chi-square values which are large due to the difference between the PSF model and the real object shapes). |
| | 11 | |
| | 12 | I've also made some progress with the astrometry catastrophes which occur in certain observations. I examined the particular case of M42, where the cross-correlation analysis is fooled by the clustering of sources in the bright nebulosity. We have code which removes extremely dense clumps from the input detection lists for this reason, but it was necessary to iterate over this process and also to reject the dense clumps from the linear fitting stage of the analysis. I believe that this particular issue, and some related outlier rejections, are responsible for more of the catastrophic astrometry failures we have seen, and I will be continuing to pursue this next week. |
