Changes between Version 8 and Version 9 of IPP_Progress_Report_20110207

Timestamp:

Feb 16, 2011, 9:59:04 AM (15 years ago)

Author:

eugene

Comment:

--

IPP_Progress_Report_20110207

@@ -8 +8 @@
  I spent most of the past week focusing on the reference astrometry & photometry database, making sure relastro and relphot would run correctly on large areas and that they were saving enough Q/A information to enable interpretation of the results easy.  I set up a large run of relastro on the big reference database (on ipp045), though it took a couple of tries to get the sizing right.  The analysis needs to run on a portion of the sky at a time because there are simply too many measurements to fit in the memory footprint of the single machine.  Initially, I was rather simplistically setting the area to be a fix size, ignoring the varying star densities.  This was fine for the region near 0,0 (where the script started), but resulted in a memory footprint of 100s of GBs near RA = 270 (the galactic plane).  I re-worked the script to use ra-ranges that scaled appropriately based on the number of stars in a region.  This is still a bit crude (the program should be smart enough to load a total amount of memory, but that would take more work that I can afford right now; alternatively, it should distribute the load in a balanced way across multiple machines, but that would also be a lot of effort; simple solutions are good enough for now).  The relastro analysis successfully ran on the full sky in about 20 hours.  Unfortunately, I discovered more recently, that I failed to update an important piece of configuration information in a reference table (the rule for the per-position errors based on the magnitude errors), so I need to re-run that analysis.
 
-I also set up a large run for relphot, using the same concept.  I let this run for a few days, before I decided it was going too slowly.  It looks like I failed to port one of the recent optimizations from relastro over to relphot -- this particular step wastes 20-30 minutes per pass in assigning images and camera mosaics, and can be done in < 1 minute with the new optimization.  I need to fix that and restart the relphot analysis this week.
+ I also set up a large run for relphot, using the same concept.  I let this run for a few days, before I decided it was going too slowly.  It looks like I failed to port one of the recent optimizations from relastro over to relphot -- this particular step wastes 20-30 minutes per pass in assigning images and camera mosaics, and can be done in < 1 minute with the new optimization.  I need to fix that and restart the relphot analysis this week.
 
-Given the successful tests of the updated difference code over the past week, I have merged that work from my development branch into the IPP trunk.  After a bit more testing this week, we will be able to make a new operations tag.  
+ Given the successful tests of the updated difference code over the past week, I have merged that work from my development branch into the IPP trunk.  After a bit more testing this week, we will be able to make a new operations tag.  
 
-While the relastro and relphot analyses were running, I have started up again on the field missing from psphotStack output that are expected by PSPS.  I have added in the covariances for the extended source fits, and will finish off the rest of the fields.  I'll then make a sample MD04 stack analysis for the PSPS team to ingest.  
+ While the relastro and relphot analyses were running, I have started up again on the field missing from psphotStack output that are expected by PSPS.  I have added in the covariances for the extended source fits, and will finish off the rest of the fields.  I'll then make a sample MD04 stack analysis for the PSPS team to ingest.  
 
 === Serge Chastel ===