| | 7 | |
| | 8 | I spent most of the past week focusing on the reference astrometry & photometry database, making sure relastro and relphot would run correctly on large areas and that they were saving enough Q/A information to enable interpretation of the results easy. I set up a large run of relastro on the big reference database (on ipp045), though it took a couple of tries to get the sizing right. The analysis needs to run on a portion of the sky at a time because there are simply too many measurements to fit in the memory footprint of the single machine. Initially, I was rather simplistically setting the area to be a fix size, ignoring the varying star densities. This was fine for the region near 0,0 (where the script started), but resulted in a memory footprint of 100s of GBs near RA = 270 (the galactic plane). I re-worked the script to use ra-ranges that scaled appropriately based on the number of stars in a region. This is still a bit crude (the program should be smart enough to load a total amount of memory, but that would take more work that I can afford right now; alternatively, it should distribute the load in a balanced way across multiple machines, but that would also be a lot of effort; simple solutions are good enough for now). The relastro analysis successfully ran on the full sky in about 20 hours. Unfortunately, I discovered more recently, that I failed to update an important piece of configuration information in a reference table (the rule for the per-position errors based on the magnitude errors), so I need to re-run that analysis. |
| | 9 | |
| | 10 | I also set up a large run for relphot, using the same concept. I let this run for a few days, before I decided it was going too slowly. It looks like I failed to port one of the recent optimizations from relastro over to relphot -- this particular step wastes 20-30 minutes per pass in assigning images and camera mosaics, and can be done in < 1 minute with the new optimization. I need to fix that and restart the relphot analysis this week. |
| | 11 | |
| | 12 | Given the successful tests of the updated difference code over the past week, I have merged that work from my development branch into the IPP trunk. After a bit more testing this week, we will be able to make a new operations tag. |
| | 13 | |
| | 14 | While the relastro and relphot analyses were running, I have started up again on the field missing from psphotStack output that are expected by PSPS. I have added in the covariances for the extended source fits, and will finish off the rest of the fields. I'll then make a sample MD04 stack analysis for the PSPS team to ingest. |
