== IPP Progress Report for the week 2010.09.27 - 2010.10.01 == [[PageOutline]] (Up to [wiki:IPP_Progress_Reports IPP Progress Reports]) === Eugene Magnier === === Serge Chastel === * IPP: Database profiling. Replication and optimization. Created nebulous_opt database on ippdb02 for optimization tests. * MOPS: Meetings. Documentation (mainly reading but also wrote a few words on retention policy on datastore). === Heather Flewelling === * caught up on email from vacation * ifaps1 things: * answered questions from: gaidos, wainscoat, tonry * transferred current dvo db to ifaps1 * skyprobe things * requested skyflats * checked into ipp trunk some fixes for isp (to get proper values in db, and to make checkexp more versatile) * verified we have downloaded all skyprobe images * working on gene's task of plotting processing throughput for various stages: * sick for 2 days. This coincided with czar duty and bad weather. === Roy Henderson === * PSPS * time with Sue investigating the source of duplicates in new merge. Sadly the source was me... * typo in stack schema table causing problems. Committed fix. * IPP * Czar tues/wed * nebulous issues: stopped and restarted everything * finished cleanup code for czartool: now trims all Db tables from previous day to save space, then optimizes everything * added functionality to stop and run pantasks servers from czartool page, including 'stop all' and 'run all' * started work on 'roboczar': now emails (me) if a particular server has been down for a given interval (default of 30 mins) * ippToPsps * pspsSchema2xml.pl script changed to generate mapping file as well. Can be viewed from wiki so people can collaborate on resolving missing fields * changes to ODM-checking script: now logs loading failures * investigated MySQL optimization * Other * flurry of mailing-list activity after Will's email regarding the outreach list === Bill Sweeney === * Updated stack processing to save the MJD-OBS value for the run in the database. (average value for input exposures). This is needed to enable image look up for detectability queries from stacks and stack-stack diffs. * Wrote programs to examine every avaliable stack FITS header to extract the MJD-OBS value and them into the database. * updated the postage stamp parser to look up stacks by MJD-OBS. * Spent some time studying the way time is handled by IPP. * Changed the postage stamp server to treat MJD-OBS values as TAI. (Previously the values of rawExp.dateobs was used. That value is UTC * Lots of wasted time dealing with faulout of ipp020 outage. Built new stacks for the M31 skycells that were lost on ipp020. * More work improving the reliability of the postage stamp server in response to errors due to unavailability of data. Temporary or permanent. Added fault_count columns to pstampJob and pstampDepenent tables. Stop reverting when count gets to 5. Todo: task to stop jobs in this state. * Spent a while working with Niall to determine why he wasn't getting postage stamps from some measurements he found in DVO. It turns out he was using a "Gene Maginer private reserve" catalog made in 2009 which contains data that hasn't been run through magic. This exposed a bug in the hooks we have for allowing IfA to have un-magicked postage stamps. Fixed that. * started working on cleanup for chip_bg and warp_bg stages. === Chris Waters ===