IPP Progress Report for the week 2011.01.24 - 2011.01.28

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

• MOPS czar
• Confirmed that some diffs were not produced/published
• Nebulous performance issues:
  • Changing O_DIRECT into O_DSYNC didn't change performance (but what is true on ippc01 is maybe not on ippdb00)
  • Looking where nebulous code could be improved

Heather Flewelling

• deleted more stuff off ipp004 (this is a slow process), freeing up room for addstar.
• liason (Zagursky, Deacon, Henry) - answered questions and copied files over.
• magic investigations - looked on warps/diffs at artifacts that are triggering magic. Mostly streaky features, sent examples to Gene to investigate.
• fringe - there are not a lot of examples with legitimate fringe. The fringe solutions are sometimes very crazy (random? not fitting properly). Started looking at the exposures used to generate the fringes (nothing to report yet).
• turned addstar back on (merging is off - will be turned on soon)
• sick 2 days

Roy Henderson

• PSPS
  • after last week's meeting with Jim and Gene, I was able to populate some more stack fields
  • did some auditing after the (successful) merge of 10,000 batches
  • meeting with Larry, Jim, Conrad and Daniel regarding using PSVO as an interface for accessing MOPS data
  • some work on PSVO to enable queries to the MOPS database

• IPP
  • ported Heather's exposure summary script into czartool. There is a link from the top of czartool
  • fixed minor bug in the czartool Pantasks class: was not getting current labels properly
  • added ability for czartool to grab current dates from pantasks servers, store them in the Db, then report them to the webpage

• Other
  • some work with the ipp metrics: rsync'd so they can be seen from the wiki

Bill Sweeney

• Implemented "cleanup" for magicRuns. This will free up several TB of disk space on ipp050 and ipp053.
• Implemented changes to the publish system to support publishing detections from uncensored data. This will be used by MOPS.
• Two days as processing czar.
• Implemented priority ordering for distribution.
• Debugged and solved ppStack memory explosion problem. This allowed the MD05 reference stacks to complete.
• Fixed bug in destreaking code caused by cfitsio giving me a 64 bit weight image. (It was all NANs).
• Investitgated why ns.stacks.run was failing repeatedly. The problem turned out to be related to the switch of MD05 to the V2 tessellation

Chris Waters

• Updates to registration/nightly science code:
  • Automatically drop exposures at the download stage if they have imfiles with fault=110 (missing files that will never be downloaded).
  • Add -checkstatus mode to regtool to aid in diagnosing problems in the registration process.
  • Diagnose and fix diff queuing code that was prematurely stopping diff processing. Confirmed that this works correctly now.
  • Worked on ipp_apply_burntool_single.pl to allow multiple sequential exposures to be processed at once. This mode should help the system catch up when large blocks are marked as pending_burntool.
• Reburntooled data that had not been updated to the newest version of burntool. All science data back to 2009-04-01 should now be ready for reprocessing.
• Added new recipes and reduction classes for stacking that select the type of fits compression to be used, based on the type of stack created.
• Began communication with OTIS about data quality/IPP-OTIS interface. Will need to rework the current code to be more useful.