IPP Progress Report for the week YYYY.MM.DD - YYYY.MM.DD

(Up to IPP Progress Reports)

Eugene Magnier

Heather Flewelling

• finished processing MD10.20090915 from last week
• processed MD01.20090921
• processed MD02.20090923
• burntooled data from 04-01-09 through 05-31-09, reburntooled data between 6-01-09 and 9-13-09, but only dates where we weren't certain it was the latest burntool, burntooled PR dates for chris B, august 2008, and sept 2008 dates (except for a few otas on sept and oct dates).
• burntooled the previous night's data, once it's been copied from the summit
• investigating why M31.2008.20090918 won't process (fails in psastro, still investigating).
• sent instructions to Bill G on how to process data for realtest
• continued md5sum check on raw images, found several more corrupt raw images, including 2 with no good copies. 2827439 images checked, 16 found to be corrupted.
• learned how to destreak images, destreaking MD02.090923

Bill Giebink

• Continued to fix broken psLib tests.
• Started "realtest".

Paul Price

• Fixed skycell hosts to include all wave 1, 2 and 3 machines.
• Turned off ignore of XY27 in chip_imfile.pl
• Stacks:
  • Look fine when target PSF is set analytically: existing image rejection is working to exclude bad images
  • Updated matching chi2 to be relative to the fake image which is common to all images
  • Bad input images have PSFs with PAR_7 < 0 (divot in PSF model); these caused the target PSF to have a similar divot
  • Branched (branches/pap) for implementation of strict and loose limits on PSF parameters
  • pmModelClassSetLimits() sets limits on all models, pmModel.modelSetLimits() sets limits for single model
  • Inputs with PSFs that exceed the limits are ignored when generating the target PSF
  • Stacking seems more robust; more testing is needed before declaring victory, but this might finally be it!
• Fixed writing of S64, U64 FITS tables, and possibly images (trunk@25478)
• Implemented "difftool -advance" with accompanying pantasks task definition (trunk@25509)
• Updated warp and stack stages so that a warp PSF is not required (e.g., for PR images, where we don't convolve to make the stack). Note: it's still the user's job to ensure PSWARP's PSF recipe parameter is compatible with PPSTACK's CONVOLVE recipe parameter.

Bill Sweeney

This week Bill finally was able to find the time to solve several little nagging problems and holes in the magic destreaking and distribution software.

• Added a task destreak.advance and the supporting magictool and SQL code to split off finishing a magicDSRun from processing of the components. (The previous implementation was causing errors due to race conditions).

• Simplified the database tables used to manage distribution.

• Implemented disttool mode to make it easier to set up "destinations" and their "interests" without using direct mysql commands.

• Implemented magictool mode to handle censoring (preventing the release of) exposures for which\ magic missed real satellite streaks.

• worked with users to fix bugs in the postage stamp server

• started implementing 'byskycell' image lookup for the postage stamp server.

• several other things too small to mention here.

Since it's not yet completely automatic Bill also spent many hours managing destreaking and distribution bundling of recently processed M31, MD01, and MD10 data. The functionality discussed above greatly improved the automation for the future.

Chris Waters

• Pushed old data through chip stage cleanup. Fixed bugs in cleanup related to old data not conforming to the current IPP status. Updated warp cleanup to properly ignore skycells that were not created during processing.

• Reading through Nebulous code to determine how much needs to be done to implement deleting a file that has only a single instance on a computer that is no longer accessible.

• Confirmed that the spurious SN transients identified by Armin Rest are in the locations for crosstalk predicted by John Tonry.

• Various burntool related tweaks: detect summit processing and correctly register that fact, only apply burntool masking to CHIP-like recipes, option to allow processing with an old version of burntool, add BTOOLAPP header keyword translation to old GPC1 formats.