IPP Progress Report for the week 2011.04.18 - 2011.04.22

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

• MOPS czar
• IPP czar
• Update of IPP-MOPS ICD: ICD Lite wikipage now contains the ICD pdf and nothing else...
• Documenting and fixing hemocrat
• MOPS detection efficiency
• bzip instead of gzip for MySQL dump archives of gpc1, ippadmin, and ippRequestServer. E-mails are now sent to ps-ipp-ops@…

Heather Flewelling

• started rsync of ThreePi db to ipp003 - it stalled out at some point because ipp003 got rebooted, and I restarted it. Not sure if it is complete yet.
• addstar -resorted, relphot'd, and merged the 3 MD04 databases for Roy. (/data/ipp005.0/gpc1/catdirs/MD04.merges/MD04.merge)
• fixed a couple things on ippMonitor (add*)
• czar monday
• sick wed and thursday
• state holiday friday

Roy Henderson

Lots of progress, and a major setback, getting stacks into PSPS this week. Still lots to do before Boston.

• Finished remaining development work for stack batches:
  • questions about how to populate PSPS flux columns: Jim and Gene settled on a solution and I implemented it in code
  • numerous schema changes
  • work on GPC1 queries to list available stacks, get contributing OTAs for each
  • new temp Db table to store meta data from DVO, eg flags, photcode etc
  • redesigned ippToPsps database to keep track of batch types, processing, whether they are loaded or merged etc
  • fixed issue of NaNs in fits headers not writing to database
  • changes to lots of SQL due to Heather's changes in gpc1 addRun table
  • now performing clean-up in Db before export to FITS, i.e. removing NULL objIDs, NULL fluxes etc

• Successfully tested loading and merging of new stack batches. Problems encountered:
  • missing tables: SkinnyObject, ObjectCalColor in stack batches. Added these to code.
  • NULLs sneaking in to new 'updated' column in Object table: we had forgotten to add a default. Changed schema
  • crazy -999 "NULL"s don't fit in byte field and so broke load. Fixed
  • merge stage was expecting all contributing images for the stack to be loaded already. Sue fixed this.

• DVO speed issue
  • encountered a serious DVO speed issue with the new MD04 database. Way too slow for us to load enough before Boston
  • formed plan to pull everything we need into a MySQL database for quicker, easier access
  • started work coding this

• the solution to the above DVO problem means my new code needs to produce detections, so:
  • lots of work filling in the last missing fields
  • porting GPC1 access methods in Jython code
  • laboriously comparing the output with that of the old code to verify nothing has been lost along the way

Bill Sweeney

• Bill spent most of this week finding a workaround to the bright star astrometry problem for the STS exposures. After much testing we came up with a recipe that uses stars only stars within a magnitude range where the astrometry is correct for final fitting. In order to get the initial fit close we changed psastro to optionally use different magnitude ranges for the initial grid search. This method was successful.
• Started reprocessing the STS.2010 data, hopefully for the final time.
• investigated problem with queueing staticsky runs.
• One day and part of the weekend as processing czar.
• Investigated and fixed several problems with the cluster that unfortunately occurred over the weekend.

Chris Waters

• Reprocessing/large area processing
  • Confirmed that even when we have a smaller number of inputs, pairwise diffs still returns better magic results than diffing against a quickstack.
  • Began design work on large area processing tool and scripts. This should manage all the processing for exposures with minimal manual intervention.
• Astrometry: Processed second microtest data. Details available here: ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/GPC1_BrightStarAstrometry_SW_LH_scan
• Started work getting Nebulous to see new large storage hosts at the ATRC.