Current Tasks for the IPP Team

(Up to Main Page)

(Recently completed tasks)

(DRAVG Issues and Concerns)

Notes from WBS entries

a) Tasks required before the start of the survey:

Task	Due	Who	State	Notes
IPP to OTIS interface	3/12	CZW	DONE	we now need to iterate with OTIS
Detectability Server : psphotDetect	3/19	CZW	DONE	output values must be double-checked
Detectability Server : pstamp integration	3/26	CZW	DONE	Tested and Committed.
Raw Distribution	3/12	WS	DONE
DVO large-file upgrade	3/21	EAM	DONE
DVO merge catalogs	3/25	EAM	DONE
DVO Image-indexing	3/27	EAM	DONE

b) Tasks not required for the survey start:

Task	Due	Who	State	Notes
Finish Extended Source Analysis Code	3/31	EAM		petrosians, surface brightness profile done
Stack Photometry Process	4/30	EAM		Initial version of the code is now working
Stacks : example PS1 astrometry	3/19	PAP	DONE	this led to discovery of a 1/2 pixel error which caused many problems
Stacks : masking strategy	3/26	PAP	DONE
Plan for 3pi re-processing for stacks	4/30	PAP
Full-Scale Photometry & Astrometry Relative Calibration Demo	3/31	EAM	DONE	once with SAS, again with 3pi yz data
IPP->PSPS : missing detection fields	4/31	RH		requires input from EAM?
IPP->PSPS : missing difference fields	4/31	RH		requires input from EAM?
IPP->PSPS : stack tables	4/31	RH		blocked by 'extended source' task above
IPP->PSPS : finalize recalibration process	4/31	RH		blocked by 'Full-Scale Photometry.. Demo'
ISP automation : create detrends	3/31	HAF	DONE
ISP automation : update recipes	4/7	HAF	DONE
Distribution of Detrend Images	3/31	CZW	DONE
Integrated Release of Uncorrected Data	4/30	PAP

Tasks for the Immediate Future (these overlap the above)

• major missing features : required for the start of the survey
  • detectability server (integration with postage stamp server in progress)

• major missing features : important, but not blocking start of the survey
  • extended source analysis
  • stack photometry
  • ipp->psps interface
  • integrated release of uncorrected data
  • processing recipe for final stacks

• major demo month lessons to be addressed
  • keeping the pantasks queues filled
  • address large load spikes
  • operational rigidity
  • how to deal with down machines (on-the-fly replication?)
  • address real vs nfs failures
  • address real failures
  • more load and disk space balancing (machine limits by job?)
  • better targeting of data to nodes
  • more regular operations schedule - procesing czar
  • better tools for examining current state (more ippMon? more other tools?)

• current dev tasks
  • addstar imports from UKIDSS & SDSS
  • optimize dvo image lookups

• cluster management
  • automate cleanups of un-destreaked images
  • define reboot procedure

• psphot work:
  • 'scale error' in stacks
  • limit max source detections in psphot

• write ups
  • ppSub updates
  • DVO usage

Eugene Magnier

• email regarding data requests

• binned 4x4 mask for ken
• extended sources
  • set upper limits on Petrosian radii
  • generate a more complete test set & run tests
• ingest giebink stuff into svn trunk
• poor pixels flagged in detections

Serge Chastel

• Learn to be a czar: DONE
• Contribute to ippMonitor generation tool
• PSPS beta testing
• MySql queries optimization in IPP

Heather Flewelling

• ippMonitor detrend verification plots
• other ippMonitor work (bug fixes and I want to add missing tables, check in my changes)
• detrend server work (I want to add things like reverts, to make it more like the other pantasks)
• dvomerge babysitting
• create new dark

Bill Giebink

• DIMM work
• psLib & psModules test code
• build a "realtest"

Roy Henderson

• DVO
  • DONE: create DVO Db for SAS with current data
  • create DVO Db for SAS with reprocessed data (after Gene's changes)
  • create DVO Db for SAS with reprocessed data and a range of detection matching diameters
  • DONE: work on framework to filter out detections during addstar import based on image flags
• ippToPsps:
  • test code against Gene's large DVO Dbs
  • produce diffs for PSPS:
    • bring code up to date
    • work on unresolved fields
    • test load to PSPS
  • produce stacks for PSPS:
    • update stack tables in schema
    • develop code within ippToPsps to populate tables
    • load MD04 to PSPS
  • object batches for PSPS
• PSPS:
  • work on test plan for interface
  • DONE: test and load 'SCR' 'survey' to sandbox then Maui
• Czartool:
  • DONE: show ns.show.dates someplace
  • generate automatic warning emails to ipp-dev
  • DONE: add burntool stats
  • add update and pstamp stuff
  • list stuff pending from previous day

Paul Price

1. MOPS support
2. Pipeline for uncorrected data
3. Write up PSF-matching in IPP

• Add skycell_id and tess_id to headers of diff + stack outputs (incl cmfs)
• Check use of FWHM for ppSub scaling

Bill Sweeney

• look into importing UKIDS data into dvo catalogs
• make sure SDSS import to DVO is working properly
• postage stamp server
  • automate handling of 'unable to update' errors
  • enhance web interface with new features especially parse results file and display
  • improve access control add quotas
  • update user documentation
  • meet with NCU folks on pstamp web interface
  • update database tables to support automatic cleanup
• automate distribution cleanup
• data store work

Chris Waters

• check the most heavily masked chips, and see if they can be unmasked
  • Look into the detector linearity of odd features, and implement a software correction in ppImage
• Fix the detectability server bug that seems to drop rows, #1420.
• Redesign replication server to cull duplicate copies as well.
• Implement a real-time burntool correction, add to image registration task
• Fix WCS in summary stage output fits images.
• Figure out why a down host prompts so many neb-replicate requests it swamps the database server.
• integrate CON_VSUM information into the headers of new detrends
• integrate tools/ directory
• sas relphot & relastro
• DONE: probably not tested: Add SweetSpot lunation-based processing to nightly science code.

Cindy Giebink & Steph Mohr

• ipp005 bios needs to be reset (wait for Gavin)
• debug hardware problems on ipp008, 9, 14, 16, 18
• move homedirs to ippc18
• areca scripts for automatic verify
• try neb-hosts with Chris
• work with Bill Sweeney to get otis