Current Tasks for the IPP Team

(Up to Main Page)

IPP_PS1_GrandReprocessingPrep

(Recently completed tasks)

(DRAVG Issues and Concerns)

(Ken's list )

Notes from WBS entries

a) Tasks required before the start of the survey:

Task	Due	Who	State	Notes
IPP to OTIS interface	3/12	CZW	DONE	we now need to iterate with OTIS
Detectability Server : psphotDetect	3/19	CZW	DONE	output values must be double-checked
Detectability Server : pstamp integration	3/26	CZW	DONE	Tested and Committed.
Raw Distribution	3/12	WS	DONE
DVO large-file upgrade	3/21	EAM	DONE
DVO merge catalogs	3/25	EAM	DONE
DVO Image-indexing	3/27	EAM	DONE

b) Tasks not required for the survey start:

Task	Due	Who	State	Notes
Finish Extended Source Analysis Code	3/31	EAM		petrosians, surface brightness profile done
Stack Photometry Process	4/30	EAM		Initial version of the code is now working
Stacks : example PS1 astrometry	3/19	PAP	DONE	this led to discovery of a 1/2 pixel error which caused many problems
Stacks : masking strategy	3/26	PAP	DONE
Plan for 3pi re-processing for stacks	4/30	PAP
Full-Scale Photometry & Astrometry Relative Calibration Demo	3/31	EAM	DONE	once with SAS, again with 3pi yz data
IPP->PSPS : missing detection fields	4/31	RH		requires input from EAM?
IPP->PSPS : missing difference fields	4/31	RH		requires input from EAM?
IPP->PSPS : stack tables	4/31	RH		blocked by 'extended source' task above
IPP->PSPS : finalize recalibration process	4/31	RH		blocked by 'Full-Scale Photometry.. Demo'
ISP automation : create detrends	3/31	HAF	DONE
ISP automation : update recipes	4/7	HAF	DONE
Distribution of Detrend Images	3/31	CZW	DONE
Integrated Release of Uncorrected Data	4/30	PAP

Tasks for the Immediate Future (these overlap the above)

• major missing features : required for the start of the survey
  • detectability server (integration with postage stamp server in progress)

• major missing features : important, but not blocking start of the survey
  • extended source analysis
  • stack photometry
  • ipp->psps interface
  • integrated release of uncorrected data
  • processing recipe for final stacks

• major demo month lessons to be addressed
  • keeping the pantasks queues filled
  • address large load spikes
  • operational rigidity
  • how to deal with down machines (on-the-fly replication?)
  • address real vs nfs failures
  • address real failures
  • more load and disk space balancing (machine limits by job?)
  • better targeting of data to nodes
  • more regular operations schedule - procesing czar
  • better tools for examining current state (more ippMon? more other tools?)

• current dev tasks
  • addstar imports from UKIDSS & SDSS
  • optimize dvo image lookups

• cluster management
  • automate cleanups of un-destreaked images
  • define reboot procedure

• psphot work:
  • 'scale error' in stacks
  • limit max source detections in psphot

• write ups
  • ppSub updates
  • DVO usage

Eugene Magnier

• email regarding data requests

• binned 4x4 mask for ken
• extended sources
  • set upper limits on Petrosian radii
  • generate a more complete test set & run tests
• ingest giebink stuff into svn trunk
• poor pixels flagged in detections

Serge Chastel

• Contribute to ippMonitor generation tool
• PSPS beta testing
• MySql queries optimization in IPP
• MySql databases replication and optimization
• IPP-MOPS ICD / code: DONE / On going
• Tessellation improvements: find a better covering (DONE -- at least partially), visualization
• Manoa Cluster: identify the services that run on it. Proposal for re-organization.

Heather Flewelling

• answer skyprobe questions
• ppMerge things?
• book flight stuff for Harvard (this didn't work last Monday with the voucher at the airport)
• addstar
  • fix the bug I found but never fixed
  • backup scheming with Bill and Roy
  • start/stop/restart MDs and ThreePis? I don't know where we are (I'll ask Bill and Roy)
• create new dark

Bill Giebink

• DIMM work
• psLib & psModules test code
• build a "realtest"

Roy Henderson

• DVO
  • create DVO Db for SAS with reprocessed data (after Gene's changes)
  • create DVO Db for SAS with reprocessed data and a range of detection matching diameters
• ippToPsps:
  • produce stacks for PSPS:
    • update stack tables in schema
    • develop code within ippToPsps to populate tables
    • load MD04 to PSPS
  • produce diffs for PSPS:
    • bring code up to date
    • work on unresolved fields
    • test load to PSPS
  • object batches for PSPS
• PSPS:
  • work on test plan for interface
• Czartool:
  • generate automatic warning emails to ipp-dev
  • fix update and pstamp stuff
  • list stuff pending from previous day

Paul Price

1. MOPS support
2. Pipeline for uncorrected data
3. Write up PSF-matching in IPP

• Add skycell_id and tess_id to headers of diff + stack outputs (incl cmfs)
• Check use of FWHM for ppSub scaling

Bill Sweeney

• look into importing UKIDS data into dvo catalogs
• make sure SDSS import to DVO is working properly
• postage stamp server
  • automate handling of 'unable to update' errors
  • enhance web interface with new features especially parse results file and display
  • improve access control add quotas
  • update user documentation
  • meet with NCU folks on pstamp web interface
  • update database tables to support automatic cleanup
• automate distribution cleanup
• data store work

Chris Waters

• TESTING: Look into the detector linearity of odd features, and implement a software correction in ppImage
• DONE: Speed up code to delete old destreak diff data to free up disk space.
• Track down ghosts identified at QUB transient meeting.
• Nightly science:
  • Ensure nightly science code properly handles OSS diff pairs.
  • Implement a real-time burntool correction, add to image registration task
• Nebulous:
  • Rework neb-shift to handle load balancing.
  • Redesign replication server to cull duplicate copies as well.
  • Figure out why a down host prompts so many neb-replicate requests it swamps the database server.
• Miscellaneous:
  • Fix WCS in summary stage output fits images.
  • integrate CON_VSUM information into the headers of new detrends
  • integrate tools/ directory
  • sas relphot & relastro
• DONE:
  • probably not tested: Add SweetSpot lunation-based processing to nightly science code.
  • TEST: Fix the detectability server bug that seems to drop rows, #1420. Fix my fix that I fixed from Belfast.
  • Speed up code to delete old destreak diff data to free up disk space.

Cindy Giebink & Steph Mohr

• ipp005 bios needs to be reset (wait for Gavin)
• debug hardware problems on ipp008, 9, 14, 16, 18
• move homedirs to ippc18
• areca scripts for automatic verify
• try neb-hosts with Chris
• work with Bill Sweeney to get otis