Summary for MD.PV3 (updates in progress)

• MediumDeepFields summary page -- quick links to all up to date refstacks
• MD.GR0 summary page -- summary of all reprocessing details and 2011/2012 refstacks and some deepstack samples
• MD.PV1 summary page -- the data set that PV2 update reprocessing is based on
• MD.PV2 summary page -- update type reprocessing

This is a summary of the MD.PV3 full pixel reprocessing. Start date and schedule TBD.

This processing will also involve other MD-like fields, including M31, STS and CNP. Criteria for the stacks and diffims are TBD. From Ken's talk at PS1SC@STSCI meeting in June 2014, the diffims described there may need to modifications:

• stack:
  • MDN -- night stack
  • MD0 -- seasonal reference stack
  • MDD -- deep stack
  • MDT -- iq stack
• diffim:
  • warp-stack (WS) -- (warp-ref) only, no (warp-deep) or only for CNP?
  • stack-stack (SS) -- must add (night-ref) but no (stack-deep)

field	stage	N imfile/skycells	ave_dtime(s)	sig_dtime(s)	status	comment	label/data_group, release name
MD	chip/camera/warp	~2500k/41717/2500k	130/610/85	130/500/65	TBD	41717 exposures *60 imfiles/55-60 skycells	MD<f#>.pv3.20141001?
	night stk	~291k	860	500	TBD	typically 8 inputs, sometimes >20 in marginal nights, ~4849*55-60 skycells
	ref stk	15141	6700	1500	TBD	Nin~80, 5 seasons
	deep stk	3033	37.5k*	13k	TBD	*using 8 theads
	iq stk	3325	2215	310	TBD	edge skycells of course run faster in all stacks
	WSref diff	2500k	?	?	TBD	done by season, same number as warps
	WSdeep diff	x	x	x	x	not going to do
	SS diff	~291k	?	?	TBD	done by season, same number as night stacks
	night staticsky	~291k	110	40	TBD	same number as night stacks (no filter groups)
	ref staticsky	3025	4960	1700	TBD
	deep staticsky	605	6300	2550	TBD
	iq staticsky	665	4660	1600	TBD
	night skycal	~291k	30	43	TBD	same as for staticsky since single filter
	ref skycal	15093	26	5	TBD
	deep skycal	3008	31	8	TBD
	iq skycal	3018	40	34	TBD
	full-forced phot	2500k	?	?	TBD
CNP	chip/camera/warp	/2695/	~115/410/60	150/430/125	TBD
	ref/deep stk	~500?	X	x	TBD	Nin~ , seasons (first/last month for ref?)
	WSref/deep diff	~2695	X	X	TBD
	ref/deep staticsky
	ref/deep skycal	~	~37	~29	TBD
	full-forced phot		?	?	TBD
M31	chip/camera/warp	/4347/	129/624/62	190/313/30	TBD
	background restore
	ref stk	~500?	X	x	TBD	Nin~?, 4 seasons
	deep stk	?
	WSref diff	~	X	X	TBD
	ref staticsky
	ref skycal	~	~37	~29	TBD
	full-forced phot		?	?	TBD
STS	chip/camera/warp	/16606/	210/1200/100	120/575/70	TBD	Nin~?, 4 seasons
	ref stk	~3500?	X	x	TBD	Nin~
	deep stk	?
	WSref diff	~	X	X	TBD
	ref staticsky
	ref skycal	~	~37	~29	TBD
	full-forced phot		?	?	TBD

• label/data_group, release name -- for gpc1 DB queries/datastore identification and specific postage-stamp server (PSS) association
• ave/sig_dtime -- roughly the dtime_script for the jobs in the gpc1 DB

If run only on the 15 ippsXX machines with high memory, 12 cores -- individual processing estimates only, some can be run ~same time like deep+iq/night/dif/staticsky/skycal possibly

• MD: will be run one field at a time to limit the queued jobs
  • chip/cam/warp ~6 jobs/node ~90 jobs, ~42+3+27 ~72 days
  • iq stack ~4 jobs/node ~60 jobs,
  • night stack ~4 jobs/node ~60 jobs,
  • ref stack ~3 jobs/node ~45 jobs,
  • deep stack ~1 jobs/node ~15 jobs, ~88 days (pv2 took ~70d, may take longer for pv3 if don't let as much get clipped for Nin on edge skycells)
  • WSref diff ~4 jobs/node 60 jobs
  • SSref diff ~4 jobs/node 60 jobs
  • all staticsky + skycal ~4 and 8 jobs/node 60 and 120 jobs
  • total ~
• CNP:
• M31:
• STS:

Issues to address:

• disk space limiting
• chip-warp:
  • refcat oddities in field
  • background restore M31 and masking for M31/STS -- test samples to be run
• stack
  • stack quality fault
  • night stack config, refstack edge loss
  • ref+deep stack input cuts
  • iq+ref+deep test stacks for CNP,M31,STS
• staticsky/skycal
  • zeropt recalib in skycal
  • stack photometry sky oddities
• diffim
  • verify similar photometry
  • verify similar detections/coverage (no sig loss on edges)
  • test samples for CNP,M31,STS
• full-forced photometry -- unknown, untested still

_General Summary Status:_

• 10/1 start chip-warp for MD should be possible, STS+M31 sample should be possible

---

chip -- camera -- warp

Should be started ASAP, so that the warps are available for some early testing of the stacks and diffims.

Testing necessary for the M31, STS and CNP processing still.

---

stack

Order of operations important -- while deep stacks take 1.5 months or so to finish, night and seasonal reference stacks must be finished before diffims can be started. Lighter jobs like difference images and iq (even reference stacks) can be run along with the deep stacks for some fields.

night

seasonal reference

deep

iq

photometry

---

difference image

Cannot start until both night and seasonal reference stacks finish.

---