Up: IPP at the ROE

IPP Data Products we (ROE Weak Lensing Group) require for Weak Lensing analysis

Pointings

There are 10 pointings customised for high-redshift supernova searches. These are the Medium Deep Surveys. Each field has a footprint the size of the detector.

There is also a deep pointing of Andromeda (MD00).

The WL group will be using data from all 11 of these pointings.

Field	Required	Available
MD01	X	X
MD02	X	X
MD03	X
MD04	X
MD05	X
MD06	X
MD07	X
MD08	X	X
MD09	X	X
MD10	X	X
11 (M31)	X

Products for the Medium Deep (MD) surveys

Imaging is done in 5 filters: g r i z y, with the highest integrated throughput in r & i.

	g	r	i	z	y
raw
chip		f	f
camera		c	c
warp		f	f
stack	f	f	f	f	f

• f => full
• c => clean

"Distribution bundles are built in either full or clean state. In the full state all of the associated data products are included. In the clean state the larger images are omitted. Once the dependent products are available at the mirror site, a run can be set to update state which causes the images to be re-created."

-- ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/GPC1_DataDistribution_Overview

All products for a later production stage can be generated from a full bundle. But though that implies all we really need is the full state of clean, the CPU time isn't worth it. Best to just grab the full warps, for example.

Data Storage Requirements

• 1.4e9 pixels
• 10 bytes/pixel
  • This is the number thrown around, and it comes from either:
    • One
      • 14 or 15 bits/pixel from ADC. In either case, requires 2 bytes / pixel / filter
      • 5 filters
    • Two
      • (2 bytes [signal] + 2 bytes [weight] + 1 byte [mask]) / pixel / filter = 5 bytes / pixel / filter
      • 2 filters
• 0.3 compression factor ( or 0.44, depending on whose numbers)
• 11 fields (10 MDS + M31)
• ~100 images

=> ~4 to 6 TB

Since it's an approximation, budget for 8 TB Medium Deep Survey pointings

Also, the following image is WRONG. The background map needs to be flipped L-R. Hence the Andromeda (M31) pointing at the far right should really be over Andromeda, which in this map is at the far left, angled the wrong way.

-- EricTittley - 2009-11-06