Large-Scale Masking Analysis

(Up to the Extra IPP Features Page)

How can we convert the masks generated for each image into a resource which helps people to determine the filling factor of their particular science? Here are some thoughts:

• we generate a mask image with each processed science exposure. These are also warped with the image pixels to skycells. The masks are compressed, and thus small (1% of the image data volume).

• we ingest exposures into a DVO database, and that is made available to people for their studies.

• anyone who cares only about a specific object or small area can always request the relevant masks from the postage stamp server

• The total data volume of per-exposure masks generated to date is in the vicinity of 2TB.

• Option 1) set up a distribution system for all masks to provide all masks to those who so desire.
  • this scheme, or others below, could use Mario Juric proposed mask compression tool to further reduce the volume.

• Option 2) set up an analysis stage which generates binned mask images on various scales
  • this system could start with the warped skycell masks
  • these masks could be combined into a lower resolution set of super-skycells
  • a static-sky worth of masks at full resolution should require ~200GB per filter
  • binning by 5x5 (1 arcsec pixels) gets us to the 10GB range.
  • what scales should be generated?

• Option 3) set up a system to rebin and distribute the masks, to reduce the volume of data I/O