IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 36 and Version 37 of Stack_Sky_Levels


Ignore:
Timestamp:
Oct 3, 2014, 2:53:18 PM (12 years ago)
Author:
watersc1
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • Stack_Sky_Levels

    v36 v37  
    22
    33In light of continuing confusion on the psphot background measurements, I performed a series of tests to verify the consistency and possible error sources of the sky levels from the raw to stack stages for the 12-input test stack for MD04/skycell.055 from exposures (o6538g0139o,o6540g0157o,o6540g0213o,o6543g0266o,o6550g0059o,o6558g0328o,o6567g0066o,o6569g0065o,o6569g0073o,o6569g0081o).  These exposures were chosen to match the test stack Peter made.  All IPP processing was done using the standard processing recipes, although with the random seed fixed to the value 1234.  This ensures that differences due to pixel selection are removed.  In addition, each processing stage was repeated both with the standard quantized integer output images as well as floating point uncompressed images.  Image histograms were performed in MANA, using a fixed 0.25 bin width for all images.  RAW and CHIP stage images had pixels with any masked bits replaced with NAN values, and were therefore removed from further consideration.  WARP and STACK images did not have this operation performed, as they have bad pixel data replaced with NAN before being written out.  Non-IPP Gaussian fits were run on these histograms, with all unpopulated and end point bins removed.  MANA assigns points outside of the histogram range to these bins, which were chosen to be sufficiently far from the sky distribution to be influential (RAW: +/- 10000, CHIP/WARP: +/- 100, STACK: +/-2000).
     4
     5=== RAW ===
     6
     7The images grouped in the RAW stage are not true raw OTA data. They have been processed with ppImage to apply all applicable detrends, and to mosaic the images into the standard 4800x4800 arrangement. However, no background subtraction was performed, leaving any sky values in place. The following plots shows one of the inputs to the stack, o6543g0266o XY33.  The top row shows the results for the compressed output image, with the uncompressed images on the bottom.  The logarithmic plots on the right indicate that the lower tail of the pixel distribution is Gaussian.
    48
    59|| || linear || log ||
    … …  
    711|| nocomp|| [[Image(02_raw_o6543g0266oXY33.nocomp.png,400px)]] || [[Image(03_raw_o6543g0266oXY33.nocomp.log.png,400px)]] ||
    812
    9 asdf
     13The next set of plots compare the background mean and sigma value calculated by psphot against the external Gaussian fits.  The means agree within a count, although there is a slight trend with sigma (and, as the mean value here is essentially sigma^2, with the mean as well), such that psphot produces higher background levels than the simple Gaussian fit (for the example OTA, the fit values are PSPHOT: (744.387622, 29.123441) GAUSSFIT: (744.217 30.1759).
    1014
    1115|| || delta mean || delta sigma ||
    … …  
    1418|| sigma/zoom ||                                       || [[Image(06a_raw_sigma_deltasigma.png,400px)]] ||
    1519
    16 
    17 asdf
     20=== CHIP ===
     21
     22The CHIP stage product has had the psphot background subtracted from it, and so is expected to be centered on zero.  Following the same OTA as before, we find that the distribution is at zero, and running the measurements again on the image yield PSPHOT: (0.091377, 27.990451) GAUSSFIT: (-0.00620696 29.5627) for the compressed image, and PSPHOT: (0.016442,28.999453) GAUSSFIT: (0.0406877 29.5508) for the uncompressed image.
     23
     24All methods agree that the pixel are within +/- 0.1 of zero.  Given that the compressed/gaussfit mean is effectively zero, but a close check of the logarithmic plot shows that that fit is biased low, there does seem to be a non-zero positive bias remaining on this image. 
     25
    1826|| || linear || log ||
    1927|| comp || [[Image(10_chip_o6543g0266oXY33.comp.png,400px)]] || [[Image(11_chip_o6543g0266oXY33.comp.log.png,400px)]] ||
    2028|| nocomp|| [[Image(12_chip_o6543g0266oXY33.nocomp.png,400px)]] || [[Image(13_chip_o6543g0266oXY33.nocomp.log.png,400px)]] ||
    2129
    22 asdf
     30Looking at the distribution for all the OTAs, a similar sigma based trend in the mean is visible between the two methods, with the compressed data having a larger effect.  Given that the BSCALE values for the chip images have a median of 2.298 (mean 2.83442, sigma 1.81574, min 1.38277, median 2.29866, max 15.38), this is at least partially an issue with the rebinning for the Gaussfit histogram.
     31
     32 
    2333|| || delta mean || delta sigma ||
    2434|| mean || [[Image(14_chip_mean_deltamean.png,400px)]] || [[Image(15_chip_mean_deltasigma.png,400px)]] ||
    2535|| sigma || [[Image(17_chip_sigma_deltamean.png,400px)]] || [[Image(16_chip_sigma_deltasigma.png,400px)]] ||
    2636
    27 asdf
     37=== WARP ===
     38
     39The following plots the skycell.055 warp for o6543g0266o, along with all the OTAs that contribute to the warp.  I did not trim the OTA to exclude regions not included in the warp, and the scaling of the OTA data is set to match the peak values.  The warp distribution is narrower than any of the OTAs.  The warping process produces an output pixel that is a combination of a set of input pixels.  This is a smoothing process, and so the distribution of pixels on a warp is expected to have a smaller sigma than on a chip.
     40
     41The logarithmic plots show that the Gaussian fit matches the central distribution somewhat tighter than in the chip.  The fit values for this warp are PSPHOT: (-0.010854,24.262478) GAUSSFIT: (0.0364079 25.6469) for the compressed image and PSPHOT: (-0.033963,25.211044) GAUSSFIT: (0.078182 25.6508) for the uncompressed image.
     42
    2843|| || linear || log ||
    2944|| comp || [[Image(20_warp_o6543g0266oXY33.comp.png,400px)]] || [[Image(21_warp_o6543g0266oXY33.comp.log.png,400px)]] ||
    3045|| nocomp|| [[Image(22_warp_o6543g0266oXY33.nocomp.png,400px)]] || [[Image(23_warp_o6543g0266oXY33.nocomp.png,400px)]] ||
    3146
    32 asdf
     47The warp distribution is similar to before, with a slight trend of increasing difference with sigma.  The compressed fits tend to be slightly higher than the uncompressed fits.  There is again an offset and larger trend in the sigma difference with the compressed data.
    3348
    3449|| || delta mean || delta sigma ||
    3550|| mean || [[Image(24_warp_mean_deltamean.png,400px)]] || [[Image(25_warp_mean_deltasigma.png,400px)]] ||
    3651|| sigma || [[Image(27_warp_sigma_deltamean.png,400px)]] || [[Image(26_warp_sigma_deltasigma.png,400px)]] ||
    37 asdf
     52
     53=== STACK ===
     54
     55The previous plots also have the values for the stack included, with a scaling factor of 14 applied (following Peter's scaling relation between the stack and warps).  Despite the labels, the output stack that was used was uncompressed.  The "comp" stack is a stack produced from the compressed warp data.  The stacks fall along the trends for the uncompressed warps, but there is no significant difference in the two stacks; ppStack produces nearly identical outputs regardless of whether it reads in compressed or uncompressed warps.
     56
     57The following pixel histograms for the stack have not been scaled by 14, and present the true image values.  The fit values for these are PSPHOT: (-0.697008,221.535437) GAUSSFIT: (0.552266 224.454) for the compressed input stack, and PSPHOT: (-1.054158, 221.984595,) GAUSSFIT: (0.552266 224.454) for the uncompressed input stack.
    3858|| comp || nocomp ||
    3959|| [[Image(30_stack.comp.png,400px)]] || [[Image(31_stack.nocomp.png,400px)]] ||