Changes between Version 3 and Version 4 of Basic_GPC1_Processing_for_End_Users
- Timestamp:
- Mar 3, 2009, 7:41:39 AM (17 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Basic_GPC1_Processing_for_End_Users
v3 v4 12 12 At Harvard we use the 'module load' system to configure things on the cluster 13 13 14 {{{ 14 15 module load hpc/ipp 15 16 psconfig default 17 }}} 16 18 17 19 and here's where we keep the data 18 20 21 {{{ 19 22 cd /n/data1/panstarrs/GPC1/MD01_20081002_1 23 }}} 20 24 21 25 In this directory are each of the different observations. The directories are named by the observation number. These span the course of several weeks. 22 26 27 {{{ 23 28 [wwoodvas@heroint2 MD01_20081002_1]$ ls 24 29 36475/ 36476/ 36477/ 36478/ 36479/ 36480/ 36481/ 36625/ stack/ 30 }}} 25 31 26 32 I created the 'stack' directory as a place where we can put our coadd stacked images. The coadded images were not distributed with this data release. It wouldn't make sense to put the coadds under any particular directory because they don't belong to any particular observation so I've just created a directory for them here at the base level. … … 28 34 Each of the observation number directories contains directories for the raw, chip, warp, and diff images 29 35 30 <code> 36 {{{ 31 37 [wwoodvas@heroint2 MD01_20081002_1]$ ls 36475 32 38 chip/ diff/ raw/ warp/ 33 </code> 39 }}} 34 40 35 41 There are .log files in these directories detailing the steps and output that were used to create them. You may find referring to them a useful exercise if you want to understand all of the details of what was done to derive the images. … … 41 47 * Photometry. Basic photometry with 'psphot' 42 48 43 <code> 49 {{{ 44 50 cd MD01_20081002_1/36475 45 51 psphot -D CAMERA gpc1 -file chip/o4741g0234o.36475.ch.11948.XY55.ch.fits \ … … 47 53 -weight chip/o4741g0234o.36475.ch.11948.XY55.ch.wt.fits \ 48 54 testout 49 </code> 55 }}} 56 50 57 will create testout.cmf. If you want to see what psphot is doing as it runs, you may be interested in the '-visual' option 51 58 52 <code> 59 {{{ 53 60 psphot -file chip/o4741g0234o.36475.ch.11948.XY55.ch.fits \ 54 61 -mask chip//o4741g0234o.36475.ch.11948.XY55.ch.mk.fits \ … … 56 63 -visual \ 57 64 testout 58 </code> 65 }}} 59 66 60 67 The main output from the psphot command will be the .cmf file, which is a binary FITS table that contains information about the objects as gleaned from the image. One of the fields that generates the most questions is the FLAGS value for objects. This is a bitmask. See [wiki:IPP_Detection_Bitmasks IPP Detection Bitmasks] for more details. … … 64 71 There are a variety of options and settings for psphot. These are controlled by command line optional arguments as well as by the "recipes" file located in the "share/ippconfig/" directory in your IPP installation. On my machine this is "/Volumes/data/PS1/code/ipp/default.darwin/share/ippconfig". The default recipes are in "share/ippconfig/recipes", with camera-specific overriding configuration files in the camera subdirectories in "share/ippconfig", e.g. "share/ippconfig/gpc1" for the PS1 GPC1 camera. 65 72 66 Here's an example of a recipes file for psphot: 67 68 <pre> 73 MWV extracted the psphot options and meanings from psphotArguments.c from an early January 2009 CVS build: 74 75 {{{ 76 -version # Version string 77 -threads # Number of processing threads to run simultaneously 78 -modeltest # run the test model (requires X,Y coordinate) 79 -model # specify the modeltest model 80 -fitmode # specify the test fit mode 81 -fitset # ???-photcode # photcode : used in output to supplement header data (argument or 82 recipe?) 83 -visual # visual : interactive display mode 84 -break # break : used from recipe throughout psphotReadout-fitmode # fitmode : used from recipe throughout psphotReadout 85 # [MWV: Seems inconsistent with previous fitmode] 86 -region # analysis region : overrides recipe value, used in psphotReadout/psphotEnsemblePSF 87 -chip # chip selection is used to limit chips to be processed 88 }}} 89 90 The bulk of psphot behavior is specified by the recipes. Here's an example of a recipes file for psphot: 91 92 {{{ 69 93 SAVE.OUTPUT BOOL TRUE 70 94 SAVE.BACKMDL BOOL FALSE … … 131 155 PSF_CLUMP_NX S32 1 # subdivide image in to NX x NY regions for 132 156 PSF_CLUMP_NY S32 1 # selecting PSF stars 133 </pre> 157 }}} 134 158 135 159 … … 139 163 * DVO. Let's say we have a set of cmf files and we want to use DVO to combine the detections from these files. A series of commands such as 140 164 165 {{{ 141 166 cd MD01_20081002_1 142 167 addstar -D CAMERA gpc1 -D CATDIR mycatdir -accept-astrom 36475/chip/o4741g0234o.36475.ch.11948.XY55.cmf … … 144 169 addstar -D CAMERA gpc1 -D CATDIR mycatdir -accept-astrom 36477/chip/o4741g0236o.36477.ch.11950.XY55.cmf 145 170 addstar -D CAMERA gpc1 -D CATDIR mycatdir -accept-astrom 36478/chip/o4741g0237o.36478.ch.11951.XY55.cmf 171 }}} 146 172 147 173 will create a DVO database and fill it with the detections in the given CMF files. Since these are the same pointings we should get multiple detections (matches) when we put together the catalog. Currently -accept-astrom is still necessary until the WCS solutions finalizes and we know our true tolerances for good WCS. Need other page reference or more notes to explore DVO further. … … 153 179 * To create warped images that conform to a given position on the sky, we need to define how we want to break up th sky. This tessellation is what IPP will use to map the chip images to what we call skycells. 154 180 181 {{{ 155 182 skycells -D CAMERA gpc1 8 -scale 0.2 -D CATDIR default 183 }}} 156 184 157 185 This will create a directory ('default') that will contain FITS files that define the tessellation of the sky. Now we can recreate the warping of chip images to warp images: … … 159 187 First let's create a sandbox for ourselves to put our new files into 160 188 189 {{{ 161 190 mkdir -p workspace/warp workspace/stack workspace/diff 191 }}} 162 192 163 193 Now we'll fill a skycell. You may wonder how to know what skycell a particular chip will warp to. That is a good question that currently has an answer that's too complicated to be practical … … 167 197 For now just take the following example: 168 198 199 {{{ 169 200 dvoImageExtract skycell.02879.51 -D CATDIR default -D CAMERA gpc1 -o skycell.02879.51_template.fits -D CAMERA gpc1 170 201 171 202 pswarp -file 36475/chip/o4741g0234o.36475.ch.11948.XY33.ch.fits -mask 36475/chip/o4741g0234o.36475.ch.11948.XY33.ch.mk.fits -weight 36475/chip/o4741g0234o.36475.ch.11948.XY33.ch.wt.fits workspace/warp/o4741g0234o.36475.wrp.11948.skycell.02879.51 skycell.02879.51_template.fits 203 }}} 172 204 173 205 ==== Stacking ==== … … 175 207 If we have a set of images that we have warped to the same skycell and we want to stack (coadd) these images together we first specify a file that defines the images, masks, weights, PSF models, etc. for the input images to the stack: 176 208 177 <code><pre> 209 {{{ 178 210 INPUT0 METADATA 179 211 IMAGE STR 36475/warp/o4741g0234o.36475.wrp.1277.skycell.02879.51.fits … … 255 287 SCALE F32 1.0 256 288 END 257 </pre></code> 289 }}} 258 290 259 291 If we called this file 'skycell.02879.51.mwv.002.mdc', then we might run the following to get the output stack … … 267 299 Once we have a coaddition that hopeful has enough dithers to fill in the area we will want to run subtractions of individual images against that coadd template. In this case we built the coadd from the images themselves, but that can't be helped for now as long as we have limited data. 268 300 301 {{{ 269 302 set ref=skycell.02879.51.mwv.002 270 303 set new=36625/warp/o4741g0239o.36625.wrp.1283.skycell.02879.51 … … 273 306 -refmask $ref.mk.fits -refweight $ref.wt.fits \ 274 307 -sources $new.cmf -type ISIS -threads 2 -photometry 308 }}} 275 309 276 310 would run a subtraction of $new minus $ref using the ISIS kernels. ppSub --help will generate a somewhat detailed list of options but they probably won't all make sense on a first reading.
