Changes between Initial Version and Version 1 of 2.2m-UH-Tek

Timestamp:

Feb 24, 2009, 4:23:53 PM (17 years ago)

Author:

trac

Comment:

--

2.2m-UH-Tek

@@ +1 @@
+Here are the issues/fixes I ran into while running IPP 2.1 on 2.2m-UH Tek data using a Linux machine. By Julia Fang
+
+== Configuration ==
+
+ * Modifications to ~/.ipprc
+I added two more data directories:
+
+ TEKDATA                STR     /home/group12/jfang/TEKDATA/
+ TEKWORK                STR     /home/group12/jfang/TEKWORK/
+
+I also changed my database information:
+ DBNAME         STR     tek
+
+I added camera information for TEK under "Setups for each camera system" and "camera names as expected by DVO". I also made sure that the TEK camera was located above the simple camera.
+ CAMERAS                METADATA
+ TEK            STR     tek/camera.config
+
+ DVO.CAMERAS            METADATA
+ TEK            STR     tek
+
+ * Initiating new database
+I created a new database called tek and granted permissions to the other computers in my cluster to access this database.
+
+ * Fits Headers of Input Images
+I used IRAF and the command "imhead file.fits lo+" to edit the headers of my fits files. The telescope, instrument, and detector names were changed. I had to add dashes to the telescope and detector names because IPP wouldn't allow spaces. The instrument name was changed to "TEK". Other additions to the fits headers:
+
+ OBSTYPE = 'object  '
+ FILTER  = 'R       '
+ UTC-OBS = '2007-06-13'
+
+I added the OBSTYPE, FILTER, and UTC-OBS.
+
+ * Tek's configuration files
+I added a tek config folder under /jfang/psconfig/default.linrh/share/ippconfig/ and named it "tek". In this folder, I copied over isp camera's configuration files from /jfang/psconfig/default.linrh/share/ippconfig/isp/ and edited them:
+
+In format.config, cmf.config, cmp.config:
+Edited the TELESCOP, INSTRUME, and DETECTOR fields. 
+I modified these fields:
+ FPA.NAME       STR     NAXIS
+ CELL.TRIMSEC.SOURCE    STR     VALUE
+ CELL.BIASSEC.SOURCE    STR     HEADER
+ CELL.TRIMSEC           STR     [21:2068,1:2048]
+ CELL.BIASSEC           STR     BIASSEC
+ FPA.FILTER     STR     FILTER
+ CELL.XBIN      STR     CCDBIN1
+ CELL.YBIN      STR     CCDBIN2
+
+In dvo.config:
+ CATDIR                 /media/usbdisk/scr1/catdir.synth.grizy/
+ DATE-KEYWORD           DATE-OBS
+ DATE-MODE              YYYY-MM-DD
+ UT-KEYWORD             UT
+ JD-KEYWORD             JD
+
+In ppImage.config:
+I turned off all dark and shutter fields to FALSE.
+
+In psastro.config, I changed the catalog path:
+ DVO.CATDIR STR /media/usbdisk/scr1/catdir.synth.grizy/
+
+In camera.config, I modified file formats and filter:
+ FORMATS         METADATA
+        TEK     STR     tek/format.config
+        CMP     STR     tek/cmp.config
+        CMF     STR     tek/cmf.config
+ END
+
+ FILTER.ID       METADATA
+ R        STR     R
+ END
+
+ DVO.CAMERADIR  STR     tek
+
+== Data Injection ==
+
+ * How to inject data
+I used the ipp_serial_inject.pl command. Looked at the notes for Essence Test to figure it out. 
+
+ > ipp_serial_inject.pl --workdir path://TEKWORK/ --dbname tek /home/group12/jfang/TEKDATA/*.fits
+
+== Detrend Construction ==
+
+ * Errors in trying to get any master image
+I went to Tek's camera.config and modified these lines to fix the problem:
+
+   PPIMAGE.CHIP          OUTPUT  {OUTPUT}.chip.fits   IMAGE     FPA        TRUE      NONE
+   PPIMAGE.CHIP.MASK     OUTPUT  {OUTPUT}.chip.mk.fits   MASK     FPA        TRUE      NONE
+   PPIMAGE.CHIP.WEIGHT   OUTPUT  {OUTPUT}.chip.wt.fits   WEIGHT     FPA        TRUE      NONE
+
+I also had to check Tek's rejections.config and changed IMFILE.STDEV and EXP.STDEV to a higher number like 10 so that it won't be so easily rejected.
+
+ * Problem where dark subtraction didn't seem to be properly turned off
+Turns out Tek's ppImage recipe didn't contain the recipe used for flat-fielding. I went to the simple camera and copied over the long list of detrend steps (at the end of the file) and turned on/off the appropriate steps.
+
+ * Error while trying to get master flat: "unmatched statement (null)"
+This was fixed by going to Tek's format.config. Under DEFAULTS, set CELL.BAD to 0 and CELL.READNOISE to 10.
+
+ * Still could not get a master flat
+In the end, turns out there was a bug in ppMerge, the code wasn't checking for NAN in the image, and when it generated one, it wasn't being masked for the statistics. So I got the new, fixed files from Paul Price and recompiled psModules, ppMerge, and ppStats. This solved the problem.
+
+== Chip and Camera ==
+
+ * Can't find expected output file: path://TEKDATA/nt2154.chp6.chip.mk.fits
+Actually the real error here was in Tek's format.config file. CELL.TRIMSEC and CELL.BIASSEC were actually on the same line although they did not appear to be. So I added a line return between them.
+
+ * No WCS metadata in header
+I needed WCS data in my Tek headers, specifically a CD matrix and CRVAL1 and CRVAL2. I added this information to my fits headers (as well as other optional WCS information):
+
+ CTYPE1  = 'RA---TAN'
+ CTYPE2  = 'DEC--TAN'
+ CRPIX1  =                 1024
+ CRPIX2  =                 1024
+ CRVAL1  =               358.37
+ CRVAL2  =              54.9575
+ CDELT1  =                 0.22
+ CDELT2  =                 0.22
+ CD1_1   =         -6.100000E-5
+ CD1_2   =                    0
+ CD2_1   =                    0
+ CD2_2   =          6.100000E-5
+
+The CD matrix determined rotation. I figured this out by going to ds9 and scaling my image to a DSS image of the same sky.
+
+ * Problem with camera processing
+I added the following lines to PPIMAGE_J1 and to PPIMAGE_J2 in /recipes/ppImage.config to fix this:
+ PHOTOM         BOOL    FALSE
+ ASTROM.CHIP    BOOL    FALSE
+ ASTROM.MOSAIC  BOOL    FALSE
+
+ * How do rerun chip/camera step
+Queue it up again:
+ > chiptool -queue -exp_type OBJECT
+Or you can replace exp_type with whatever field you would like to use in your queue, like a specific image you want to queue up.
+
+== Warping ==
+
+ * No valid photcode is supplied
+To fix this, I went to /ippconfig/dvo.photcodes and put in the appropriate photcode entry for Tek:
+ 4405  Tektronix-2048^2-CCD.R dep   25.00  0.00     -       -  -   -      -        1
+
+ * Bad astrometric solution in header (for 2 out of 7 images)
+Fixed by modifying the psastro.config in /tek and /recipes. I changed the ranges and offset to larger values for pmAstromGridMatch and pmAstromGridAngle. I also went to check those two images with the DSS image again and looked for 1) correct RA and DEC as well as 2) correct rotation. Then I reran the chip and camera step.
+
+== Difference ==
+
+ * What to do
+I went to /jfang/TEKWORK/tess_Skycells/ to look for all the skycells with at least two warps in them (so that way something can be differenced). A - B (template). 
+
+For each of those skyfiles, use a series of 4 difftool commands like the following examples:
+ > difftool -definerun -workdir path://TEKWORK/ -skycell_id skycell.0372023 -tess_id skycells
+ > difftool -addinputskyfile -diff_id 1 -warp_id 1 -skycell_id skycell.0372023 -tess_id skycells
+ > difftool -addinputskyfile -diff_id 1 -warp_id 2 -skycell_id skycell.0372023 -tess_id skycells -template
+ > difftool -updaterun -diff_id 1 -state run 
+
+ * Unable to find PPSUB.OUTPUT within metadata
+I went to simtest/camera.config to copy over the all the fields for PPSUB and PPSTACK (do a search through the file for them) to Tek's camera.config. For some reason those were missing.
+
+ * Error performing subtraction. pmSubtractionSolveEquation: parameter out-of-range.
+This error is given when the two images are blanks. Overlaps aren't calculated exactly, so sometimes we get skycells that don't have a real overlap when we come to do the warp. They are just created as completely blank fields. So I went to ds9 to find those that were blanks and just ignored them.
+
+== Stacking ==
+
+ * Skycell identifiers don't match
+This came up because there was a bug in IPP 2.1 (but fixed in the current version), so I just copied over some stack config files from Paul Price and rebuilt ippTools.
+
+Everything finished...but the stacked images look only partial. Paul reassured me that this is normal because if my image only happens to overlap a particular skycell a little bit, then I will only get little bits of the image and the rest will be blank.