warptool complains about missing PSASTRO.OUTPUT.MEF within metadata AND FILERRULES

[jester@…]

I'm trying to run warptool on images which came with their own astrometry, i.e. psastro wasn't run on them. warptool doesn't seem to like that.

Starting script /home/panstarrs/ipp/psconfig/ipp-2.2.lin64/bin/warp_overlap.pl on ipp000

Running -imfile -warp_id 1...
warpInputImfile MULTI

warpInputImfile METADATA

cam_id STR 1
exp_id S64 1
exp_name STR fpC-001056-i1-0192
camera STR SDSS
telescope STR 2.5m
dateobs TAI 1999-10-16T08:36:06Z
tmp_class_id STR fpa
class_id STR fpa
uri STR path://S82IN/fpC-001056-i1-0192.fits
exp_type STR science
filelevel STR FPA
filter STR i
airmass F32 1.3
ra F64 6.15307502570069
decl F64 0
exp_time F32 53.9075
sat_pixel_frac F32 0
bg F64 1309.571
bg_stdev F64 9.730103
bg_mean_stdev F64 0
alt F64 28.937027
az F64 290.92166
ccd_temp F32 0
posang F64 38.2970008850098
user_1 F64 0
user_2 F64 0
user_3 F64 0
user_4 F64 0
user_5 F64 0
object STR 82 S
fault S16 0
chip_path_base STR path://S82IN/fpC-001056-i1-0192.1.ch.1
cam_path_base STR /data/ipp001.0/jester/stripe82/coadd/input/fpC-001056-i1-0192.1/fpC-001056-i1-0192.1.cm.1

END

* stderr *
tessellation: Skycells, /data/ipp001.0/jester/stripe82/coadd/skycells
Unable to find PSASTRO.OUTPUT.MEF within metadata.

at /home/panstarrs/ipp/psconfig/ipp-2.2.lin64/bin/warp_overlap.pl line 104

Can't find PSASTRO.OUTPUT.MEF within FILERULES in camera configuration.

at /home/panstarrs/ipp/psconfig/ipp-2.2.lin64/bin/warp_overlap.pl line 104

Use of uninitialized value in concatenation (.) or string at /home/panstarrs/ipp/psconfig/ipp-2.2.lin64/bin/warp_overlap.pl line 108.
psastroRootFile:
Use of uninitialized value in pattern match (m) at /home/panstarrs/ipp/psconfig/ipp-2.2.lin64/lib/PS/IPP/Config.pm line 271.

failure for: warp_overlap.pl --warp_id 1 --camera SDSS --workdir path://S82/run_ipp/warp/

[Paul Price]

dvoImageOverlaps, which calculates the overlaps for the warps, requires the astrometry file. As far as I know (I just tried it), it cannot be run on an image WCS. You'll need to turn on the astrometry for your sdss data.

[jester@…]

Gene has changed the call to dvoImageOverlap in warp_overlap.pl line 111 to include -accept-astrom, i.e. it should be fine to run on just a WCS. However, before it gets there, it barfs about the missing PSASTRO.OUTPUT.MEF file on line 104...

Should -accept-astrom be an option that I pass to warptool, so that it only gets turned on when I explictly want it to be on? Then warp_overlap.pl would have to check if it's given or not, and complain about missing psastro files only if accept-astrom wasn't given.

[jester@…]

So in fact, I think dvoImageOverlap *did* run, and probably even successfully. At least there's no message that dvoImageOverlap failed, but IPP lists them as failures in warp.exp.run because warp_overlap.pl is throwing errors.

[Paul Price]

I've added the pieces to the ippScripts (with additions to ippTools and ippconfig as well) required to make this work. The PSWARP recipe should contain an entry "ASTROM.SOURCE" of type STR that is the name of the file rule for the astrometry reference file. If this is NULL, then warp_overlap.pl (and warp_skycell.pl) will attempt to use the existing WCS in the image as the astrometric source.

Looks like this part is working. However, dvoImageOverlaps dies, lacking the photcode and number of stars. It seems to me that these are unnecessary for this operation, and these requirements should be removed. I leave this to Gene.

price@alala:/home/panstarrs/price>dvoImageOverlaps -D CATDIR /data/ipp001.0/jester/stripe82/coadd/skycells /data/ipp001.0/jester/stripe82/coadd/run_ipp/fpC-005924-i1-0379.50/fpC-005924-i1-0379.50.ch.50.chip.fits -accept-astrom
photcode not supplied in header
skipping /data/ipp001.0/jester/stripe82/coadd/run_ipp/fpC-005924-i1-0379.50/fpC-005924-i1-0379.50.ch.50.chip.fits
ERROR: no valid image data in /data/ipp001.0/jester/stripe82/coadd/run_ipp/fpC-005924-i1-0379.50/fpC-005924-i1-0379.50.ch.50.chip.fits, giving up
price@alala:/home/panstarrs/price>dvoImageOverlaps -D CATDIR /data/ipp001.0/jester/stripe82/coadd/skycells /data/ipp001.0/jester/stripe82/coadd/input/fpC-004247-i1-0063.fits -accept-astrom -D CAMERA sdss
photcode not supplied in header
skipping /data/ipp001.0/jester/stripe82/coadd/input/fpC-004247-i1-0063.fits
ERROR: no valid image data in /data/ipp001.0/jester/stripe82/coadd/input/fpC-004247-i1-0063.fits, giving up

[Copied an image to own version so I can edit the headers]

price@alala:/home/panstarrs/price/temp>sethead fpC-005924-i1-0379.50.ch.50.chip.fits PHOTCODE='i'
price@alala:/home/panstarrs/price/temp>dvoImageOverlaps -D CATDIR /data/ipp001.0/jester/stripe82/coadd/skycells fpC-005924-i1-0379.50.ch.50.chip.fits -accept-astrom
can't get NSTARS from header
skipping fpC-005924-i1-0379.50.ch.50.chip.fits
ERROR: no valid image data in fpC-005924-i1-0379.50.ch.50.chip.fits, giving up
price@alala:/home/panstarrs/price/temp>sethead fpC-005924-i1-0379.50.ch.50.chip.fits PHOTCODE='i' NSTARS=0
price@alala:/home/panstarrs/price/temp>dvoImageOverlaps -D CATDIR /data/ipp001.0/jester/stripe82/coadd/skycells fpC-005924-i1-0379.50.ch.50.chip.fits -accept-astrom
fpC-005924-i1-0379.50.ch.50.chip.fits : skycell.0484666
fpC-005924-i1-0379.50.ch.50.chip.fits : skycell.0484667
fpC-005924-i1-0379.50.ch.50.chip.fits : skycell.0484668

[Paul Price]

OK, Gene has fixed that problem in DVO.

I've added the ASTROM.ACCEPT boolean option to the PSWARP recipe, which turns on/off the -accept-astrom flag being passed to dvoImageOverlaps.

We should be all done on this.

[jester@…]

Where do I need to put the updated pswarp.config?

I copied the original pswarp recipe from
$PSCONFDIR/$IPPVERS.$ARCH/share/ippconfig/recipes/pswarp.config
to my sdss config directory, but that doesn't seem to be the right place.

[Paul Price]

Since this is something that you want to set per camera (as opposed to globally), it should go in your camera directory. Then you need to tell the system that it's there by adding a PSWARP line in the RECIPES in your camera configuration.

[jester@…]

ok, added the recipe in the right place. New error:

Running -camera SDSS -dump-recipe PSWARP -...
* stderr *

psMetadataAddItem (psMetadata.c:671) : Existing item ASTROM.ACCEPT does not match type (10000) for item requiring matching type (1301)

Error copying ASTROM.ACCEPT

p_psMetadataUpdate (psMetadata.c:523) : failed to update metadata
loadRecipeCamera (pmConfigRecipes.c:383) : Failed to update recipe for PSWARP from camera recipe
pmConfigReadRecipes (pmConfigRecipes.c:82) : Failed to read recipes from camera config
pmConfigRead (pmConfig.c:656) : Failed to read recipes from camera file

Unable to perform ppConfigDump: 3 at /home/panstarrs/ipp/psconfig/ipp-2.2.lin64/bin/warp_overlap.pl line 211.

config error for: warp_overlap.pl --warp_id 2 --camera SDSS --workdir /data/ipp001.0/jester/stripe82/coadd/run_ipp/
job exit status: 3
job host: ipp002
job dtime: 0.83328
stop

This is in sdss/camera.config:

# Recipe options
RECIPES METADATA

PSPHOT STR sdss/psphot.config
PSASTRO STR sdss/psastro.config
PPIMAGE STR sdss/ppImage.config
PPMERGE STR sdss/ppMerge.config
PSWARP STR sdss/pswarp.config
REJECTIONS STR sdss/rejections.config

END

and pswarp.config has

GRID.NX S32 128
GRID.NY S32 128
INTERPOLATION.MODE STR LANCZOS3
MASK.IN STR BAD,SAT
MASK.POOR STR SUSPECT
MASK.BAD STR BLANK
POOR.FRAC F32 0.01
ASTROM.SOURCE STR NULL
ASTROM.ACCEPT BOOL TRUE

[Paul Price]

Whoops, I had the wrong type for ASTROM.ACCEPT in the recipes/pswarp.config

[jester@…]

If you mean just BOOL vs STR, I had fixed that (before, it was saying that it couldn't find ASTROM.ACCEPT of type BOOL in config file; now the error is different) so there is another error.

Well in fact, right now my pantask segfaults, so I can't double-check.

[Paul Price]

This was a bug in our Perl metadata config file parser (PS::IPP::Metadata::Config). It would only recognise values of T or F. Changed in CVS head to allow it to recognise TRUE, FALSE, T or F.