IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links
wiki:ippToPsps_Detections

Back to ippToPsps main page

Detection Batches

The input for the detection batch is one IPP camera-stage smf file for a given exposure, as well as an associated DVO database from which to retrieve object, and other, IDs. One FITS file is generated for each exposure.

PSPS tables

1 Primary extension
1 FrameMeta extension
1 ImageMeta extension per chip
1 Detection extension per chip
1 SkinnyObject extension per chip
1 ObjectCalColor extension per chip

The full table definitions can be seen here

The 'object ID' is featured in the last three tables, and must remain unique across the exposure (it is generated within DVO). In the merged PSPS database, the primary key on the detections table is both the object ID and detection ID, meaning the the same object can appear in multiple, overlapping exposures as they will have different detection IDs.

Current IPP to PSPS mappings

The following table details the current mapping between IPP and PSPS. The source of smf files queued by ippToPsps is a list of cam_ids that are loaded into the relevant DVO database. These cam_ids are then used to obtain metadata from the IPP GPC1 database, as well as the location of the smfs themselves.

FrameMeta
ODM attribute units Data type Default Description IPP source IPP variable/notes
frameIDdimensionlessINT0unique exposure/frame identifier. GPC1 database SELECT exp_id FROM camRun JOIN chipRun USING(chip_id) JOIN rawExp USING(exp_id) WHERE camRun.cam_id = cam_id
frameNamedimensionlessVARCHAR(32) frame name provided by camera software GPC1 database SELECT exp_name FROM camRun JOIN chipRun USING(chip_id) JOIN rawExp USING(exp_id) WHERE camRun.cam_id = cam_id
surveyIDdimensionlessTINYINT0survey flag identifier GPC1 database SELECT camRun.dist_group FROM camRun JOIN chipRun USING(chip_id) JOIN rawExp USING(exp_id) WHERE camRun.cam_id = cam_id. The IPP dist_group is then used to get the surveyID from the PSPS 'Survey' table here
filterIDdimensionlessTINYINT0filter identifier smf primary header Single character filter (grizy) is used to get PSPS ID from Filter table here
cameraIDdimensionlessSMALLINT0camera identifier NA hardcoded to 1
cameraConfigIDdimensionlessSMALLINT0camera configuration identifier NA hardcoded to 1
telescopeIDdimensionlessSMALLINT0telescope identifier NA hardcoded to 1
analysisVeralphanumericVARCHAR(MAX) IPP software analysis release GPC1 database camRun.software_ver for this cam_id
p1RecipalphanumericVARCHAR(MAX) IPP phase 1 MD5 ChecksumNA empty
p2RecipalphanumericVARCHAR(MAX) IPP phase 2 MD5 Checksum NA empty
p3RecipalphanumericVARCHAR(MAX) IPP phase 3 MD5 Checksum NA empty
nOTAdimensionlessSMALLINT-999number of valid OTA/CCD images in this exposure smf file count of psf extensions successfully read from the smf file
photoScatmagnitudesREAL-999 global photometric scatter smf primary header ZPT_ERR
numPhotoRefdimensionlessINT-999number of photometric reference sources smf XYnn.hdr extensions sum of NASTRO over all OTAs
expStartdaysFLOAT-999 exposure start time in MJDsmf primary header MJD-OBS
expTimesecondsREAL-999 exposure time smf primary header EXPREQ
airmassdimensionlessREAL-999airmass at mid-exposure (check - might be at start)smf primary header AIRMASS
raBoredegreesFLOAT-999 RA of telescope boresight - in future this needs work smf primary header RA
decBoredegreesFLOAT-999 DEC of telescope boresight smf primary header DEC
ctype1alphanumericVARCHAR(MAX) name of astrometric projection in RAsmf primary header CTYPE1
ctype2alphanumericVARCHAR(MAX) name of astrometric projection in DECsmf primary header CTYPE2
crval1degreesFLOAT-999 RA corresponding to reference pixelsmf primary header CRVAL1
crval2degreesFLOAT-999 DEC corresponding to reference pixelsmf primary header CRVAL2
crpix1dimensionlessFLOAT-999reference pixel value for RA smf primary headerCRPIX1
crpix2dimensionlessFLOAT-999reference pixel value for DEC smf primary header CRPIX2
cdelt1degrees/pixelFLOAT-999scale factor for RAsmf primary header CDELT1
cdelt2degrees/pixelFLOAT-999scale factor for DECsmf primary header CDELT2
pc001001dimensionless FLOAT-999elements of rotation/Dcale matrix smf primary header PC001001
pc001002dimensionlessFLOAT-999elements of rotation/Dcale matrix smf primary header PC001002
pc002001dimensionlessFLOAT-999elements of rotation/Dcale matrix smf primary headerPC002001
pc002002dimensionlessFLOAT-999elements of rotation/Dcale matrixsmf primary header PC002002
polyOrderdimensionlessTINYINT255polynomial order of astrometry fitsmf primary header NPLYTERM
pca1x3y0dimensionlessFLOAT-999polynomial coefficients for the astrometric fitsmf primary header PCA1X3Y0
pca1x2y1dimensionless FLOAT-999polynomial coefficients for the astrometric fit smf primary header PCA1X2Y1
pca1x1y2dimensionlessFLOAT-999polynomial coefficients for the astrometric fitsmf primary header PCA1X1Y2
pca1x0y3dimensionlessFLOAT-999polynomial coefficients for the astrometric fit smf primary headerPCA1X0Y3
pca1x2y0dimensionlessFLOAT-999polynomial coefficients for the astrometric fitsmf primary header PCA1X2Y0
pca1x1y1dimensionlessFLOAT-999polynomial coefficients for the astrometric fitsmf primary header PCA1X1Y1
pca1x0y2dimensionlessFLOAT-999polynomial coefficients for the astrometric fitsmf primary header PCA1X0Y2
pca2x3y0dimensionlessFLOAT-999polynomial coefficients for the astrometric fitsmf primary header PCA2X3Y0
pca2x2y1dimensionlessFLOAT-999polynomial coefficients for the astrometric fit smf primary header PCA2X2Y1
pca2x1y2dimensionlessFLOAT-999polynomial coefficients for the astrometric fit smf primary headerPCA2X1Y2
pca2x0y3dimensionlessFLOAT-999polynomial coefficients for the astrometric fit smf primary headerPCA2X0Y3
pca2x2y0dimensionlessFLOAT-999polynomial coefficients for the astrometric fitsmf primary header PCA2X2Y0
pca2x1y1dimensionlessFLOAT-999polynomial coefficients for the astrometric fit smf primary header PCA2X1Y1
pca2x0y2dimensionlessFLOAT-999polynomial coefficients for the astrometric fit smf primary headerPCA2X0Y2
calibModNumdimensionlessSMALLINT0calibration modification numberNA hardcoded to 1
batchIDdimensionlessINT320batchIDNA hardcoded to 1
dataReleasedimensionlessTINYINT0Data release version number NA hardcoded to 1
ImageMeta
ODM attribute units Data type Default Description IPP source IPP variable/notes
imageIDdimensionlessBIGINT0unique ID for each image, hashed from frameID and ccdID (ALEX) GPC1 and smf extension name frameID (below) mashed together with OTA number
frameIDdimensionlessINT0frame identifier, also known as exposure number GPC1 database SELECT exp_id FROM camRun JOIN chipRun USING(chip_id) JOIN rawExp USING(exp_id) WHERE camRun.cam_id = cam_id
ccdIDdimensionlessSMALLINT0ccd cell identifier in the focal plane smf OTA number
photoCalIDdimensionlessINT0photometry reduction code identifier DVO image->photcode from DVO Image structure
filterIDdimensionlessTINYINT0filter ID smf primary header same as for FrameMeta (see above)
biasADUREAL-999 detector bias level GPC1 database rawExp.bg
biasScatADUREAL-999 scatter in bias levelGPC1 database rawExp.bg_stdev
skyADU/pixelREAL-999 mean sky level smf XYnn.hdr extension MSKY_MN
skyScatADU/pixelREAL-999 scatter in mean sky level smf XYnn.hdr extension MSKY_SIG
nDetectdimensionlessINT-999number of detections on CCD smf XYnn.psf extension count of detections retrieved from the psf extension
magSatmagnitudeREAL-999 saturation magnitude level smf XYnn.hdr extension FSATUR
completMagmagnitudeREAL-999 95% completion level in mag smf XYnn.hdr extension FLIMIT
astroScat arcsecREAL-999 astrometric scatter for chipsmf XYnn.hdr extension CERROR
photoScatmagnitudeREAL-999photometric scatter for chip smf primary header ZPT_OBS
numAstroRefdimensionlessINT-999number of astrometric reference sourcessmf XYnn.hdr extension NASTRO
numPhotoRefdimensionlessINT-999number of photometric reference sourcessmf XYnn.hdr extension NASTRO
nxpixelsSMALLINT-999chip dimension in xsmf XYnn.hdr extension CNAXIS1
nypixelsSMALLINT-999chip dimension in ysmf XYnn.hdr extension CNAXIS2
psfModelIDdimensionlessINT-999PSF model identifier NA empty CZW: Available from config as PSF_MODEL = PS_MODEL_PS1_V1
psfFwhmraw pixelsREAL-999 model psf full width at half maximum at chip center - raw pixels are approx 0.256 arcsec smf XYnn.hdr extension (FWHM_MAJ + FWHM_MIN) / 2.0
psfWidMajorraw pixelsREAL-999 model PSF parameters at chip centersmf XYnn.hdr extension FWHM_MAJ
psfWidMinorraw pixelsREAL-999 model PSF parameters at chip center smf XYnn.hdr extension FWHM_MIN
psfThetadegreesREAL-999 model PSF parameters at chip center smf XYnn.hdr extension ANGLE
momentFwhmraw pixelsREAL-999 average PSF full width at half maximum from moments smf XYnn.hdr extension (IQ_FW1 + IQ_FW2) / 2.0
momentWidMajorraw pixelsREAL-999 average PSF parameters from moments smf XYnn.hdr extensionIQ_FW1
momentWidMinorraw pixelsREAL-999 average PSF parameters from moments smf XYnn.hdr extension IQ_FW2
apResidmagnitudesREAL-999corrected aperture residualsmf XYnn.hdr extension APMIFIT
dapResidmagnitudesREAL-999scatter of aperture correctionssmf XYnn.hdr extension DAPMIFIT
detectorIDalphnumericVARCHAR(32) identifier for actual CCD chipsmf XYnn.hdr extension DETECTOR
qaFlagsdimensionlessBIGINT-999Q/A flags for this OTA DVO image->flags from DVO Image structure
detrend1alphanumericVARCHAR(MAX) identifier of detrend image 1 smf XYnn.hdr extension DETREND.MASK
detrend2alphanumeric VARCHAR(MAX) identifier of detrend image 2smf XYnn.hdr extension DETREND.DARK
detrend3alphanumericVARCHAR(MAX) identifier of detrend image 3smf XYnn.hdr extensionDETREND.FLAT
detrend4alphanumericVARCHAR(MAX) identifier of detrend image 4 NA empty CZW: can we add DETREND.FRINGE
detrend5alphanumericVARCHAR(MAX) identifier of detrend image 5 NAempty CZW: can we add DETREND.LINEARITY
detrend6alphanumericVARCHAR(MAX) identifier of detrend image 6 NAempty
detrend7alphanumericVARCHAR(MAX) identifier of detrend image 7 NA empty
detrend8alphanumericVARCHAR(MAX) identifier of detrend image 8 NAempty
photoZeromagnitudeREAL-999 local derived photometric zero point smf primary header ZPT_OBS
ctype1alphanumericVARCHAR(MAX) name of astrometric projection in RA smf XYnn.hdr extension CTYPE1
ctype2alphanumericVARCHAR(MAX) name of astrometric projection in DEC smf XYnn.hdr extensionCTYPE2
crval1degreesFLOAT-999 RA corresponding to reference pixelsmf XYnn.hdr extension CRVAL1
crval2degreesFLOAT-999 DEC corresponding to reference pixelsmf XYnn.hdr extension CRVAL2
crpix1pixelsFLOAT-999 reference pixel value for RAsmf XYnn.hdr extension CRPIX1
crpix2pixelsFLOAT-999 reference pixel value for DECsmf XYnn.hdr extension CRPIX2
cdelt1degrees/pixelFLOAT-999scale factor for RAsmf XYnn.hdr extension CDELT1
cdelt2degrees/pixelFLOAT-999scale factor for DECsmf XYnn.hdr extension CDELT2
pc001001dimensionslessFLOAT-999elements of rotation/Dcale matrixsmf XYnn.hdr extension PC001001
pc001002dimensionslessFLOAT-999elements of rotation/Dcale matrixsmf XYnn.hdr extension PC001002
pc002001dimensionslessFLOAT-999elements of rotation/Dcale matrixsmf XYnn.hdr extension PC002001
pc002002dimensionslessFLOAT-999elements of rotation/Dcale matrix smf XYnn.hdr extension PC002002
polyOrderdimensionless - ditto TINYINT255polynomial order of astrometry fitsmf XYnn.hdr extension NPLYTERM
pca1x3y0dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA1X3Y0
pca1x2y1dimensionslessFLOAT-999polynomial coefficients for the astrometric fit smf XYnn.hdr extensionPCA1X2Y1
pca1x1y2dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA1X1Y2
pca1x0y3dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA1X0Y3
pca1x2y0dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA1X2Y0
pca1x1y1dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA1X1Y1
pca1x0y2dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA1X0Y2
pca2x3y0dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA2X3Y0
pca2x2y1dimensionslessFLOAT-999polynomial coefficients for the astrometric fit smf XYnn.hdr extensionPCA2X2Y1
pca2x1y2dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA2X1Y2
pca2x0y3dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA2X0Y3
pca2x2y0dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA2X2Y0
pca2x1y1dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA2X1Y1
pca2x0y2dimensionslessFLOAT-999polynomial coefficients for the astrometric fitsmf XYnn.hdr extension PCA2X0Y2
calibModNumdimensionlessSMALLINT0calibration modification number NA hardcoded to 1
dataReleasedimensionlessTINYINT0Data release NA hardcoded to 1
Detection
ODM attribute units Data type Default Description IPP source IPP variable/notes
objIDdimensionlessBIGINT0ODM object identifier DVO ave.extID from dvoDetection structure
detectIDdimensionlessBIGINT0ODM detection identifier DVO meas.extID from dvoDetection structure
ippObjIDdimensionlessBIGINT0object id generated by IPP DVO ave.catID*1000000000 + ave.objID from dvoDetection structure
ippDetectIDdimensionlessBIGINT0detection id generated by IPP smf XYnn.psf extension IPP_IDET
filterIDdimensionlessTINYINT0filter ID: g=1, r=2, i=3, z=4, y=5, w=6, ... smf primary header same as for FrameMeta (see above)
surveyIDdimensionlessTINYINT0survey flag identifier GPC1 same as for FrameMeta (see above)
imageIDdimensionlessBIGINT0image id hash of frame identifier & OTA/CCD number GPC1 and smf extension name same as for ImageMeta (see above)
obsTimeMean Julian DaysFLOAT-999 Time of mid observation smf primary header (MJD-OBS + EXPTIME) / 172800.0
xPosraw pixelsREAL-999 measured x on CCD from PSF fitsmf XYnn.psf extension X_PSF
yPosraw pixelsREAL-999 measured y on CCD from PSF fitsmf XYnn.psf extension Y_PSF
xPosErrraw pixelsREAL-999 estimated error in xsmf XYnn.psf extension X_PSF_SIG
yPosErrraw pixelsREAL-999 estimated error in y smf XYnn.psf extension Y_PSF_SIG
psfFlux adu/secREAL-999 PSF instrumental flux (in analog-to-digital units/sec) or adu/sec smf primary header (EXPTIME) and XYnn.psf extension POW(10.0, (-0.4*PSF_INST_MAG)) / EXPTIME
psfFluxErr adu/secREAL-999 estimated error in flux smf primary header (EXPTIME) and XYnn.psf extension ABS((PSF_INST_MAG_SIG*(POW(10.0, (-0.4*PSF_INST_MAG)) / EXPTIME)) / 1.085736)
peakADUaduREAL-999 peak count in source pixel - where counts are anlog-to-digital units smf primary header (EXPTIME) and XYnn.psf extension POW(10.0, (-0.4*PEAK_FLUX_AS_MAG)) / EXPTIME
psfWidMajorraw pixelsREAL-999 model PSF width in major axissmf XYnn.psf extension PSF_MAJOR
psfWidMinorraw pixelsREAL-999 model PSF width in minor axissmf XYnn.psf extension PSF_MINOR
psfThetadegrees East of NorthREAL-999 model PSF orientation angle smf XYnn.psf extensionPSF_THETA
psfLikelihooddimensionlessREAL-999PSF likelihoodsmf XYnn.psf extension ~erfc(abs(EXT_NSIGMA) / sqrt(2)) ~ sign(EXT_NSIGMA) * erfc(abs(EXT_NSIGMA) / sqrt(2))
psfQf noneREAL-999PSF coverage factorsmf XYnn.psf extension PSF_QF
momentXXraw pixelsREAL-999 moment XXsmf XYnn.psf extension MOMENTS_XX
momentXYraw pixelsREAL-999 moment XYsmf XYnn.psf extension MOMENTS_XY
momentYYraw pixelsREAL-999 momeny YY smf XYnn.psf extensionMOMENTS_YY
apFlux adu/secREAL-999 Aperture Fluxsmf XYnn.psf extension AP_MAG (converted to flux)
apFluxErradu/secREAL-999 Aperture magnitude error NA empty
kronFluxadu/secREAL-999 Kron flux smf XYnn.psf extension KRON_FLUX / expTime convert to adu/s
kronFluxErradu/secREAL-999 Kron flux error smf XYnn.psf extension KRON_FLUX_ERR / expTime convert to adu/s
kronRadraw pixelsREAL-999 Kron radius NA empty
kronRadErrraw pixelsREAL-999 Kron radius error NA empty
psfQfPerfectdimensionlessREAL-999~Likelihood the source is a cosmic ray~ smf XYnn.psf extension ~(EXT_NSIGMA > 0) ? 0 : 1 - psfLikelihood~ psf_qf_perfect
psfChiSqdimensionlessREAL-999~Likelihood the source is extended~ smf XYnn.psf extension ~(EXT_NSIGMA > 0) ? 1 - psfLikelihood : 0~ psf_chisq
infoFlagnoneBIGINT-999flag indicating provenance informationsmf XYnn.psf extension and DVO dvoDetection.meas.dbFlags << 46| FLAGS2 <<32 | FLAGS
sky adu/sec/pixel REAL-999 PSF sky level at source smf XYnn.psf extension SKY
skyErradu/sec/pixel REAL-999 estimated error in sky smf XYnn.psf extension SKY_SIGMA
sgSepdimensionlessREAL-999star/galaxy separator ~smf XYnn.psf extension ~ ~EXT_NSIGMA~ TBD
activeFlagdimensionlessTINYINT-999indicates whether this detection/orphan is still a detection/orphan NA hardcoded to 0
assocDatedimensionlessDATE28881231date object association assigned NA date that batch has been prepared
historyModNumdimensionlessSMALLINT0modification number in the O-D association history NA hardcoded to 0
dataReleasedimensionlessTINYINT0Data release when this detection was originally taken. Recalibrations do not affect this value. NA hardcoded to 0
SkinnyObject
ODM attribute units Data type Default Description IPP source IPP variable/notes
objIDunknownBIGINT0ODM object identifier index DVO same as Detection table - see above
ippObjIDunknownBIGINT0IPP object number DVO same as Detection table - see above
projectionCellIDunknownINT-999projection cell identifier at discovery time NA empty CZW: the skycell_id contains this
dataReleaseunknownTINYINT0Data release to propagate to the object NA hardcoded to 0
surveyIDunknownTINYINT255surveyID GPC1 same as for FrameMeta (see above)
ObjectCalColor
ODM attribute units Data type Default Description IPP source IPP variable/notes
objIDunknownBIGINT0ODM object identifier DVO same as Detection table - see above
ippObjIDunknownBIGINT0ipp object identifier DVO same as Detection table - see above
filterIDunknownTINYINT0filter identifier smf primary header same as for FrameMeta (see above)
calColormagREAL0 ~color adopted for magnitude calculation~ obselete NA empty
calColorErrmagREAL0 ~error in calibrating color~ obselete NA empty
calibModNumunknownSMALLINT0calibration modification number NA hardcoded to 1
dataReleaseunknownTINYINT0Data release when this color calibration was established NA hardcoded to 0
dataReleaseunknownTINYINT0Data release when this color calibration was established NA hardcoded to 0
DetectionCalib - previously all done by psps
ODM attribute units Data type Default Description IPP source IPP variable/notes
objID dimensionless BIGINT 8 NA ODM object identifier dvo
detectID dimensionless BIGINT 8 NA ODM detection identifier dvo
ippObjID dimensionless BIGINT 8 NA IPP object identifierdvo
ippDetectID dimensionless BIGINT 8 NA detection ID generated by IPP dvo
filterID dimensionless TINYINT 1 NA filter identifier dvo
surveyID dimensionless TINYINT 1 NA survey flag identifier dvo
ra degrees FLOAT 8 NA Right Ascensiondvo
dec degrees FLOAT 8 NA No commentdvo
raErr arcsecREAL 4 0.0 Right Ascension error dvo
decErr arcsec REAL 4 NA No comment dvo
zp magnitudes REAL 4 NA zeropoint dvo
zperr magnitudes REAL 4 0.0 zeropoint err dvo
exptime seconds REAL 4 NA exptime dvo
airmassdimensionless REAL 4 0.0 airmass dvo
dataRelease TINYINT 1 dimensionless Data release when this detection was generated.

desirement for re-ordering

collect star galaxy separators ==

after KronRadErr:

sgSep psfLikelihood psfQf psfQfperfect psfChiSq

future requirements for frame metadata

scheduling chunk name

mount (alt, az, timestamp), calibrated (alt, az, timestamp), names of mount model, pointing model, secondary mirror model, primary mirror model, focus model, (l) otis release number

observatory atmospheric pressure, temperature, dew point/humidity

future requirements for image metadata

gain, detection efficiency ....

Last modified 13 years ago Last modified on Feb 15, 2013, 1:54:26 PM
Note: See TracWiki for help on using the wiki.